Friday, May 29, 2009
Caching in Java + Spring
In our application, we use Spring as IoC container, though set up the Ehcache manager is the easy task but we would like to use it with our Spring beans and exposed it as Osgi service. Spring modules is the good library help us easy in configuration. It provides the consistent abstraction caching to our application, hence we can use our cache facade with other caching libraries such as JBoss cache, JCS, OScache without chaging our application code. You could download the Spring Modules at https://springmodules.dev.java.net/.
Here are the following steps to configure Ehcache in Spring application
Configuring caching via ehcache.xml file
[?xml version="1.0" encoding="UTF-8"?]
[ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="ehcache.xsd"]
[!--
Configure it if you have overflowToDisk or diskPersistent enabled for any cache.
java.io.tmpdir - Default temp file path
--]
[diskStore path="java.io.tmpdir" /]
[!--
Mandatory Default Cache configuration. These settings will be applied to caches
created programmtically using CacheManager.add(String cacheName).
The defaultCache has an implicit name "default" which is a reserved cache name.
--]
[defaultCache maxElementsInMemory="10000" eternal="false"
timeToIdleSeconds="120" timeToLiveSeconds="120" overflowToDisk="true"
diskSpoolBufferSizeMB="30" maxElementsOnDisk="10000000"
diskPersistent="false" diskExpiryThreadIntervalSeconds="120"
memoryStoreEvictionPolicy="LRU" /]
[!--
Sample cache named sampleCache1
This cache contains a maximum in memory of 10000 elements, and will expire
an element if it is idle for more than 5 minutes and lives for more than
10 minutes.
If there are more than 10000 elements it will overflow to the
disk cache, which in this configuration will go to wherever java.io.tmp is
defined on your system. On a standard Linux system this will be /tmp"
--]
[cache name="engroupCache" maxElementsInMemory="10000"
maxElementsOnDisk="10000000" eternal="false" overflowToDisk="true"
diskSpoolBufferSizeMB="20" timeToIdleSeconds="300"
timeToLiveSeconds="600" memoryStoreEvictionPolicy="LFU" /]
[/ehcache]
Declarating Ehcache in Spring context
[bean id="cacheManager"
class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean"]
[property name="configLocation"]
[value]classpath:ehcache.xml[/value]
[/property]
[/bean]
[/pre]
[pre lang="xml" xml:space="preserve" xml:lang="xml"]
[bean id="cacheProviderFacade"
class="org.springmodules.cache.provider.ehcache.EhCacheFacade"]
[property name="cacheManager" ref="cacheManager" /]
[/bean]
Programmatic use
Using ehcache is so simple, a unit test to make the test whether configuration is set up correctly and take the simple put/get could explain all.
@RunWith(EngroupClassRunner.class)
@ContextConfiguration(locations = { "/META-INF/spring/cache-context.xml" })
public class CacheTest {
@Autowired
protected CacheProviderFacade cacheProviderFacade;
@Test
public void testCacheConfiguration() {
Assert.assertNotNull(cacheProviderFacade);
EhCacheCachingModel model = new EhCacheCachingModel();
model.setCacheName("engroupCache");
cacheProviderFacade.putInCache("a", model, new Integer(1));
Integer fromCache = (Integer) cacheProviderFacade.getFromCache("a",
model);
Assert.assertEquals(new Integer(1), fromCache);
}
}
Besides of using ehcache to create the session like variable in our application, we would like to use ehcache to cache SQL statement in iBatis and integrate with Terracotta to complete our distributing caching solution in our future project. I will keep the post on further caching usage in future posts. Keep reading!
Tuesday, May 26, 2009
use maven to generate auto generate schema
change the content inside your srcFile tag and of course convert the [] to the diamond brakets.
[plugin]
[groupId]org.codehaus.mojo[/groupId]
[artifactId]sql-maven-plugin[/artifactId]
[version]1.0[/version]
[configuration]
[driver]${jdbc.driverClassName}[/driver]
[username]${jdbc.username}[/username]
[password]${jdbc.password}[/password]
[url]${jdbc.url}[/url]
[autocommit]true[/autocommit]
[skip]${maven.test.skip}[/skip]
[/configuration]
[executions]
[execution]
[id]create-schema[/id]
[phase]process-test-resources[/phase]
[goals]
[goal]execute[/goal]
[/goals]
[configuration]
[autocommit]true[/autocommit]
[srcFiles]
[srcFile]src/test/resources/${jdbc.groupId}-schema.sql[/srcFile]
[/srcFiles]
[/configuration]
[/execution]
[/executions]
[dependencies]
[dependency]
[groupId]${jdbc.groupId}[/groupId]
[artifactId]${jdbc.artifactId}[/artifactId]
[version]${jdbc.version}[/version]
[/dependency]
[/dependencies]
[/plugin]
Monday, May 25, 2009
Hibernate oneToMany lazy load correctly
1). POJO Bean
1). override equal() and hashCode(). This is very important. I have seen a lots of programmer think these functions are not useful. without override these function, the Hibernate lazy loading will not work correctly.
2). in your @OneToMany and @ManyToOne relation attribute. indicate these are lazy loading. To do that. add fetch=fetchType.lazy inside the bracket eg. @ManyToOne(fetch=fetchType.lazy)
3). in your web.xml file add the following entities
<filter-mapping >
<filter-name>lazyLoadingFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
and
<filter>
<filter-name>lazyLoadingFilter</filter-name>
<filter-class>org.springframework.orm.hibernate3.support.OpenSessionInViewFilter</filter-class>
</filter>
Hibernate enable lazy loading in your web app
added the following
[filter-mapping]
[filter-name]lazyLoadingFilter[/filter-name>
[url-pattern]/*[/url-pattern]
[/filter-mapping]
and
[filter]
[filter-name]lazyLoadingFilter[/filter-name]
[filter-class]org.springframework.orm.hibernate3.support.OpenSessionInViewFilter[/filter-class]
[/filter]
replace [ and ] to > and <
damn blogspot editor
Wednesday, May 20, 2009
How to use OR to rewrite SQL Query
Tuesday, May 19, 2009
DBMS_STAT tables and backup the stat informaiton
to create a stat backup.
login to sqlplus
SQL> begin
2 dbms_stats.create_stat_table(ownname => 'TEST',stattab => 'STAT_TABLE');
3 end;
4 /
analyze information
SQL> BEGIN
2 --DBMS_STATS.delete_table_stats(ownname => 'TEST',tabname => 'A');
3 DBMS_STATS.gather_table_stats(ownname => 'TEST',tabname => 'A');
4 END;
5 /
export analyzed data to stat_table
SQL> BEGIN
2 dbms_stats.export_table_stats(ownname => 'TEST',tabname => 'A',stattab => 'STAT_TABLE');
3 END;
4 /
check how many rows in a table
SQL> select count(*) from stat_table;
delete analyzed data
SQL> BEGIN
2 DBMS_STATS.delete_table_stats(ownname => 'TEST',tabname => 'A');
3 END;
4 /
import analyzed data
SQL> BEGIN
2 DBMS_STATS.import_table_stats(ownname => 'TEST',tabname => 'A',stattab => 'STAT_TABLE');
3 END;
4 /
Done
Oracle show parameters tips and tricks
for instance, if I want to check "db_" parameters, I can use
SQL> show parameter db_;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_16k_cache_size big integer 0
db_2k_cache_size big integer 0
db_32k_cache_size big integer 0
db_4k_cache_size big integer 0
db_8k_cache_size big integer 0
db_block_buffers integer 0
db_block_checking string FALSE
db_block_checksum string TRUE
db_block_size integer 8192
db_cache_advice string ON
db_cache_size big integer 0
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_create_file_dest string
db_create_online_log_dest_1 string
db_create_online_log_dest_2 string
db_create_online_log_dest_3 string
db_create_online_log_dest_4 string
db_create_online_log_dest_5 string
db_domain string
db_file_multiblock_read_count integer 8
db_file_name_convert string
db_files integer 200
db_flashback_retention_target integer 1440
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_keep_cache_size big integer 0
db_name string Ora813
db_recovery_file_dest string /home/oracle/product/10.2.0/flash
_recovery_area
db_recovery_file_dest_size big integer 2G
db_recycle_cache_size big integer 0
db_unique_name string Ora813
db_writer_processes integer 1
dbwr_io_slaves integer 0
rdbms_server_dn string
standby_archive_dest string %ORACLE_HOME%/RDBMS
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
standby_file_management string MANUAL
SQL> spool off;
I have difficulty to remember all these parameters. Oracle have already thought about these and you can use fuzzy to query the parameters.
SQL> show parameter db_c;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_cache_advice string ON
db_cache_size big integer 0
db_create_file_dest string
db_create_online_log_dest_1 string
db_create_online_log_dest_2 string
db_create_online_log_dest_3 string
db_create_online_log_dest_4 string
db_create_online_log_dest_5 string
SQL> spool off;
Oracle lag and Lead
SQL> with a as (select 1 id,'a' name from dual
2 union
3 select 2 id,'b' name from dual
4 union
5 select 3 id,'c' name from dual
6 union
7 select 4 id,'d' name from dual
8 union
9 select 5 id,'e' name from dual
10 )
11 select id,name,lag(id,1,'')over(order by name) from a
12 /
ID NAME LAG(ID,1,'')OVER(ORDERBYNAME)
---------- ---- -----------------------------
1 a
2 b 1
3 c 2
4 d 3
5 e 4
LEAD(EXPRESION,[OFFSET],[DEFAULT])
SQL> with a as (select 1 id,'a' name from dual
2 union
3 select 2 id,'b' name from dual
4 union
5 select 3 id,'c' name from dual
6 union
7 select 4 id,'d' name from dual
8 union
9 select 5 id,'e' name from dual
10 )
11 select id,name,lead(id,1,'')over(order by name) from a
12 /
ID NAME LEAD(ID,1,'')OVER(ORDERBYNAME)
---------- ---- ------------------------------
1 a 2
2 b 3
3 c 4
4 d 5
5 e
Monday, May 18, 2009
Use Maven to manage your Java Project
Oracle Analytic Fucntions
Oracle Analytic Fucntions
The general syntax of analytic function is:
Function(arg1,..., argn) OVER ( [PARTITION BY <...>] [ORDER BY <....>] [] )
is like "ROW " or "RANK "
the usages are:
Types of Analytical Functions
Ranking Functions
This family consists of rank, dense_rank, row_number, ntile,percent_rank and cume_dist.
See also dense rank vs rank vs row_number.
Window Functions
Window Functions consist of all agregate functions (sum,max, avg...)
Reporting Functions
These are queries that ask something like: find the sales that contribute 10 % or more to total sales. The most prominent function for this is ratio_to_report.
Lag/Lead Functions
Lag/Lead functions make it possible to access values in other rows than the current one. The corresponding functions are lag and lead.
The partition clause
An important part in analytic functions is the Partition Claus
Wednesday, May 13, 2009
measuring Oracle cache hit rate
Quick and Easy ways to copy Oracle Schema method 1
the syntax is
Tuesday, May 12, 2009
5 easy steps to create a PDF Document in Java
- Setup the environment first.
Difference between "in (...)" and "= any(...)" queries when using arrays
difference between them and I wonder:
why this works:
select * from table_of_integers where integer_column = any (array[5,6]);
and this doesn't:
select * from table_of_integers where integer_column in (array[5,6]);
Although documentation says:
9.20.4. ANY/SOME
[...]
SOME is a synonym for ANY. IN is equivalent to = ANY.
[...]
I thought that if IN is equivalent to = any, both queries above should work.
after "google is your friend", I am able to find my answers.
the answers are
select * from table_of_integers where integer_column = any (array[5,6]);
This checks if integer_column matches any value in the specified array.
select * from table_of_integers where integer_column in (array[5,6]);
Monday, May 11, 2009
Converting an ext3 filesystem to ext4
since ext4 is backwards compatible, you can mount an ext3 partition as ext4:
where device is the device you want to mount, and mountpoint is where you want to mount it.
The filesystem will not, however, use many of the new features of ext4. You can enable those with the following:
It is important that you then run fsck, to make sure everything is ok:
Appfuse experience
Tips: Alter postgresql.conf to increase PostgreSQL performance
There are several postmaster options that can be set that drastically affect performance, below is a list of the most commonly used and how they effect performance:
- max_connections =
— This option sets the maximum number of database backend to have at any one time. Use this feature to ensure that you do not launch so many backends that you begin swapping to disk and kill the performance of all the children. Depending on your application it may be better to deny the connection entirely rather than degrade the performance of all of the other children. - shared_buffers =
— Editing this option is the simplest way to improve the performance of your database server. The default is pretty low for most modern hardware. General wisdom says that this should be set to roughly 25% of available RAM on the system. Like most of the options I will outline here you will simply need to try them at different levels (both up and down ) and see how well it works on your particular system. Most people find that setting it larger than a third starts to degrade performance. - effective_cache_size =
— This value tells PostgreSQL's optimizer how much memory PostgreSQL has available for caching data and helps in determing whether or not it use an index or not. The larger the value increases the likely hood of using an index. This should be set to the amount of memory allocated to shared_buffers plus the amount of OS cache available. Often this is more than 50% of the total system memory. - work_mem =
— This option is used to control the amount of memory using in sort operations and hash tables. While you may need to increase the amount of memory if you do a ton of sorting in your application, care needs to be taken. This isn't a system wide parameter, but a per operation one. So if a complex query has several sort operations in it it will use multiple work_mem units of memory. Not to mention that multiple backends could be doing this at once. This query can often lead your database server to swap if the value is too large. This option was previously called sort_mem in older versions of PostgreSQL. - max_fsm_pages =
— This option helps to control the free space map. When something is deleted from a table it isn't removed from the disk immediately, it is simply marked as "free" in the free space map. The space can then be reused for any new INSERTs that you do on the table. If your setup has a high rate of DELETEs and INSERTs it may be necessary increase this value to avoid table bloat. - fsync =
— This option determines if all your WAL pages are fsync()'ed to disk before a transactions is committed. Having this on is safer, but can reduce write performance. If fsync is not enabled there is the chance of unrecoverable data corruption. Turn this off at your own risk. - commit_delay =
and commit_siblings = — These options are used in concert to help improve performance by writing out multiple transactions that are committing at once. If there are commit_siblings number of backends active at the instant your transaction is committing then the server waiting commit_delay microseconds to try and commit multiple transactions at once. - random_page_cost =
— random_page_cost controls the way PostgreSQL views non-sequential disk reads. A higher value makes it more likely that a sequential scan will be used over an index scan indicating that your server has very fast disks.
If this is still confusing to you, Revolution Systems does offer a PostgreSQL Tuning Service
Note that many of these options consume shared memory and it will probably be necessary to increase the amount of shared memory allowed on your system to get the most out of these options.
Hardware Issues
Obviously the type and quality of the hardware you use for your database server drastically impacts the performance of your database. Here are a few tips to use when purchasing hardware for your database server (in order of importance):
- RAM — The more RAM you have the more disk cache you will have. This greatly impacts performance considering memory I/O is thousands of times faster than disk I/O.
- Disk types — Obviously fast Ultra-320 SCSI disks are your best option, however high end SATA drives are also very good. With SATA each disk is substantially cheaper and with that you can afford more spindles than with SCSI on the same budget.
- Disk configuration — The optimum configuration is RAID 1+0 with as many disks as possible and with your transaction log (pg_xlog) on a separate disk ( or stripe ) all by itself. RAID 5 is not a very good option for databases unless you have more than 6 disks in your volume. With newer versions of PostgreSQL you can also use the tablespaces option to put different tables, databases, and indexes on different disks to help optimize performance. Such as putting your often used tables on a fast SCSI disk and the less used ones slower IDE or SATA drives.
- CPUs — The more CPUs the better, however if your database does not use many complex functions your money is best spent on more RAM or a better disk subsystem.
In general the more RAM and disk spindles you have in your system the better it will perform. This is because with the extra RAM you will access your disks less. And the extra spindles help spread the reads and writes over multiple disks to increase throughput and to reduce drive head congestion.
Another good idea is to separate your application code and your database server onto different hardware. Not only does this provide more hardware dedicated to the database server, but the operating system's disk cache will contain more PostgreSQL data and not other various application or system data this way.
For example, if you have one web server and one database server you can use a cross-over cable on a separate ethernet interface to handle just the web server to database network traffic to ensure you reduce any possible bottlenecks there. You can also obviously create an entirely different physical network for database traffic if you have multiple servers that access the same database server.
Saturday, May 9, 2009
Writing your own Velocity macro
these are the logics
#if #else #end -- conditional
#foreach #end -- Iteration
#set() assign new variable value
#stop stop execution the following
#evaluate() evaluate a function
#define
#macro
to define your own macro
#macro(invoke $__p_page)
#if($__p_page.startsWith("/"))
#parse($__p_page)
#else
#set($__uri = $resource.this_vm())
#set($__path = $__uri.substring(0, $__uri.lastIndexOf("/")))
#parse("$__path/$__p_page")
#end
#end
to use it.
#invoke( hello.vm)
Friday, May 8, 2009
NC linux command
you can use it for troubleshooting your network apps.
Ford Escape
I like this small suv very much. Compare to my 98 Acura Integra. This car is big and comfy.
will post some pictures when I have time