There are some good notes on JVM tuning from Mike Shaw on Steven Chan’s blog here , here and here and some good Metalink notes at end of this post.
Important thing missing from all these notes (for some one like me who is new to Java) is basics of Garbage Collection, Generation and how to read GC output.
In this post I’ll start with basics of JVM GC (Garbage Collection) and then in next post apply this theory for real time performance issues w.r.t. JVM (11i Java Virtual Machine) .
Garbage – Java object is considered garbage when it can no longer be reached from any pointer in the running program.
Generations – Memory in JVM is managed in terms of generation i.e. Young generation and tenured generation. Memory pool holding object of different ages like young, tenured. If a particular generation fills up, garbage collection occurs.
A. Young generation – Objects are initially allocated in Young generation (most of objects die here). When Young generation fills up, it causes Minor Garbage Collection. Any objects survived after Minor GC (Garbage Collection) are moved to Tenured Generation. Minor Garbage collection is quick as compared to Full/Major GC.
B. Tenured generation – Surviving objects (from young generation) after minor garbage collection are moved to area called tenured generation, When tenured generation fills up it causes major collection (aka Full GC or Full Garbage Collection). Major collection is slow as it involves all live objects.
Garbage Collection (GC) – is program which clears garbage(dead java objects). Garbage Collection work on fundamental principle that majority of java objects die young (quickly after arriving in JVM). There are two kind of Garbage Collection Minor Garbage Collection and Major Garbage Collection (aka Full GC)
Example of Minor GC – 3824.164: [GC 196725K->141181K(209864K), 0.3295949 secs]
Example of Minor GC – 3841.051: [Full GC 150466K->87061K(217032K), 3.2626248 secs]
Pauses: is the time when application becomes unresponsive because garbage collection is occurring.
.
Understanding JVM parameter for 11i
Sizing the generation is very important in tuning JVM GC. Before jumping to Sizing generation (Young and Tenured) lets look at default 11i JVM parameters
In context file($APPL_TOP/ admin/ $CONTEXT_NAME.xml) default entry for JVM is like
<jvm_options oa_var=”s_jvm_options” osd=”Solaris”>-verbose:gc -Xmx512M -Xms128M -XX:MaxPermSize=128M -XX:NewRatio=2-XX:+PrintGCTimeStamps -XX:+UseTLAB </jvm_options>
1. Above line represents JVM (OACoreGroup) size in 11i
2. -Xms128M, means start with 128MB heap size
3. -Xmx512M, means grow JVM heap size upto max size of 512 MB
4. -XX:NewRatio=2 is to control young generation i.e. ratio between young and tenured generation is 1:2 (i.e. if size of young generation is 50 MB then size of tenured generation should be approx. 100MB)
5. -XX:MaxPermSize=128M limit the permanent generation to 128M (permanent generation is part/area in tenured generation)
6. -XX:+UseTLAB represents to use thread-local object allocation
7. There are two more parameters (11i JVM uses default values) -XX:MinHeapFreeRatio=<minimum> & -XX:MaxHeapFreeRatio=<maximum> with default value of 40 & 70 resp. (for Solaris)
If percentage of free space in generation falls below 40%, size of generation will expand and if percentage of free space exceeds 70%, the size of generation will shrunk.
.
Various type of Garbage Collector
From JDK 1.4.2 there are total 4 type of collectors (prior to 1.4.2 it was just one collector i.e. default collector)
1. Default Collector: JDK prior to 1.4.2 uses default collector. If you don’t specify any parameter with JVM default is default collector.
2. ThroughPut Collector : This collector uses parallel version of young generation collector but Tenrured generation is collected in normal way. To set throughput collector use –XX:+UseParallelGC so change
<jvm_options oa_var=”s_jvm_options” osd=”Solaris”>-verbose:gc -Xmx512M -Xms128M -XX:MaxPermSize=128M -XX:NewRatio=2 -XX:+PrintGCTimeStamps -XX:+UseTLAB </jvm_options>
to
<jvm_options oa_var=”s_jvm_options” osd=”Solaris”>-verbose:gc -Xmx512M -Xms128M -XX:MaxPermSize=128M -XX:NewRatio=2 -XX:+PrintGCTimeStamps -XX:+UseTLAB -XX:+UseParallelGC</jvm_options>
3. Concurrent Low Pause Collector : Concurrent Collector is used to collect tenured generation collection concurrently with execution of application. Parallel version of collector is used for young generation. To set Concurrent Low Pause Collector use –XX:+UseConcMarkSweepGC
like
<jvm_options oa_var=”s_jvm_options” osd=”Solaris”>-verbose:gc -Xmx512M -Xms128M -XX:MaxPermSize=128M -XX:NewRatio=2 -XX:+PrintGCTimeStamps -XX:+UseTLAB -XX:+UseConcMarkSweepGC</jvm_options>
4. Incremental low pause collector : This collector collects just portion of tenured generation at each minor garbage collection. To use Incremental low pause collector use
-Xincgc
If you are on JDK 1.4.2 with multi CPU try setting Concurrent Low Pause Collectoras Garbage Collector.
Thumb rule for Grabage Collection/ JVM tuning w.r.t. 11i
1.Stay on latest JVM/JDK version where ever possible (latest certified with 11i is JRE 6, you should be at-least 1.4.2 and higher)
2. For OACoreGroup consider no more than 100 active users per JVM
3. There should NOT be more than 1 active JVM per CPU
4. Try to reduce GC (Garbage Collection) frequency (specially Major/Full GC). Play with various JVM parameters like (-Xmx, -Xms, -XX:MaxPermSize, -XX:NewRatio, -XX:+UseParallelGC/ -XX:+UseConcMarkSweepGC)
5. If you are on JDK 1.4.2 with multiple CPU middle tier, use Concurrent Low Pause Garbage Collector by setting -XX:+UseConcMarkSweepGC with JVM
6. If you are using Oracle Configurator, assign dedicated JVM for configurator requests
7. Try setting JVM max size NOTgreater than 1 GB, (use multiple JVM’s of 512MB or 1024 MB), this is to reduce GC time (more heap size means more time in GC)
8. Minor GC should be occurring at interval long enough to allow many objects to die young (i.e. lot of objects should die between two minor GC).
9. Throughput (which is time NOT spent on GC) is inversely proportion to amount of memory. Higher the memory for JVM, more time for GC meaning low throughput.
10. Unless you have problems with pauses (time when application becomes unresponsive because garbage collection is occurring), try granting as much memory as possible to VM (128 to 512 is good start and fine tune as per load testing results)
.
How to find JDK version used by Apache/Jserv (JVM) in 11i ?
In context file search for parameter like s_jdktop
<JDK_TOP oa_var=”s_jdktop”>/oracle/apps/11i/vis11icomn/util/java/1.4/j2sdk1.4.2_04</JDK_TOP>
Where is JVM log location in 11i ?
$IAS_ORACLE_HOME/ Apache/ Jserv/ logs/ jvm/ OACoreGroup.0.stdout (GC output)
$IAS_ORACLE_HOME/ Apache/ Jserv/ logs/ jvm/ OACoreGroup.0.stderr (JVM Error)
.
How to read GC (JVM stdout) file ?
Example of JVM out file to understand Garbage Collection in 11i
3824.164: [GC 196725K->141181K(209864K), 0.3295949 secs]
3840.734: [GC 207741K->150466K(217032K), 0.3168890 secs]
3841.051: [Full GC 150466K->87061K(217032K), 3.2626248 secs]
3854.717: [GC 155413K->97857K(215568K), 0.2732267 secs]
3874.714: [GC 166209K->109946K(215568K), 0.3498301 secs]
1. Line 1,2 4 and 5 are example of Minor Collection
2. Line 3 (Full GC) is example of Major Collection
3. First entry in each line is time in seconds since JVM started, To find out time between two GC (Garbage Collection) just subtract second entry from first i.e. (3840.734 – 3824.164 = 16.57 seconds)
4. 196725K->141181K in first line indicates combined size of live objects before and after Garbage Collection (GC)
5. (209864K) in first line in parenthesis, represents object after minor collection that aren’t necessarily alive but can’t be reclaimed, either because they are directly alive, or because they are referenced from objects in tenured generation.
6. 0.3295949 secs in first line represents time taken to run minor collection.
7. Full GC in line three represents Full Garbage Collection or Major Collection
.
References
- 362851.1 Guidelines to setup the JVM in Apps E-Business Suite 11i and R12
- 370583.1 Basic troubleshooting of JVM consuming cpu or too many JDBC connections in Apps 11i
- 567647.1 Using Various Garbage Collection Methods For JVM Tuning
- 390031.1 Performance Tuning Forms Listener Servlet In Oracle Applications
- Garbage Collection in Sun JDK
- GC Example
More on tuning Apache and JVM in 11i/R12 coming soon ….
