Memory space manipulating in Java(Section four:Problems Resolving - part four)

4. Fragmentation

4-1. Phenomenon

 Even there are enough space to be allocated, JVM got down with an OutOfMemory Error without generating a core file.

  As it may also depend on usage of your application or the system, the problem does not always can be reappeared.

  So what does the word Fragmentation means to your Java Heap? Concretely, it indicates that there are too many pinned/dosed objects exist in Java Heap, which can not be target of Compaction Action and would sprinkle about the heap in fragments. In such a condition, the trying to allocate space for a big object will end in failure.

  (Fragmentation issues are often also exacerbated due to frequent object allocations of large sizes, e.g. exceeding 1 MB).

• Pinned Objects: are those that cannot be moved during heap compaction because of JNI (Java Native Interface) access to these objects. Also known as Un-Movable Object, which can not be moved out from memory by GC, due to they are referred by program languages (other than Java) via JNI.

• Dosed Objects: Dosed Objects are those that cannot be moved during heap compaction due to references from the thread stacks directly. Normally, they would used as root objects to be scanning to identify all the reachable objects on Mark Phase of GC.

 

4-2. Handling

  Below is a simple approach to solve problems caused by fragmentation.

         

  1. Run your java application with a JVM argument –Xverbose:gc which will enable the output of information about GC activities.

    As an option, you can also set a JVM argument as follows to run your JVM:

-Dibm.dg.trc.print=st_verify (gives the number of pinned and dosed objects)

-Dibm.dg.trc.print=st_compact_verbose (lists the pinned and dosed objects)

This would enable the JVM to output information of pinned/dosed objects in GC activities in the following ways:

<GC(VFY-SUM): pinned=6696(classes=6472/freeclasses=0) dosed=2231

movable=622699 free=644085>

<GC(VFY-SUM): freeblocks: max=1543520 ave=139 (90010352/644085)>

<GC(VFYAC-SUM): freeblocks: max=2647896 ave=98024 (89986240/918)>

Notice that by default information outlined above would be outputted in the section of <GC> in your verbose:gc.

  2. When you see (from verbosegc) that the Java heap has a lot of free space, but the allocation request still fails, it usually points to a fragmented heap.

Here I will give an example to show you how the output of GC activity would be when a fragmentation was caused.

<AF[151]: Allocation Failure. need 33554448 bytes, 61085 ms since last AF>

------------------ (omission)

<GC(153): freed 37680 bytes, 75% free (404986096/536803840), in 2213 ms>

------------------ (omission)

<AF[151]: managing allocation failure, action=6 (404986096/536803840)>

<AF[151]: totally out of heap space>

    You should have noticed in where the problem be caused by seeing the above data.

    1) An AF happened due to the requirement for 33MB cannot be met in this case.

    2) Then a GC activity was invoked and 404MB in the heap become available.

    3) However, the AF still ends up with a failure and finally an out of heap space message was being outputted.

  3. If you have started your JVM with setting the argument -Dibm.dg.trc.print which I have introduced in step 1, check the number of pinned/dosed objects with the outputted information of the latest GC activities, a high number of which indicates a fragmented heap.

  4. There do have several ways for us to solve such a problem.

    Note that the following suggestions might not help avoid fragmentation in all cases.

• Start with a small heap. Set -Xms far lower than -Xmx. It might be appropriate to allow -Xms to default, because the default is a low value.

• Increase the maximum heap size, -Xmx.

• If the application uses JNI, make sure JNI references are properly cleared. All objects being referenced by JNI are pinned and not moved during compaction, contributing significantly to heap fragmentation.