Memory space manipulating in Java(Section two:Process Memory Model on AIX part three-The Large & Very Large Memory Model)

The Large Address-Space Model

Because the system places user data, heap, and stack within the program data segment (segment 0x2), the system limits the maximum amount of stack, heap, and static data to slightly less than 256MB. This size is adequate for most applications.

However, applications with one or more of the following characteristics do not work well with the default memory model:

l         Applications that require large user data (initialized or uninitialized static data) areas in the data section of a program

l         Applications that use malloc, brk, or sbrk subroutines to dynamically allocate large data items in the heap section of a program

l         Applications that create lots of threads, and therefore have to use malloc to dynamically allocate large number of thread stacks

l         Applications that have a deep calling chain or require large stack areas.

(Imaging that your WebSphere server running on a 256MB only environment? Though still decide by your code running on it, maybe you will soon be met with an OutOfMemory error.)

So, as I have talked a little in the default process memory model section, here we need to move down the fence between program data and shared memory, by giving o_maxdata a bigger value.

(You move the boundary by altering the o_maxdata setting in the Executable Common Object File Format (XCOFF) header for an application. You can alter the o_maxdata setting by:

l         Setting the value of o_maxdata at compile time by using the -bmaxdata flag with the ld command.

l         Setting the o_maxdata value by using the LDR_CNTRL=MAXDATA=0xn0000000 (n segments) environment variable.)

What change will be happened to us with a bigger value of o_maxdata? See the following picture to find the answer.

This picture shows the segment usage of a 32-bit process with a o_maxdata value of 0x30000000. In this large address space model:

l         user data and heap move to the three extra segments allocated for program data, or segments 3, 4, and 5.

l         The per process kernel data and user stack remains in segment 2. AIX reserves the three 256 MB segments for user data and heap, starting with segment 3. The initialized and uninitialized data goes into the lower addresses of segment 3, and the process heap continues right after the static data ends, growing towards higher number segments.

l         The number of segments reserved for the process' shared memory is now reduced to 8 segments, instead of 11 as in the default 32-bit address space model. Again, unless the users specify an address in shmat() or mmap() calls, these segments are allocated in order, starting from segment 0x6 towards higher number segments.

With The Large Address-Space Model, you can get a maximum size up to 8 segments (2GB) allocated for your 32-bit applications’ program data, in which case you have to set MAXDATA to a value of 0x80000000. You should be noticed that if o_maxdata is being set as 0x80000000, it means your shared memory will get limited to a size of 3 segments (768MB), pay attention to this as sometimes your application can not run on such condition.

 

Very Large Address-Space Model

From AIX 5.1, it introduces a very large address-space model that supports a more flexible mechanism for 32-bit programs to make maximum use of the available segments, as either program data or shared memory. The very large address-space model allows segments to be dynamically allocated between program data and shared memory. The o_maxdata value only serves as a cap for the program heap, but does not make the system reserve the segments for program heap only.

  In this model, 32-bit application is allowed to grow their data heap up to 13 segments. To enable this new model, there is a new property called dynamic segment allocation (DSA) be introduced, which must be specified with a maxdata value in form of 0xX0000000@DSA. (here X can be varied from 0 to D)

By specifying different value for o_maxdata, 32-bit application which needs either a large program heap or a large shared memory can be satisfied with this model.

The following picture shows a process is running with o_maxdata=0xA0000000@DSA :

The heap of the process can be as big as 10 segments (2.5 GB). Notice the system does not reserve 10 segments (0x3 - 0xC) for program heap. Segment 0x3 is always taken up by program data because of the presence of initialized and unitialized data. The rest of those 10 segments (0x4 - 0xC) are open for dynamic allocation between program heap and shared memory, so the program heap can potentially take 10 segments (0x3 - 0xC). Because o_maxdata is set to 0xA0000000, by putting an upper bound for program heap at segment 0xC the system reserves segment 0xE exclusively for shared memory, which can also potentially take 10 segments (0x4 -0xC, 0xE) and grow to 2.5 GB.

 Next let’s see what will be happened when the process is running with o_maxdata =0xB0000000@DSA:

  

The heap of the process can be as big as 11 segments (2.75 GB). Notice the shared library text and data are not occupying segment 0xD and 0xF anymore, and are moved to segment 0x2. Again, the lower address end of segment 0x3 is first taken up by the initialized and uninitialized data, therefore program heap. All the other 10 segments (0x4 - 0xD) are not reserved for either the program heap or shared memory, but open for competition between the two. However, segments 0xE and 0xF are exclusively reserved for shared memory. In this example when maxdata value is 0xB0000000/dsa, the program heap can potentially take 11 segments (0x3 - 0xD), and the shared memory can also potentially take up 12 segments (0x4 - 0xF) and grow to 3.0 GB.

Finally, you should know this model does allow users to specify o_maxdata value as big as 0xD0000000, in which case the program heap can potentially take 13 segments (3.25 GB).

The following is a summary of the features of the very large address-space model:

l         To enable a program for the very large address-space model, a new property called dynamic segment allocation (DSA) must be specified with a o_maxdata value.

l         When both a non-zero o_maxdata and DSA are specified, segments are dynamically allocated to either the process heap or shared memory. The lower address end of segment 3 is still taken up by static data that occupies as many segments as needed. The remaining data segments, however, are not reserved for user heap, but are obtained dynamically. Until a segment is needed for a program's data area, it can be used by the shmat or mmap subroutines. The maxdata value only serves as an upper bound of the number of segments that the program heap can grow up to. When a process tries to expand its data area into a new segment, the operation succeeds as long as the segment is not being used by shmat or mmap. A program can call the shmdt or munmap subroutine to stop using a segment so that the segment can be used for the data area. After a segment has been used for the data area, however, it can no longer be used for any other purpose, even if the size of the data area is reduced.

 

For the segments to be dynamically allocated between program heap and shared memory,

the operating system uses a different rule for choosing an address to be returned for shared

memory calls (if a specific address is not requested). Normally, the shmat or mmap

subroutines return an address in the lowest available segment. When the very large

address-space model is used, these subroutines will return an address in the highest

available segment. A request for a specific address will succeed, as long as the address is

not in a segment that has already been used for the data area. This behavior is followed for

all processes that specify the DSA property.