JAVA Memory Concept

Problem Description

Out Of Memory (OOM) - An application displays Out of Memory errors due to memory exhaustion, either in java heap or native memory.

Memory Leak- Constant memory growth in either java heap or native memory, whichwill eventually end up in out of memory situation. The techniques todebug the memory leak situations are the same as the out of memorysituations.

Background

Java Heap, Native Memory and Process size

Java heap – This is the memory that the JVM uses to allocate java objects. The maximum value of java heap memory is specified using –Xmxflag in the java command line. If the maximum heap size is notspecified, then the limit is decided by the JVM considering factors likethe amount of physical memory in the machine and the amount of freememory available at that moment. It is always recommended to specify themax java heap value.

Native memory –This is the memory that the JVM uses for its own internal operations.The amount of native memory heap that will be used by the JVM depends onthe amount of code generated, threads created, memory used during GCfor keeping java object information and temporary space used during codegeneration, optimization etc.

Ifthere is a third party native module, it could also use the nativememory. For example, native JDBC drivers allocate native memory.

Themax amount of native memory is limited by the virtual process sizelimitation on any given OS and the amount of memory already committedfor the java heap with –Xmxflag. For example, if the application can allocate a total of 3 GB andif the max java heap is 1 GB, then the max possible native memory isapproximately 2 GB.

Process size –Process size will be the sum of the java heap, native memory and thememory occupied by the loaded executables and libraries. On 32 bitoperating systems, the virtual address space of a process can go up to 4GB. Out of this 4 GB, the OS kernel reserves some part for itself(typically 1 – 2 GB). The remaining is available for the application.

Windows – By default, 2 GB is available for the application and 2 GB is reserved for Kernel’suse. However, on some variants of Windows, there is a /3GB switch whichcan be used to change this ratio such that the application gets 3 GB.

RH Linux AS 2.1 – 3 GB is available for the application.

For other operating systems, please refer to the OS documentation for your configuration.

Difference between process address space and physical memory

Eachprocess gets its own address space. In 32 bit operating systems, thisaddress space will range from 0 to 4 GB. This is independent of theavailable RAM or swap space in the machine. The total physical memoryavailable on the machine is the sum of RAM and the swap space availableon that machine. All the running processes share this physical memory.

Thememory address within a process is virtual. The kernel maps thisvirtual address to the physical address. The physical address points to alocation somewhere in the physical memory. At any given time, the sumof all the virtual memory used by the running processes in a machinecannot exceed the total physical memory available on that machine.

Why does the OOM problem occur and What does the JVM do in this situation?

Out of memory in java heap

TheJVM throws java out of memory (java OOM) error if it is not able getmore memory in java heap to allocate more java objects. The JVM cannotallocate more java objects if the java heap is full of live objects andit is not able to expand the java heap anymore.

Inthis situation, the JVM lets the application decide on what to do afterthrowing the java.lang.OutOfMemoryError?. For example, the applicationmay handle this error and decide to shut down itself in a safe way ordecide to run ignoring this error. If the application doesn’thandle this error, then the thread that throws this error will exit(you will not see this thread if you take a java thread dump).

Out of memory in native heap

TheJVM throws native out of memory (native OOM) if it is not able to getany more native memory. This usually happens when the process reachesthe process size limitation on that OS or the machine runs out of RAMand swap space.

Whenthis happens, the JVM handles the native OOM condition, logs a messagesaying that it ran out of native memory or unable to acquire memory andexits. If the JVM or any other loaded module (like libc or a third partymodule) doesn’thandle this native OOM situation, then the OS will send a sigabortsignal to the JVM which will make the JVM exit. Usually, the JVMs willgenerate a core file when it gets a sigabort signal.

Steps to debug the problem

Determine whether it is a Java OOM or Native OOM:

If the stdout/stderr/server log message says that this is a java.lang.OutOfMemoryError?, then this is Java OOM

If the stdout/stderr message says that it failed to acquire memory, then this is a Native OOM

For Java OOM

Reproduce the OOM

OOMalmost always need a very long time to reproduce. Firstly we need find aas simple as possible way to reproduce it. The best one is to reproducethe OOM when run one case repeating.

Whatever, simple and less time is the key. This can also narrow down the bug.

Things can be checked in component

Theapplication might be leaking some java memory constantly, which maycause this problem. Or, the application uses more live objects and itneeds more java heap memory. The following things can be checked in theapplication:

Cachingin the application - If the application caches java objects in memory,then we should make sure that this cache is not growing constantly.There should be a limit for the number of objects in the cache. We cantry reducing this limit to see if it reduces the java heap usage.

Longliving objects - If there are long living objects in the application,then we can try reducing the life of the objects if possible. Forexample, tuning HTTP session timeout will help in reclaiming the idlesession objects faster.

Memory leaks –One example of memory leak is when using database connection pools inapplication server. When using connection pools, the JDBC statement andresultset objects must be explicitly closed in a finally block. This isdue to the fact that calling close() on the connection objects from poolwill simply return the connection back to the pool for re-use and itdoesn’t actually close the connection and the associated statement/resultset objects.

Increase the java heap - We can also try increasing the java heap if possible to see whether that solves the problem.

If none of the above suggestion is applicable

Thenwe need to use a JVMPI (JVM Profiler Interface) based profiler likeOptimizeIt? or Jprob to find out which objects are occupying the javaheap.

JVM issues

Thiscan not occur almost. Anyway, before throwing a java OOM JVM should dofull GC and full compaction. If the GC cycle or compaction do not happen, then this is a JVM bug.

For Native OOM

Memory availability in the machine

If the machine doesn’thave enough RAM and swap space, then the OS will not be able to givemore memory to this process that could also result in out of memory.Make sure that the sum of RAM and swap space in the disk is sufficientto cater to all the running processes in that machine.

Tuning java heap

Ifthe java heap usage is well within the max heap, then reducing the javamax heap will give more native memory to the JVM. This is not asolution but a workaround that can be tried. Since the OS limits theprocess size, we need to strike a balance between the java heap and thenative heap.

Native memory usage by the JVM

Theamount of native memory usage by the JVM is expected to flatten outafter all the classes are loaded and the methods have been called (codegeneration is over). This usually happens within the first few hours formost of the applications. After that, the JVM uses only little nativememory that may be due to run time class loading, code generation due tooptimization etc.

In order to narrow down the problem, try disabling run time optimizations and check whether that makes any difference.

o In case of Jrockit, -Xnoopt flag can be used to disable run time optimizations.

o In case of SUN hotspot JVM, -Xint flag will force the JVM to run in interpreted mode (no code generation).

Ifthe native memory usage continues to grow constantly throughout therun, then this could be a memory leak in the native code.

Third party native modules or JNI code in the application

Checkwhether you are using any third party native module like databasedrivers. These native modules could also allocate native memory and theleak may be from these modules. In order to narrow down the problem, youshould attempt to reproduce the problem without these third partymodules. For example, you can use pure java drivers instead of nativedatabase drivers.

Checkwhether your application uses some JNI code. This could also be causingnative memory leak and you can try to run the application without theJNI code if possible.