Java & Memory in an iAS environment

Everyone who worked with java applications probably heard of the OutOfMemory error. Like the name says, it indicates that the java application reached the limit of available memory and tries to go beyond that limit, with the OutOfMemory as a logical consequence. But why does this error occur? Your 8 CPU, 20GB server has enough resources to serve 50 of these applications simultaneously and still, the error pops-up now and then, crashing your entire application.

Before you add another set of memory modules, let’s take a quick look at the JVM internal memory management.

Automatic Memory Management
Java uses automatic instead of explicit memory management. This means that the developer is only response for the allocation of memory (Object x = new Object();), and the JVM, more specific, the garbage collector automatically cleans up unused objects.

So it is impossible to have memory leaks in my application? Well, not quite, it is not as obvious as in programming languages with explicit memory management but you still have to free your allocated resources in a proper way (ref. DB connections, Streams, …)

The Heap
The memory allocated by the JVM is called the Heap. The heap itself is divided into 3 areas also called ‘generations’.

  • young generation
  • old generation
  • permanent generation

Most objects are initially allocated in the young generation. The old generation contains objects that have survived some number of young generation collections, as well as some large objects that may be allocated directly in the old generation.
The permanent generation holds objects that the JVM finds convenient to have the garbage collector manage, such as objects describing classes and methods, as well as the classes and methods themselves.

The young generation consists of an area called Eden plus two smaller survivor spaces.
Most objects are initially allocated in Eden. (As mentioned, a few large objects may be allocated directly in the old generation.)
The survivor spaces hold objects that have survived at least one young generation collection and have thus been given additional chances to die before being considered “old enough” to be promoted to the old generation. At any given time, one of the survivor spaces holds such objects,
while the other is empty and remains unused until the next collection.

Now that we know that the young generation contains short living objects and the permanent generation contains (very)long living objects
it is easier to pinpoint the OutOfMemory errors.

iAS & memory
When OutOfMemory problems occur in your newly deployed and bugfree application :-) , you can use the Oracle Enterprise Manager (OEM) web interface
to get a first impression about the memory usage of the application.

In the system components view (The view where all the OC4J instances are listed) you can see the Memory usage in MB per OC4J.
The view also shows the total amount of memory in the server as a pie chart at the top of the screen. The pie chart is divided into 3 parts.

  • Application Server Memory Usage
  • Free Memory
  • Other Memory Usage

Be very careful while interpreting the pie chart. Unix and Linux allocate free memory in a buffer. When an application needs extra memory,
the OS can free a part of the allocated buffer and give it to the application.
Due to the allocated buffer, the pie chart shows that almost all of the system memory is allocated in the ‘Other Memory Usage’

On the other hand, there are the log files where OutOfMemory errors are logged.

  • application.log located in the %ORACLE_HOME%/j2ee/%OC4J%/application-deployments/%APPLICATION% directory.
  • Standard output log named OC4J~\{OC4J\}~\{island\}~\{jvm process\}.log (eg. OC4J~OC4J_DEMOS~default_island~1.log) located in the %ORACLE_HOME%/opmn/logs directory

Be sure to check logging just before the OutOfMemory for more details.

Solving the memory problem
The simplest way to solve OutOfMemory errors is to increase the size of the heap. To do this, you can add command line parameters to the java command.

java -Xms256m -Xmx512m

Where -Xms defines the minimum and -Xmx the maximum amount of memory allocated by the JVM.

In the application server you can alter this in the OC4J instance -> server properties -> command line options -> java options field.
Note that the properties affect all OC4J instances in the cluster for that specific OC4J.

Sometimes this does not resolve the problem. In some cases an OutOfMemory error occurs while there is still lots of memory free for the JVM.
Check standard output log files for the following message.

Permanent generation is full…
increase MaxPermSize (current capacity is set to: 67108864 bytes)

As we have seen before, the permanent generation contains long living objects. By default, the JVM allocates 64MB to the permanent generation.
For the most application this will suffice. The size permanent generation must be set apart from the heap size. To do this use the command line:

java -XX:PermSize=64m -XX:MaxPermSize=256m

Where -XX:PermSize=xxx defines the minimum and -XX:MaxPermSize=xxx the maximum amount of memory allocated for the permanent generation.

Always make sure that the -server option is added to your java options. This setting will preconfigure your JVM to run as optimal as possible in a server environment.

If these settings do not work, you should check you application for memory leaks. You can use a profile to detect these leaks… but that will be explained in a future post

Memory Management in the Java HotSpot Virtual Machine (pdf)
Oracle® Application Server 10g Performance Guide