Garbage Collection, like Backgammon takes minutes to learn and a lifetime to master.
In his talk Visualizing Garbage Collection Master trainer/consultant Ben Evans discusses GC from the ground up.
A brief summary of his talk follows.
Basics
When it comes to freeing up memory that is no longer used, GC has largely replaced earlier techniques, such as manual memory management and reference counting.
This is a good thing, as memory management is boring, pedantic bookkeeping that computers excel at whereas people do not. Language runtimes are better at this than humans are.
Modern GC is highly efficient, far more so than manual allocation typical in earlier languages. People from other language backgrounds often focus on GC pauses without fully understanding the context that automatic memory management operates in.
Mark & Sweep is the fundamental algorithm used for GC by Java (and other runtimes).
In the Mark & Sweep algorithm you have references pointing from the frames of each stack's thread, which point into program heap. So we start in the stack, follow pointers to all possible references, and then follow those references, recursively.
When you’re done, you have all live objects, and everything else is garbage.
Note that one point people often miss is that the runtime itself also has a list of pointers to every object called the “allocation list” that is maintained by the garbage collector and helps the garbage collector to clean up. So the runtime can always find an object that it created but has not yet collected.
Figure 1
The stack depicted in the illustration above is just the stack associated with a single application thread; there is a similar stack for each and every application thread, with its own set of pointers into the heap.
If the garbage collector were to attempt to get a snapshot of what’s living while the application was running, then it would be chasing a moving target, and could easily miss some badly timed objects allocations, and could not get an accurate snapshot. Therefore it is necessary to “Stop the World”; i.e. stop the application threads long enough to capture the live object snapshot.
There are two golden rules that the garbage collector must abide by:
- The garbage collector must collect all of the garbage.
- The garbage collector must never collect any live object.
But these rules are not created equal; if rule 2 would be violated, you would end up with corrupted data.
On the other hand, if rule 1 were violated, and instead we had a system that did not collect all of the garbage all of the time, but rather only collected it eventually, then that could be tolerated, and in fact could be the basis of a garbage collector.
HotSpot
Now let’s talk about HotSpot, which is actually a conglomeration of C and C++ as well as a lot of platform-specific assembler.
When people think of an interpreter, they think of a big while-loop with a large switch statement. But the HotSpot interpreter is much more sophisticated than that (for performance reasons). When you start looking at the JDK source code, you realize just how much assembler code is in Hotspot.
Object Creation
In Java we allocate a large contiguous amount of space up front, which is what we know as “the heap”. This memory is then managed, purely in user space, by HotSpot.
If you see a Java process that is using a huge amount of system (or kernel) time, then you can rest assured it is not doing garbage collection - because all of our GC memory bookkeeping is done in user space.
Memory Pools
Figure 2
PermGen is the storage area for things like class metadata, that need to remain alive for the life of the program. However with the advent of application servers that have their own classloaders and need to reload class metadata, PermGen starts looking like a bad optimization decision, which fortunately is going away in Java 8.
A new concept will be used called “Metaspace” which is not exactly the same thing as PermGen. Metaspace is outside the heap, and is managed by the operating system. That means it will be going not into the Java heap, but rather into native memory. Currently this is not such good news because there aren’t many tools that allow you to look easily into native memory. So it is good that PermGen is going away but it is going to take some time until the tooling can catch up.
Java Heap Layout
Now let’s take a look at the Java heap. Notice the Virtual spaces between the heap spaces. These provide a little wiggle to allow some amount of resizing of the pools without suffering the expense of moving everything.
Figure 3
Weak Generational Hypothesis
Now, why do we actually separate the heap into all of these memory pools?
Figure 4
There are runtime facts that cannot be deduced by static analysis. The graph above illustrates that there are two groups of objects; those that die young and those that live for a long time – so it makes sense to do extra bookkeeping to take advantage of that fact. The Java platform is littered with similar facts that have been codified into the platform as optimizations.
Demos
A series of animated demos were performed. The first demo in Flash illustrates the movement between Eden and one of the young gen survivor spaces, and finally into tenured.
Figure 5 presents a JavaFX rendition of the same.
Figure 5
Runtime Switches
'Mandatory' Flags
- -verbose:gc – Get me some GC output
- -Xloggc:<pathtofile> – Path to the log output, make sure you've got disk space
- -XX:+PrintGCDetails – Minimum information for tools to help
– Replace -verbose:gc with this
- -XX:+PrintTenuringDistribution – Premature promotion information
Basic Heap Sizing Flags
- -Xms<size> – Set the minimum size reserved for the heap
- -Xmx<size> – Set the maximum size reserved for the heap
- -XX:MaxPermSize=<size> – Set the maximum size of your perm gen – Good for Spring apps and App servers
In the old days, we were taught to set –Xms to be the same value as –Xmx. However this has changed. So now you can set –Xms to something reasonably small, or just not set it at all, because the heap adaptiveness is now very good.
Other Flags
- -XX:NewRatio=N
- -XX:NewSize=N
- -XX:MaxNewSize=N
- -XX:MaxHeapFreeRatio
- -XX:MinHeapFreeRatio
- -XX:SurvivorRatio=N
- -XX:MaxTenuringThreshold=N
Figure 6
Why Log Files
Log files have the advantage of being able to be used in forensic analysis, which can save you from having to run the code a second time to reproduce the issue (not easy if it's a rare production bug).
They also have more information than the JMX MXBeans for memory, not to mention that polling JMX can introduce its own set of GC problems.
Tooling
- HP JMeter (Google it)
– Free, reasonably solid, but no longer supported / enhanced
– Free, OSS, but a bit ugly
– Best name
– J9 support
– The prettiest and most useful – But we're biased!
In Summary
- You need to understand some basic GC theory
- You want most objects to die young in young gen
- Turn on GC logging! – Reading raw log files is hard – Use a tool
- Use tools to help you tweak – Measure, don't guess
Here you can access the full presentation.
About the Author
Ben Evans is the CEO of jClarity, a Java/JVM performance analysis startup. In his spare time he is one of the leaders of the London Java Community and holds a seat on the Java Community Process Executive Committee. His previous projects include performance testing the Google IPO, financial trading systems, writing award-winning websites for some of the biggest films of the 90s, and others.