Azul Systems has today announced the immediate availability of Zing 17.03, with full support for Azul’s new Falcon just-in-time (JIT) compiler based on LLVM and designed to replace the C2 compiler used in prior versions of Zing, as well as in Oracle HotSpot and OpenJDK. Falcon is the first new production JIT compiler available for Java SE since C2 was introduced at JavaOne in 1997.
LLVM is a modern open source compiler framework and the preferred back-end "engine" for many new languages like Swift and Rust. It has great support from the academic community, and a wide range of commercial backers including AMD, Apple, Cray, IBM, Intel, Microsoft, Sony and SAP. This broad support helps it take full advantage of new server hardware and new instruction sets, often taking advantage of new processor technologies such as the vectorization and crypto optimizations in Intel’s Broadwell and Skylake processors, long before support appears in other languages.
Azul CTO Gil Tene told InfoQ that:
Using LLVM provides us with tremendous leverage, as the work of literally hundreds of other people on optimizations and on keeping up with the latest processor capabilities now immediately applies to Java code.
A simple and concrete example of this is how Falcon will already optimize regular Java loops to use the latest/greatest vector instruction capabilities on the latest Intel CPUs (e.g. the AVX2 instruction set on the current crop of Broadwell Intel E5-v4 servers). As AVX2 adds predicated vector operation capabilities, Falcon is able to vectorize predicated operations in loops (e.g. "add all even numbers in this array up") that were not vectorize-able on previous hardware, so the same exact Java classes execute much faster on newer servers. [Current] HotSpot doesn't even try…
But the most important part is that we get this benefit not from a bunch of Azul engineers spending time on vectorizing optimizations and on choosing the right way to match each processor; but we get it from e.g. Intel having invested in and contributed these optimizations to LLVM over the past couple of years. Their contribution applies to C/C++/Clang/Rust just as much as it (now) applies to Java.
Falcon allows us to do some cool JVM-related optimizations that we had not been able to apply with C2. GC-barrier-related optimization are one example of this. Higher level language optimizations are there as well. A concrete example of higher level optimization that we had introduced with Falcon at a dramatically lower investment than it would have taken us with C2, is what we call "truly final" field optimization. [The challenge with instance-final fields is that they might not be "truly final", since reflection and Unsafe can overwrite them, and often do]. Falcon is able to optimize operations that use instance-final fields in Java in ways that previous JITs had not been doing (including hoisting array range checks on final arrays out of loops, for example). This is a feature that has been in prototype-mode development in both HotSpot and Zing over several years (you can find discussions about it in some OpenJDK mailing lists) but has never been productized in a JVM before, partly due to the complexity of getting the JIT compiler to deal with it well.
Given this, it is unsurprising that Falcon is able to outperform C2 on newer hardware. For example, Tene told InfoQ that it is up to 3.5x faster than Oracle HotSpot on specific crypto workload benchmarks, and around 10% faster that Oracle HotSpot on Cassandra benchmarks using Skylake-based servers (Intel Xeon E5-xxxx). It is also considerably faster than previous versions of Zing in low latency environments. Tene told us that one existing Azul Zing customer running trading infrastructure saw a speed-up of around 18-24%.
Azul is the first organization to use LLVM in a production-grade managed runtime. In order to make this work, LLVM has been enhanced with additional functionality from Azul including optimization in the presence of safepoints and GC barriers, the ability to provide JIT code replacement and deoptimization, and speculative runtime optimizations that include implicit null checks, speculative devirtualization and guarded inlining.
As one would expect, Falcon also supports Azul’s ReadyNow technology, introduced in 2014. ReadyNow is designed to obviate the need for "warming-up" the Java Virtual Machine, a practice used by application owners attempting to take advantage of the JIT compiler optimizations after the JVM has had enough time to learn which code to compile to the machine level.
Zing 17.03 is available for immediate download by existing customers from the Zing support portal. The new Zing release is also available for 30-day trials from the Azul website. The single-license annual subscription price for Zing is $3500 USD per server, with significantly lower prices for higher volumes and longer-term subscriptions.
On May 4, Azul CTO Gil Tene will present a webinar entitled "Inside the Falcon Compiler" at 11:00 AM Pacific time. The webinar plus Q&A will run approximately one hour.