At the InfoQ Dev Summit in Boston, Bruno Borges, who has been a principal PM manager at Microsoft for over six years, shared insights on optimizing Java applications on Kubernetes. His session focused primarily on leveraging JVM ergonomics, understanding the impact of CPU throttling, and effectively managing garbage collection processes.
Borges began his interactive session with some puzzles, during which he questioned the audience. He then emphasized that deploying Java applications on Kubernetes may seem straightforward due to YAML configurations and Dockerfiles; however, he asserted challenges arise when optimizing application performance within constrained resources.
He highlighted three main areas for reducing image size – base image layer (using slim versions or distroless images), application layer (including only necessary dependencies) and JVM runtime (shrinking down runtime).
"It's about reducing the surface attack area, it's about reducing components that are shipped in the image and may become a second vector." - Bruno Borges
Borges then moved onto discussing startup time optimization strategies including using class data sharing which can improve startup time significantly. He mentioned ongoing projects like Project Leyden by Oracle and Project CRaC by Azul Systems, which aim to improve startup times further.
"The JVM has something called ergonomics, and almost every language runtime stack has defaults." According to Borges, these are conservative settings designed to work for most applications, but they need tuning depending on specific use cases. For instance, memory allocation is set conservatively, leading to underutilization if not tuned properly. "If you don't tune properly the JVM, you're gonna get really bad configuration," he said.
One key point raised during this discussion was about horizontal scaling versus vertical scaling. While horizontal scaling (adding more nodes) is a common approach, Borges argued that it's not always the most effective solution and can often be just 'throwing money at the problem'. He suggested considering vertical scaling (increasing resources of existing nodes) as an alternative or complementary strategy.
To find out which configuration performs better under real-world conditions, Borges also introduced the concept of A/B performance testing in production, where different configurations are tested against each other. "We see A/B testing all the time, mostly for features in a system, but what if we could do A/B testing for production performance as well?" This could involve changing garbage collector settings, JVM tuning parameters, resource allocations or other knobs. To fine-tune their applications, he also encouraged developers to observe memory, CPU usage, garbage collection and JIT compilation metrics in production environments.
Developers interested in learning more about the InfoQ Dev Summit series can visit the website and stay tuned to InfoQ for the release of the talk recordings in the coming months.