The new release of ByteBuddy, the widely-used Java bytecode engineering library, now fully supports Java 11 and all new classfile and bytecode features introduced since Java 8. This includes the new ConstantDynamic (sometime called condy) feature and Java 11 Nestmates.
InfoQ spoke to Rafael Winterhalter, author of ByteBuddy, to find out more.
InfoQ: Thanks for taking the time to talk with us. Can you start by explaining what Byte Buddy is? What capabilities does it have? What kind of applications is it used for? How long has the project been running? How can developers get started with using it?
Rafael Winterhalter: Byte Buddy is a code generation library that allows to define new classes or to modify existing classes using a simple Java API. For this to happen the library generates and manipulates Java byte code. By working on the byte code level, it can interact with code that is written in any JVM language and the library can be used during a Java application’s runtime to modify the code that is currently executed, even including its own code.
Byte Buddy is mainly used by other libraries and frameworks. For example, Hibernate is using Byte Buddy to implement entity proxies and Mockito is using it to generate mock classes. Increasingly, Byte Buddy is however also used for developing Java agents that change the behavior of entire applications. Using such agents, APM tools such as for example Instana, are using Byte Buddy to collect metrics of applications during runtime.
I started working on Byte Buddy in 2014 and released a first non-beta version in 2015. It has gained quite a lot of traction ever since. As of today, the library is downloaded almost one hundred million times a year.
To get started, all it takes is adding the library to a project and to generate a class using its DSL. The GitHub page and homepage of Byte Buddy both show a simple example of how to create a simple class. To extend on this example, there is comprehensive documentation on the webpage and a lot of material on different blogs and on YouTube and Vimeo.
InfoQ: Let's talk about the new release. What has been added recently? Any new features that users have been specifically asking for?
Winterhalter: In the latest releases I have worked with support for Java 11 and 12. But most of my time went into adding support for the Java module system what was a long road to walk. But with the last releases, support for the module system has become stable and Byte Buddy will even add a module-info.class in its upcoming 1.9.0 release. At the same time, the library retains compatibility to Java 6, 7 and 8 where this class is ignored.
Feature requests typically include support for newer Java versions, as a lack of support often stops projects from even building on newer VMs. Similarly, many feature requests entail support for JVM languages like Kotlin or Scala that show some particularities when it comes to their byte code translation. If the Java language is adding a new feature, this is done very conservatively and often by widening the capabilities of the JVM where it makes sense. Other languages sometimes try to emulate a specific behavior, instead what can cause Byte Buddy to fail a task.
InfoQ: Moving on to the new release cycle - as a toolmaker, how have you been impacted by the faster pace and changes to the classfile format?
Winterhalter: For every Java version, Byte Buddy needs to be adjusted to the changes in the Java class file format. Often these changes are trivial but they can also be fairly complex. For example, when Java 8 was released, Byte Buddy had to adapt to the concept of code in interfaces via default methods. This sounds like a small change but it required a lot of refactoring as the assumption that methods were defined in the class hierarchy was taken for granted at several places within the implementation. With the faster pace of Java releases, my work has unfortunately become a bit more tiresome, but at the end of the day I cannot complain about new features being added to the platform that stands in the center of my professional life.
One problem I face by being so far down the stack is that many libraries depend on Byte Buddy, such that I get asked for support rather quickly after every new major Java release. Without these updates, libraries that depends on Byte Buddy cannot support a new Java version, what thwarts the adoption of other maintainers. At the same time, a lot of tooling like Maven often needs some time to support a new Java version, too, what makes it also difficult for me to move fast. And half a year is over quick. I do however feel like other tool vendors are already adopting to the changes. Supporting Java 10 to Java 12 was fairly easy compared to previous releases, also because less change is bundled into a single update.
InfoQ: Java 11 ships ConstantDynamic as a new feature. What is the significance of this? How does it work? Is it related to InvokeDynamic? If so, how?
Winterhalter: Java developers typically define their constants by storing a value in a static final field. The Java class file format does however allow for class file constants which are not stored in fields but which are referred to by some symbol. Several types already express such symbols as literals; Java strings that are expressed as a literal are for example bound to such symbols in the class file’s constant pool.
With ConstantDynamic it becomes possible to express any instant as a constant value that is referenced in the constant pool. Using class file constants has the major advantage that the constant value only needs to be created when it is used for the first time and not when the class is loaded, as it is the case for static final fields. Using ConstantDynamic, a lot of eager loading in the JVM could be avoided in the future. For example, on any startup of the JVM it must initialize the Locale class which refers to all languages that the JVM supports. This initialization is fairly costly and often unnecessary since most programs only ever use the default language. By using ConstantDynamic, the core libraries could be improved in the future to allow for a faster JVM startup and of course other libraries could do similar things.
As of today, ConstantDynamic is not exposed by the Java language or by any other JVM language. But since its JVM support is fully functional, Byte Buddy is capable of creating such dynamic constants in byte code. Dynamic constants are created by implementing a bootstrap method that returns the constant’s value as its result. This bootstrap method is then referenced at the location where the constant value should be used.
I have blogged in detail about both ConstantDynamic and InvokeDynamic and how to make use of both in Byte Buddy. InvokeDynamic is actually fairly similar where the difference is that it uses a bootstrap method to bind a call site and not a constant value. And of course Byte Buddy offers an API for it, as well.
InfoQ: Have there been any other significant changes to the classfile format since Java 8? How has ByteBuddy responded to the changes?
Winterhalter: Besides ConstantDynamic, the JVM has added the concept of nest mates to provide better method access control for nested classes. By defining two classes as nest mates in their respective class files, they gain the privilege to invoke one another's private methods. Previously, nested classes could only invoke each other's private methods because the javac compiler added package-private accessor methods for any such call. This does of course open the class beyond the intended scope which is why an explicit mechanism is preferable and also easier to understand.
Byte Buddy supports existing nestmates but currently its DSL does not allow to change or add nest mates. This will be a bigger feature which I want to tackle some time this year.
InfoQ: What about the future? What's your take on the upcoming features that are currently in the roadmap?
Winterhalter: At the moment, almost any upcoming Java feature excites me. The feature that I anticipate the most is project Loom which adds native support for continuations to the JVM. In my work as a software consultant I regularly come across projects that attempt concurrency by using abstractions such as an actor model or reactive callbacks. Often these applications grow in complexity over time where business logic gets buried within the ceremonial code that models the concurrency. This makes it sometimes difficult to refactor business code since it gets distributed in ways that follow technical demands instead of domain logic. I hope that Loom makes it unnecessary to explicitly model concurrency throughout business code in most cases.
Besides Loom, I am looking forward to extended support for the Graal compiler which I think is an important cornerstone for the JVM to keep its place as one of the world’s best runtimes. Besides JIT compilation, the capabilities for AOT compilation and native image creation will extend the scope in which Java is used in the future. I also hope that project Metropolis will help to bridge some of the gap between the knowledge of most Java developers and the functioning of the JVM.
InfoQ: Any other comments or thoughts you'd like to share with our readers?
Winterhalter: I often feel like a share of Java developers is sceptical of Oracle’s stewardship where they feel as if features such as the module system or the new release cycle imply more work to them than benefits.
In contrast, I think that Oracle finally took care of many problems that would have jeopardized the long term future of the platform. The JVM is in a better shape then ever and many of the upcoming changes are based on this foundation. It is an exciting time to be a Java developer and we all have a lot to look forward to.
For the interested developer, several videos are available with more information.
