The Maven Daemon project intends to increase the speed of the Maven builds by using techniques known from Takari and Gradle. Its designers focused on optimization and state-of-the-art technologies and methods: GraalVM is used instead of a JVM process, JIT compiled classes are cached, and spawning multiple processes if needed.
The oldest version of Maven that Maven’s history page records is 1.0-beta-2 on 2002-03-30, almost 19 years ago. Maven brought a paradigm change in the Java ecosystem build tools; it shifted from the procedural approach used by Apache Ant to a declarative model. Also, it shifted towards a more conventional way of organising a project, ensuring that the plugins can be reused multiple times.
Almost two decades later, Apache Maven is the leader of JVM’s build tools based on Snyk’s JVM Ecosystem report with a 64% share. Nevertheless, the increasing complexity and size of today’s software projects, together with the need to deploy faster, more often and with minimal or even no service disruption, have made build speed a priority. A study conducted by Gradle shows Maven as being in some cases 100 times slower than builds run with Gradle.
In order to get a better understanding of what Maven Daemon is and how we can benefit from it, InfoQ reached out to Guillaume Nodet, mvnd’s initiator, and Peter Palaga, one of the main contributors, for a round of questions.
InfoQ: Thank you for taking the time to answer some questions for our readers. Can we start by asking you to introduce yourself and describe your day-to-day job and your role in developing mvnd?
Peter Palaga: I am a part of the Red Hat Integration team and I am mostly porting Camel components to Quarkus in the project called Camel Quarkus. Build tools are my hobby.
InfoQ: What was the reasoning behind the creation of mvnd?
Palaga: Building of large source trees (Camel Quarkus has 1200+ Maven modules) is a part of my daily routine. I got excited immediately when I heard that Guillaume had a working prototype of mvnd. Indeed, it saves a lot of time.
Guillaume Nodet: Over the past months, I’ve been working at reducing the build time of the Apache Camel project. At some point, I had to start digging into Maven itself to improve its speed. The idea came out of this work to use a client/server architecture to benefit from caching and JIT.
InfoQ: How many contributors currently work on mvnd? How sustainable is the project? How do you expect the community to grow?
Palaga: Guillaume and I are the main contributors at the moment. We had a few code contributions from the community - the Homebrew package by Michael Simons being the most prominent one. There are around 10 people watching our GitHub repository, some of them being active in discussing issues and new features. I think we are on a good way to build a healthy community.
InfoQ: Why would one use the daemon? What are the speed gains when using it?
Palaga: Saving time is the most compelling reason to use it. The speed gains depend on the scenario. When building one or two modules, the speedup can be seven, 10 or more times compared to stock Maven. Let’s take camel-activemq as an example. It has ~4060 lines of Java code, so it is not big at all. Building that single module with stock Maven (mvn install -DskipTests, having all dependencies cached in the local Maven repository) takes 5.8 sec on my machine.
The first build with mvnd that starts the daemon in the background takes roughly the same time. Each following build is much faster: the second takes just 0.32 sec (18xspeedup), because the daemon is running already, no JARs need to be read from the disk, no Maven core and plugin classes have to be loaded. Everything is in the memory ready to execute the build request. Further builds may become even faster, because JVM’s Just-In-Time (JIT) compiler jumps in and optimizes the code based on the runtime data. In our example, with the fourth build, the time goes down to 0.22 sec (26x speedup).
That was about time savings when building a small set of modules. Now let’s have a look at building large sets of modules. Let’s take Camel Quarkus with its 1242 modules as an example. Building it with stock Maven (mvn clean install -Dquickly, with all dependencies cached) takes 2 m 51 sec. (The -Dquickly flag disables tests, source checks and other stuff that is typically not needed when rebuilding the whole tree).
Already the first build with mvnd (mvnd clean install -Dquickly) is much faster: 45.9 sec (3.7x speedup). This is thanks to the fact that mvnd builds the modules in parallel and thanks to the fact that I happen to own a machine with 24 cores. This particular source tree allows high degree of concurrency, so most of the time, the daemon can utilize all cores to build the individual modules.
As we have seen in the previous example, the second build has to be faster, because the daemon does not need to load its classes again, plus the JIT compiler may have optimized some hot paths. In case of Camel Quarkus, the second build takes 32 sec (5.2x speedup against stock Maven). The third build is a few more seconds faster due to more JIT compilation: 29.7 sec (5.7x speedup).
InfoQ: In which context you would not recommend using the daemon?
Palaga: mvnd is primarily designed for iterative development on a developer workstation. I think it is worth trying as a drop-in replacement of stock Maven. In case of issues, the users are invited to report them in the project. I see little potential for mvnd in the area of continuous integration (CI). Nodet: currently, mvnd is bound to Maven 3.6.3, so if your build really depends on an older version of Maven, this may cause some problems. The best way would be to upgrade to the latest Maven to be able to leverage mvnd.
InfoQ: Currently, mvd is at version 0.1.0. Is it stable enough to be used in production? Or is it still in trial mode?
Palaga: It is pretty stable and usable in the set of scenarios I happen to use it :). Potential users are welcome to try and see how well it will work for them. Judging on the growing numbers of GitHub stars and Twitter followers, I believe, mvnd works well for many.
InfoQ: What was the biggest challenge you faced in developing the daemon?
Palaga: Porting the client side of mvnd to GraalVM was a challenge, but rather a pleasant one. Thanks to that, the mvnd command line tool is a native executable that makes the builds even faster, because no JVM needs to get started on the client side. The native executable starts blazing fast and spends a fraction of memory compared to the traditional JVM. The mvnd client is there to forward the build request to the daemon and present the results. You can see a short recording of mvnd running on 23 cores.
InfoQ: What is the future roadmap? How far are you from a 1.0 version?
Palaga: The current plan is to improve the GUI in small steps, have some proper documentation (a community member has even donated the mvndaemon.org domain to us!) and catalyze the ecosystem of Maven plugins to better support parallel builds. Contributions are welcome, especially getting somebody to prepare a Chocolatey package for Windows users would be nice!
InfoQ: What is the download rate? Do you have any statistics about its usage?
Palaga: This site aggregates some statistics from the GiHub API. So far we used to have a few hundreds of downloads per release. These numbers should cover both direct downloads from GitHub as well downloads through SDKMAN!, Homebrew and asdf package managers.
InfoQ: Is there a question that I should’ve asked you, but I didn’t?
Palaga: Of course there is!
InfoQ: What is the relationship between mvnd and the ASF Maven project?
Palaga: We are formally independent, not underlying any ASF processes. However, we take care to have all our code under Apache License and once we think our code is stable and battle tested enough, we’d like mvnd to become a part of the ASF Maven project.