JUnit, a unit testing framework for Java, is one of the most popular libraries used by Java developers. The JUnit team recently announced they've adopted Azure Pipelines for continuous integration (CI).
In a post on Microsoft's Open Source blog, JUnit's Christian Stein described their journey from three unique CI setups to one. JUnit runs on Windows, Linux, and MacOS environments, and previously used Jenkins, Travis CI, and AppVeyor to produce builds. To consolidate and simplify, the JUnit team chose Microsoft's hosted build service.
With Microsoft releasing Azure Pipelines to the public and with a good free plan for open source projects, the JUnit team quickly saw that this service matched all of the technical requirements needed to build, test, and release the JUnit project, under a single hood.
That’s why Azure Pipelines is now the official CI build service for JUnit 5. It performs all of the checks on submitted pull requests, including builds on top of the latest released version of OpenJDK and early access builds of the next OpenJDK, and does so across the three major operating systems: macOS, Linux, and Windows.
Azure Pipelines builds the JUnit project (see the pipeline here), and also updates snapshot releases stored by Sonatype. In the future, the JUnit team may also release stable versions of the library to Maven Central.
Microsoft's Azure Pipelines supports multiple programming languages, and parallel jobs on Linux, MacOS and Windows. They cater to open source projects like JUnit by offering 10 free parallel jobs with unlimited minutes each month. Users not running open source projects get one free job per month with 1,800 minutes before paying $40 for additional jobs. AWS CodePipeline is a similar service, and costs $1.00 per pipeline per month. Google's Cloud Build service offers 120 free build-minutes per day, and charges $0.0034 per minute after that.
InfoQ reached out to Microsoft's Bruno Borges to understand more about the CI migration of this influential Java project.
InfoQ: Today, Azure Pipelines is used to build JUnit, but it looks like releases to the repos are still handled separately?
Bruno Borges: Right now the JUnit team is using it to build, test, and publish Snapshot releases, which currently are hosted on a public Sonatype repository. Next step will be to use it to publish stable/final releases into Maven Central.
InfoQ: To be clear, a single pipeline handles the Windows/Mac/Linux builds, which results in no duplicated logic?
Borges: That is correct. The entire pipeline is defined in two files, where one has the build jobs defined as well conditions between them, given some depend on others, and then on the second file you can find the specific Gradle setup for building the project. This file is configured in a way that works in any platform, so the Gradle build steps can be reused.
InfoQ: What are the primary components of the build jobs? Does it use built-in Azure Pipelines capabilities, or call out to scripts or custom activities?
Borges: We helped the JUnit team migrate with as minimal changes as possible to their current setup. To accomplish that, we focused on using shell scripts mostly, since their project is already configured with Gradle Wrapper. An interesting capability is the support for using Docker containers to build the project inside a build agent (VM). This allows them to test JUnit on OpenJDK 12 very easily, even though this version is not installed in the build agents, and it does not require them to manually install it either. Just point to the docker image and done.
InfoQ: How long did builds take before, versus now? Is there a difference in cost between the previous, and current solutions?
Borges: The JUnit team evaluated the Gradle build scans and found out that the performance is now about 2x-3x faster compared to the previous CI setup. Build jobs also happen in parallel for Linux, Windows, and Mac, which gives them immediate results to verify their project build health.
InfoQ: Is the JUnit team using other parts of Azure, or just Azure DevOps?
Borges: There might be other areas to help with, and we will keep looking, but for now the JUnit team is really happy with Azure Pipelines and so are we with this open source adoption. If other open source projects want to try Azure Pipelines, they are more than welcome to contact me for initial guidance.