Deep Dive into JUnit 5 Extension Model

JUnit, the most popular testing framework on the JVM was completely overhauled in its 5th major release. JUnit 5 is packed with rich features––from improved annotations, tagging, and filtering to conditional test execution and lazy evaluation of assertion messages. This makes writing unit tests in the spirit of TDD a breeze. The new framework also brings in a single, yet powerful extension model. Extension developers can use this new model to add custom features to JUnit 5. This article walks you through the design and implementation of a custom extension. This custom extension provides a way for Java programmers to create and execute stories and behaviors i.e. BDD specification tests.

We begin by exploring a sample story and a behavior (a test method) written using JUnit 5 and our custom extension, conveniently  named "StoryExtension". The example uses two new custom annotations, “@Story” and “@Scenario”, and a new “Scene” class that we are going to design in order to support our custom StoryExtension:

import org.junit.jupiter.api.extension.ExtendWith;

import ud.junit.bdd.ext.Scenario;
import ud.junit.bdd.ext.Scene;
import ud.junit.bdd.ext.Story;
import ud.junit.bdd.ext.StoryExtension; 

@ExtendWith(StoryExtension.class)
@Story(name=“Returns go back to the stockpile”, description=“...“)
public class StoreFrontTest {

    @Scenario(“Refunded items should be returned to the stockpile”)
    public void refundedItemsShouldBeRestocked(Scene scene) {
        scene
            .given(“customer bought a blue sweater”,
                     () -> buySweater(scene, “blue”))

            .and(“I have three blue sweaters in stock”,
                     () -> assertEquals(3, sweaterCount(scene, “blue”),
                             “Store should carry 3 blue sweaters”))

            .when(“the customer returns the blue sweater for a refund”,
                     () -> refund(scene, 1, “blue”))

            .then(“I should have four blue sweaters in stock”,
                     () -> assertEquals(4, sweaterCount(scene, “blue”),
                             “Store should carry 4 blue sweaters”))
            .run();
    }
}

From the code snippet we see that the Jupiter’s extension model is pretty powerful. We can also observe that our custom extension, and its corresponding annotations, provide test writers a way to write simple and clean BDD specifications.

As an added bonus, when tests are executed with our custom extension, text reports like the one shown below are generated:

STORY: Returns go back to the stockpile

As a store owner, in order to keep track of stock, I want to add items back to stock when they’re returned.

SCENARIO: Refunded items should be returned to stock
   GIVEN that a customer previously bought a blue sweater from me
     AND I have three blue sweaters in stock
    WHEN the customer returns the blue sweater for a refund
    THEN I should have four blue sweaters in stock

These reports can serve as live documentation of your application's feature-sets.

The custom extension, StoryExtension, is able to support and execute stories and behaviors with the help of the following core concepts:

1. Annotations to decorate test classes and test methods
2. JUnit 5 Jupiter’s lifecycle callbacks
3. Dynamic parameter resolution

Annotations

The "@ExtendWith" annotation that was seen in the sample story earlier is a markup interface provided by Jupiter. It is a declarative way to register a custom extension on a test class or method. It tells Jupiter’s test engine to invoke the custom extension for the given class or method. Alternatively, a custom extension can be programmatically registered by the test writers or it can be automatically registered (globally) using the service loader mechanism.

Our custom extension needs a way to identify a story. In order to do so, we define a custom annotation class named "Story" that looks like this:

import org.junit.platform.commons.annotation.Testable;

@Testable
public @interface Story {...}

Test writers should use this custom annotation to mark test classes as stories. Notice that the annotation is meta-annotated with JUnit 5's built-in "@Testable" annotation. This annotation gives IDEs and other tools a way to identify classes and methods that are testable––meaning the annotated class or method can be executed by a test engine like JUnit 5 Jupiter test engine.

Our custom extension also needs a way to identify behaviors or scenarios in a story. To do so, we define a custom annotation class named, you guessed it, "Scenario" that looks like this:

import org.junit.jupiter.api.Test;

@Test
public @interface Scenario {...}

Test writers should use this custom annotation to mark test methods as scenarios. The annotation is meta-annotated with JUnit 5 Jupiter's built-in "@Test" annotation. When IDEs and test engines scan through a given set of test classes and find this custom @Scenario annotation on public instance methods, they mark those methods as test methods to be executed.

Note that unlike the JUnit 4 @Test annotation, Jupiter’s @Test annotation does not support the optional “expected” exception and the “timeout” parameters. And unlike JUnit 4 @Test annotation, the Jupiter’s @Test annotation was designed from the ground up with custom extensions in mind.

Lifecycle

JUnit 5 Jupiter provides extension authors callbacks that can be used to tap into the lifecycle events of tests. The extension model provides several interfaces for extending tests at various points in the test execution lifecycle:

Extension authors are free to implement all or some of these lifecycle interfaces.

The “BeforeAllCallback” interface provides a way to initialize an extension and add custom logic before tests within a JUnit test container are invoked. Our StoryExtension class will implement this interface to ensure that a given test class is decorated with the “@Story” annotation.

import org.junit.jupiter.api.extension.BeforeAllCallback;

public class StoryExtension implements BeforeAllCallback {
    @Override
    public void beforeAll(ExtensionContext context) throws Exception {

        if (!AnnotationSupport
               .isAnnotated(context.getRequiredTestClass(), Story.class)) {
            throw new Exception(“Use @Story annotation...“);
        }
    }
}

Jupiter engine will provide an execution context instance under which an extension is to operate. We use this context to determine if the test class under execution is decorated with the required “@Story” annotation. We make use of the AnnotationSupport helper class provided by the JUnit platform to check for the presence of an annotation.

Recall that our custom extension generates BDD reports after executing the tests. Some parts of these reports are pulled from the elements of the “@Store” annotation. We use the beforeAll callback to store these strings. Later, at the end of the execution lifecycle, we retrieve these strings to generate reports. A simple POJO is used for this purpose. We name this class “StoryDetails”. The following code snippet demonstrates the process of creating an instance of this class and save the elements of the annotation into the instance:

public class StoryExtension implements BeforeAllCallback {
    @Override
    public void beforeAll(ExtensionContext context) throws Exception {

        Class<?> clazz = context.getRequiredTestClass();
        Story story = clazz.getAnnotation(Story.class);

        StoryDetails storyDetails = new StoryDetails()
                .setName(story.name())
                .setDescription(story.description())
                .setClassName(clazz.getName());

        context.getStore(NAMESPACE).put(clazz.getName(), storyDetails);
    }
}

The last statement in the method above warrants a more detailed explanation. We are essentially retrieving a named store from the execution context and pushing our newly created “StoryDetails” instance into this store.

A store is a holder that can be used by custom extensions to save and retrieve arbitrary data––basically a super charged in-memory map. In order to avoid accidental key collisions between multiple extensions, the good folk at JUnit introduced the concept of a namespace. A namespace is a way to scope the data saved by extensions. One common method employed to uniquely scope extension data is to use the custom extension class name:

private static final Namespace NAMESPACE = Namespace
            .create(StoryExtension.class);

The other custom annotation that our extension needs is the “@Scenario” annotation. This annotation is used to mark a test method as describing a scenario or behavior within a story. Our extension will parse these scenarios in order to execute them as JUnit tests and generate reports. Recall the “BeforeEachCallback” interface from the lifecycle diagram we saw earlier; we will use the callback to add this additional logic right before each test method is invoked:

import org.junit.jupiter.api.extension.BeforeEachCallback;

public class StoryExtension implements BeforeEachCallback {
    @Override
    public void beforeEach(ExtensionContext context) throws Exception {
      if (!AnnotationSupport.
            isAnnotated(context.getRequiredTestMethod(), Scenario.class)) {
              throw new Exception(“Use @Scenario annotation...“);
      }
    }
}

As mentioned before, the Jupiter engine will provide an execution context instance under which an extension is to operate. We use the context to determine if the test method under execution is decorated with the required “@Scenario” annotation.

Referring back to the beginning of this article where we described a sample story in code, our custom extension is responsible for injecting an instance of the “Scene” class into each test method. The Scene class enables test writers to define scenarios (behaviors) using steps like "given", "then", and "when" that are written as lambda expressions. The Scene class is the central unit of our custom extension that holds test method specific state information. The state information can be passed around between the various steps of a scenario. We use the “BeforeEachCallback” interface to prepare a Scene instance right before the invocation of a test method:As mentioned before, the Jupiter engine will provide an execution context instance under which an extension is to operate. We use the context to determine if the test method under execution is decorated with the required “@Scenario” annotation.

public class StoryExtension implements BeforeEachCallback {
    @Override
    public void beforeEach(ExtensionContext context) throws Exception {
        Scene scene = new Scene()
                .setDescription(getValue(context, Scenario.class));

        Class<?> clazz = context.getRequiredTestClass();

        StoryDetails details = context.getStore(NAMESPACE)
                .get(clazz.getName(), StoryDetails.class);

        details.put(scene.getMethodName(), scene);
    }
}

The code above is pretty similar to what we did in the “BeforeAllCallback” interface method.

Dynamic Parameter Resolution

The piece of the puzzle missing at this point is the ability to inject unique scene instances into test methods. Jupiter’s extension model provides us an interface for this specific purpose. It is called the “ParameterResolver” interface. This interface provides a way for the test engine to identify extensions that wish to dynamically inject parameters during test execution. We need to implement the two methods provided by this interface in order to inject our scene instances:

import org.junit.jupiter.api.extension.ParameterResolver;

public class StoryExtension implements ParameterResolver {
    @Override
    public boolean supportsParameter(ParameterContext parameterContext,
                                     ExtensionContext extensionContext) {
        Parameter parameter = parameterContext.getParameter();

        return Scene.class.equals(parameter.getType());
    }

    @Override
    public Object resolveParameter(ParameterContext parameterContext,
                                   ExtensionContext extensionContext) {
        Class<?> clazz = extensionContext.getRequiredTestClass();

        StoryDetails details = extensionContext.getStore(NAMESPACE)
                .get(clazz.getName(), StoryDetails.class);

        return details.get(extensionContext
                            .getRequiredTestMethod().getName());
    }
}

The first method above tells Jupiter whether or not our custom extension can inject the required parameter used by a test method.

In the second method, “resolveParameter()”, we are retrieving the StoryDetails instance from the execution context’s namespace scoped store. From there, we retrieve the previously created scene instance for the given test method and pass it to the test engine. The test engine will inject this scene instance into the test method and execute the test. Note that the “resolveParameter()” method is invoked only when the “supportsParameter()” method returns a true value.

Finally, in order to generate reports after executing all stories and scenarios, the custom extension hooks into the “AfterAllCallback” interface:

import org.junit.jupiter.api.extension.AfterAllCallback;

public class StoryExtension implements AfterAllCallback {    
    @Override
    public void afterAll(ExtensionContext context) throws Exception {
        
        new StoryWriter(getStoryDetails(context)).write();
    }
}

“StoryWriter” is a custom class that generates reports and saves them into JSON or text files.

With all the key pieces in place, let’s see how we can use this custom extension to write BDD style tests using gradle. Gradle 4.6 and above support running unit tests using JUnit 5. You can use the build.gradle file to configure JUnit 5.

dependencies {
    testCompile group: “ud.junit.bdd”, name: “bdd-junit”,
                version: “0.0.1-SNAPSHOT”

    testCompile group: “org.junit.jupiter”, name: “junit-jupiter-api”,
                version: “5.2.0"
    testRuntime group: “org.junit.jupiter”, name: “junit-jupiter-engine”,
                version: “5.2.0”
}

test {
    useJUnitPlatform()
}

As you can see, using the “useJUnitPlatform()” method we explicitly ask gradle to use JUnit 5. We can then start writing tests using the StoryExtension class. Here’s the sample from the beginning of this article:

import org.junit.jupiter.api.extension.ExtendWith;

import ud.junit.bdd.ext.Scenario;
import ud.junit.bdd.ext.Story;
import ud.junit.bdd.ext.StoryExtension; 

@ExtendWith(StoryExtension.class)
@Story(name=“Returns go back to the stockpile”, description=“...“)
public class StoreFrontTest {

    @Scenario(“Refunded items should be returned to the stockpile”)
    public void refundedItemsShouldBeRestocked(Scene scene) {
        scene
            .given(“customer bought a blue sweater”,
                     () -> buySweater(scene, “blue”))

            .and(“I have three blue sweaters in stock”,
                     () -> assertEquals(3, sweaterCount(scene, “blue”),
                             “Store should carry 3 blue sweaters”))

            .when(“the customer returns the blue sweater for a refund”,
                     () -> refund(scene, 1, “blue”))

            .then(“I should have four blue sweaters in stock”,
                     () -> assertEquals(4, sweaterCount(scene, “blue”),
                             “Store should carry 4 blue sweaters”))
            .run();
    }
}

We can run the tests using “gradle testClasses” or use your favorite IDE that supports JUnit 5. Along with the regular test reports, the custom extension generates BDD documentation for all the test classes that make use of it.

Conclusion

We described JUnit 5 extension model and how it can be leveraged to create custom extensions. In the process, we designed and implemented a custom extension that can be used by test writers to create and execute stories. Head over to GitHub to grab the code and study the custom extension and how it is implemented using the Jupiter extension model and its APIs.

About the Author

Uday Tatiraju is a principal engineer at Oracle with over a decade of experience in ecommerce platforms, search engines, backend systems, and web and mobile programming.