Asserting JDK Flight Recorder Events with JfrUnit

Gunnar Morling, open source software engineer at Red Hat, introduced JfrUnit, a new testing utility which may be used to detect performance regressions with JUnit or the Spock Framework. Interpreting performance test results, such as response times, may be difficult as there may be regressions caused by other factors, such as other processes or the network, than the application itself. JfrUnit may be used to test the performance of the application by measuring memory allocation, IO, database queries, or other elements that are application-specific.

JDK Flight Recorder (JFR) collects events from a running application that may be used to diagnose or profile an application. These events can be almost anything, from memory allocation to garbage collection. It’s possible to use the tool directly from the command line, but it’s often used together with JDK Mission Control which provides a GUI and various plugins which can be used in conjunction with JFR. JfrUnit makes it possible to create assertions that verify JFR events from the application.

JfrUnit supports OpenJDK 16 and the dependency is available on Maven Central:

<dependency>
  <groupId>org.moditect.jfrunit</groupId>
  <artifactId>jfrunit</artifactId>
  <version>1.0.0.Alpha1</version>
  <scope>test</scope>
</dependency>

Implementing a JUnit test starts by adding the @JfrEventTest annotation to the unit test class, unless the test is marked with the @QuarkusTest annotation as the Quarkus test framework automatically interacts with the JFR recording. Tests use the @EnableEvent annotation to collect specific events, for instance, garbage collection events. After executing the program logic, the jfrEvents.awaitEvents() method waits for any JFR events from the JVM or the application, before assertions are used to verify that the event occurred:

@JfrEventTest
public class GarbageCollectionTest {
    public JfrEvents jfrEvents = new JfrEvents();

    @Test
    @EnableEvent("jdk.GarbageCollection")
    public void testGarbageCollectionEvent() throws Exception {
        System.gc();

        jfrEvents.awaitEvents();

        assertThat(jfrEvents).contains(event("jdk.GarbageCollection"));
    }
}

Alternatively, the Spock framework can be used to write the same test:

class GarbageCollectionSpec extends Specification {
    JfrEvents jfrEvents = new JfrEvents()

    @EnableEvent('jdk.GarbageCollection')
    def 'Contains a garbage collection Jfr event'() {
        when:
        System.gc()

        then:
        jfrEvents['jdk.GarbageCollection']
    }
}

Apart from verifying if an event occurred, it’s also possible to verify details of an event, such as the duration of a Thread.sleep() method:

@Test
@EnableEvent("jdk.ThreadSleep")
public void testThreadSleepEvent() throws Exception {
    Thread.sleep(42);

    jfrEvents.awaitEvents();

    assertThat(jfrEvents)
            .contains(event("jdk.ThreadSleep")
            .with("time", Duration.ofMillis(42)));
}

JfrUnit allows to create even more complex scenarios. Consider the following example that collects memory allocation events and sums them before asserting that the memory allocation is between certain values:

@Test
@EnableEvent("jdk.ObjectAllocationInNewTLAB")
@EnableEvent("jdk.ObjectAllocationOutsideTLAB")
public void testAllocationEvent() throws Exception {
    String threadName = Thread.currentThread().getName();

    // Application logic which creates objects

    jfrEvents.awaitEvents();
    long sum = jfrEvents.filter(this::isObjectAllocationEvent)
            .filter(event -> event.getThread().getJavaName().equals(threadName))
            .mapToLong(this::getAllocationSize)
            .sum();

    assertThat(sum).isLessThan(43_000_000);
    assertThat(sum).isGreaterThan(42_000_000);
}

private boolean isObjectAllocationEvent(RecordedEvent re) {
    String name = re.getEventType().getName();
    return name.equals("jdk.ObjectAllocationInNewTLAB") ||
            name.equals("jdk.ObjectAllocationOutsideTLAB");
}

private long getAllocationSize(RecordedEvent recordedEvent) {
    return recordedEvent.getEventType().getName()
            .equals("jdk.ObjectAllocationInNewTLAB") ?
            recordedEvent.getLong("tlabSize") :
            recordedEvent.getLong("allocationSize");
}

Enabling multiple events is also possible by using the wildcard character "*", for instance, @EnableEvent("jdk.ObjectAllocation*") may be used to activate all ObjectAllocation events.

To reset the events collected, the jfrEvents.reset() method may be used to make sure only the events after the reset() method are collected. For example, when running several iterations and asserting the results per iteration:

for (int i = 0; i < ITERATIONS; i++) {
    // Application logic

    jfrEvents.awaitEvents();


    // Assertions

    jfrEvents.reset();
}

Frameworks such as Hibernate don’t emit events themselves, but in those cases, the JMC agent may be used to create events. With the JMC agent, SQL query events may be generated, which can then be used to assert the (number of) SQL queries going to the database. This is demonstrated in the Continuous performance regression testing with JfrUnit session and the example is available on Examples for JfrUnit.