JEP 428: Structured Concurrency to Simplify Java Multithreaded Programming

JEP 428, Structured Concurrency (Incubator), has been promoted from Proposed to Target to Targeted status for JDK 19. Under the umbrella of Project Loom, this JEP proposes simplifying multithreaded programming by introducing a library to treat multiple tasks running on different threads as an atomic operation. As a result, it will streamline error handling and cancellation, improve reliability, and enhance observability. This is still an incubating API.

This allows developers to organize their concurrency code using the StructuredTaskScope class. It will treat a family of subtasks as a unit. The subtasks will be created on their own threads by forking them individually but then joined as a unit and possibly canceled as a unit; their exceptions or successful results will be aggregated and handled by the parent task. Let’s see an example:

Response handle() throws ExecutionException, InterruptedException {
   try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
       Future<String> user = scope.fork(() -> findUser());
       Future<Integer> order = scope.fork(() -> fetchOrder());

       scope.join();          // Join both forks
       scope.throwIfFailed(); // ... and propagate errors

       // Here, both forks have succeeded, so compose their results
       return new Response(user.resultNow(), order.resultNow());
   }
}

The above handle() method represents a task in a server application. It handles an incoming request by creating two subtasks. Like ExecutorService.submit(), StructuredTaskScope.fork() takes a Callable and returns a Future. Unlike ExecutorService, the returned Future is not joined via Future.get(). This API runs on top of JEP 425, Virtual Threads (Preview), also targeted for JDK 19.

The examples above use the StructuredTaskScope API, so to run them on JDK 19, a developer must add the jdk.incubator.concurrent module, as well as enable preview features to use virtual threads:

Compile the above code as shown in the following command:

javac --release 19 --enable-preview --add-modules jdk.incubator.concurrent Main.java

The same flag is also required to run the program:

java --enable-preview --add-modules jdk.incubator.concurrent Main;

However, one can directly run this using the source code launcher. In that case, the command line would be:

java --source 19 --enable-preview --add-modules jdk.incubator.concurrent Main.java

The jshell option is also available, but requires enabling the preview feature as well:

jshell --enable-preview --add-modules jdk.incubator.concurrent

The benefits structured concurrency brings are numerous. It creates a child-parent relationship between the invoker method and its subtasks. For instance, from the example above, the handle() task is a parent and its subtasks, findUser() and fetchOrder(), are children. As a result, the whole block of code becomes atomic. It ensures observability by demonstrating task hierarchy in the thread dump. It also enables short-circuiting in error handling. If one of the sub-tasks fails, the other tasks will be canceled if not completed. If the parent task’s thread is interrupted before or during the call to join(), both forks will be automatically canceled when the scope exits. These bring clarity to the structure of the concurrent code, and the developer can now reason and follow the code as if they read through as if they are running in a single-threaded environment.

In the early days of programming, the flow of a program was controlled by the pervasive use of the GOTO statement, and it resulted in confusing and spaghetti code which was hard to read and debug. As the programming paradigm matured, the programming community understood that the GOTO statement was evil. In 1969, Donald Knuth, a computer scientist widely known for the book ​​The Art of Computer Programming, defended that programs can be written efficiently without GOTO. Later, structured programming emerged to solve all those shortcomings. Consider the following example:

Response handle() throws IOException {
   String theUser = findUser();
   int theOrder = fetchOrder();
   return new Response(theUser, theOrder);
}

The code above is an example of structured code. In a single-threaded environment, it is executed sequentially when the handle() method is called. The fetchOrder() method does not start before the findUser() method. If the findUser() method fails, the following method invocation will not start at all, and the handle() method implicitly fails, which in turn ensures that the atomic operation is either successful or not successful. It gives us a parent-child relationship between the handle() method and its child method calls, which follows error propagation and gives us a call stack at runtime.

However, this approach and reasoning do not work with our current thread programming model. For example, if we want to write the above code with ExecutorService, the code becomes as follows:

Response handle() throws ExecutionException, InterruptedException {
   Future<String>  user  = executorService.submit(() -> findUser());
   Future<Integer> order = executorService.submit(() -> fetchOrder());
   String theUser  = user.get();   // Join findUser
   int theOrder = order.get();  // Join fetchOrder
   return new Response(theUser, theOrder);
}

The subtasks in ExecutorService run independently, so they can succeed or fail independently. The interruption doesn’t propagate to the sub-tasks even if the parent is interrupted and thus it creates a leaking scenario. It loses the parent relationship. It also makes it difficult to debug as the parent and child tasks appear on the call stack of unrelated threads in the thread dump. Although the code may seem logically structured, it remains in the developer's mind rather than in execution; thus, the concurrent code becomes unstructured.

Observing all these problems with unstructured concurrent code, the term “Structured Concurrency” was coined by Martin Sústrik in his blog post and then popularized by Nathaniel J. Smith in his Notes on structured concurrency article. About structured concurrency, Ron Pressler, consulting member of the technical staff at Oracle and project lead of Project Loom, in an InfoQ podcast, says:

Structured means that if you spawn something, you have to wait for it and join it. And the word structure here is similar to its use in structured programming. And the idea is that the block structure of your code mirrors the runtime behaviour of the program. So just like structured programming gives you that for sequential control flow, structured concurrency does the same for concurrency.

Developers interested in a deep dive into structured concurrency and learning the backstory can listen to the InfoQ Podcast, a YouTube session by Ron Pressler and the Inside Java articles.

Editor's Note

This news story has been updated to reflect the change in status of JEP 428 to Targeted for JDK 19