Embracing Reactive Applications on JVM: a Deep Dive into Modern I/O Models and Vert.x

Overview

This article explores the transition from blocking I/O to non-blocking and asynchronous I/O models, emphasizing their importance in modern software development. It highlights the rise of reactive applications driven by cloud computing, Big Data, and IoT. Specifically, it focuses on Vert.x, a powerful toolkit for building reactive applications on the JVM, known for features like the Multi-Reactor Pattern, Event Bus, and Verticles. Benchmarking results show Vert.x's superior performance, making it ideal for high-concurrency environments. Real-world use cases and a case study on a COVID-19 global dashboard illustrate Vert.x's practical benefits and applications.

Exploring the Evolution of I/O Models

• BIO (Blocking I/O): This model is synchronous and blocking. A dedicated thread handles each client connection, leading to inefficiency in resource use, especially under high load.
• NIO (Non-blocking I/O): NIO operates synchronously but is non-blocking. A single thread on a server manages multiple client requests simultaneously, improving scalability. NIO's architecture encompasses three main components: Buffer, Channel, and Selector.
• AIO (Asynchronous I/O or NIO 2.0): AIO extends NIO's capabilities by introducing an asynchronous, non-blocking model that uses callbacks to handle operations. Such a model is well-suited for applications that maintain a high number of connections or require long connection durations.

Trends in I/O for Enterprise Software

The landscape of application requirements has shifted significantly recently, particularly with the advent of cloud computing, Big Data, and IoT. Adopting a reactive architectural style is essential to thrive in the modern application landscape. The popularity of frameworks like Play and Vert.x underscores the growing need for non-blocking I/O. Play leverages Akka and non-blocking I/O, integrating Netty and Akka HTTP to optimize operations. Vert.x design is event-driven and non-blocking, enabling it to support numerous concurrent users efficiently with fewer threads. According to the TechEmpower Web Framework Benchmarks, applications developed using Vert.x surpass their competitors in various standard tests. Notable implementations of these technologies include MasterCard's use of Vert.x and Netflix’s Gateway transition to Async NIO. However, many organizations like Capital One and Red Hat use Vert.x in their technology stack.

For example, MasterCard chose Vert.x as its payment gateway due to its event-driven, non-blocking architecture, which ensures high throughput and low latency. Vert.x's ability to handle a large number of concurrent transactions efficiently is crucial for processing hundreds of millions of payments daily. The framework's horizontal and vertical scalability allows Mastercard to meet growing demands. Additionally, Vert.x worker verticles enable the management of long-running tasks without blocking the main event loop, maintaining system responsiveness. This makes Vert.x an ideal choice for building resilient, high-performance, and highly available payment systems.

Another interesting use case is Dream11, which needed a highly performant, reliable framework to handle unpredictable traffic surges during sports matches. A single-match viewership can reach up to half a billion. Fans log in to Dream 11 to create dream teams, watch live stats, communicate, etc., posing serious scaling challenges. Evaluating existing frameworks, Dream11 found it still needs to meet performance requirements, functional programming, and ZIO support. Vert.x offered excellent performance but lacked functional Scala features. With tight deadlines, Dream11 built ZIO HTTP in-house, leveraging their expertise in scalable systems. Dream11 extensively optimized ZIO Http, achieving performance surpassing Vert.x and other frameworks. ZIO HTTP now handles millions of concurrent requests efficiently, making it the most performant and functional HTTP library for Scala at Dream11.

• Vertx-rest: An Abstraction over resteasy-vertx to simplify writing a vert.x REST application based on JAX-RS annotations.
• AeroSpike Client: The Vert.x Aerospike client provides an asynchronous API for interacting with the Aerospike server.

Matches, like any other sport, are seasonal. They also depend on the teams playing, the crowd, and the interest they will attract. Dream11 has scaled for more than 100 million users. Let's look at some of the use cases to better understand how Vert.x is serving those requirements.

Elastic Nature of Dream11 Traffic During Cricket Matches

Example 1: Pre-Match Surge

Before the start of a major cricket match, millions of users flock to Dream11 to create their fantasy teams. This results in a massive surge in traffic, often within a short time window. During this period, users are:

• Logging into their accounts.
• Browsing player statistics.
• Finalizing their fantasy teams.
• Making last-minute changes before the match starts.

Example 2: In-Match Engagement

During the match, the platform continues to experience high traffic as users:

• Check live scores and player performances.
• Engage in discussions on forums or social features.
• Make real-time decisions if the platform allows mid-game changes or updates.

Example 3: Post-Match Analysis

Once the match is over, users return to the platform to:

• Review their team’s performance.
• Check their rankings and points.
• Withdraw winnings or set up teams for future matches.

For more insights, you can read the full interview with Amit Sharma, Chief Technology Officer at Dream11 on Hackerrank's blog.

"We do some standardization when it comes to our programming languages and frameworks – more than 90% of our services are written on vert.x." — Amit Sharma, CTO of Dream11

Let’s examine in depth why Vert.x is suitable for building distributed applications at scale, followed by a benchmark and a reference case study.

Understanding the Reactor Model

The reactor model is a cornerstone of event-driven architecture. It is characterized by a single-threaded event loop. In this model, events, including I/O events like data readiness or buffer completion, are queued as they occur. For example, a lightweight, high-performance web application that handles thousands of concurrent connections. The server needs to manage client requests, including HTTP requests, database queries, and file I/O operations. The traditional multi-threaded approach is resource-intensive, with each client likely spawning a new thread, eventually leading to high memory usage and context switching overhead, especially under heavy load. However, a Reactor Model handles connections using an event loop that does not block the I/O.

Reactor: Operating on a dedicated thread called the event loop, this component efficiently routes incoming I/O events to their designated handlers. It functions autonomously, ensuring seamless event processing and distribution.

Handler: The reactor orchestrates I/O events, guiding handlers to tackle specific tasks without blocking. This approach leverages NIO's core strengths, embracing non-blocking communication and event-driven processing. Handlers manage pending I/O activities, executing operations swiftly and efficiently. The model's foundation rests on NIO's robust capabilities, ensuring smooth, responsive system performance.

The Four Properties of Reactive Systems

As outlined in The Reactive Manifesto, reactive systems are designed to be:

Responsive: The system consistently responds in a timely manner, ensuring a reliable user interface.

Resilient: The system remains responsive even in the face of failure. This resilience is achieved through replication, containment, isolation, and delegation.

Elastic: The system can efficiently adjust its resources to match varying loads, ensuring consistent performance under different operational conditions.

Message-Driven: The system relies on asynchronous message-passing to ensure loose coupling, isolation, and location transparency. This approach also enhances load management and error handling capabilities.

Image source here

Building Reactive Applications on the JVM with Eclipse Vert.x:

"Eclipse Vert.x is a toolkit for building reactive applications on the JVM."

It's crucial to recognize that Vert.x is a toolkit, not a framework. This distinction highlights its flexibility and modular nature, allowing developers to select the components required for an application.

Vert.x is built using the Netty project and is known for its high-performance, asynchronous networking capabilities on the JVM. This foundation makes Vert.x ideal for developing reactive applications requiring efficient, scalable, and non-blocking I/O operations.

Key Features of Vert.x (Core)

• Multi-Reactor Pattern enhances the scalability and efficiency of handling concurrent operations in Vert.x. Unlike traditional models that use a single event loop, Vert.x employs multiple event loops. Each instance of Vert.x maintains several event loops, typically one per CPU core available on the machine, although the number can be manually adjusted.
• Event Bus serves as the backbone of communication within a Vert.x application, facilitating various communication patterns among different parts of the application. It supports:
• Point-to-point messaging: Direct communication between two parties.
• Request-response messaging:  A two-way communication pattern where responses are returned following requests.
• Publish/subscribe: Allows message broadcast to multiple subscribers.
• Verticles are the fundamental units of code execution in Vert.x, encapsulating the application logic deployed and managed by Vert.x. Vertices run atop an event loop, processing events. For example, network data, timer events, or inter-vertical messages. There are two main types of vertices:
• Standard Vertices: Always run on an event loop thread, ensuring non-blocking operations.
• Worker Verticles: Operate on threads from a dedicated worker pool designed for tasks that might block the event loop.
• The executeBlocking() method allows the execution of blocking code. It runs the specified handler using a thread from the worker pool, thereby preventing the main event loop from being blocked, which maintains the application's responsiveness and performance.

Comparative Analysis:

• Performance: Vert.x and Akka are both designed for high-performance, non-blocking operations and scale well across multiple cores, whereas Jetty, while efficient, traditionally handles fewer concurrent connections due to its synchronous nature.
• Complexity: Jetty is easy to use for straightforward web server needs; Vert.x provides a flexible, albeit more complex, toolkit for reactive applications; Akka offers powerful abstractions with a steeper learning curve for mastering the Actor model.
• Suitability: Choose Jetty for traditional web applications and when embedding HTTP/servlet functionality is needed. Vert.x is excellent for asynchronous, reactive applications on the JVM. Akka fits best with systems requiring robustness, excellent concurrency, and actor-based orchestration.

Creating a reactive HTTP Server using Vert.x

Here is an example of an HTTP server created using Vert.x in Java. The server listens on port 8080 and responds with a simple message to every request.

import io.vertx.core.Vertx;
import io.vertx.core.http.HttpServer;

public class SimpleHttpServer {
    public static void main(String[] args) {
        Vertx vertx = Vertx.vertx();❶
        HttpServer server = vertx.createHttpServer();❷
        server.requestHandler(request -> {❸
            request.response()
                   .putHeader("content-type", "text/plain")
                   .end("Hello from Vert.x HTTP Server!");
        });
        server.listen(8080, result -> {❹
            if (result.succeeded()) {
                System.out.println("Server is now listening on port 8080!");
            } else {
                System.out.println("Failed to bind to port 8080: " + result.cause().getMessage());
            }
        });
    }
}

❶ Creating an Instance of Vertx: This line initializes a new instance of Vertx. Vert.x core provides low-level API and building blocks. Many of the Vert.x extensions use Vert.x core.
❷ Creating the HTTP Server: This line creates a new HTTP server instance.
❸ Handling Incoming Requests: This block creates a request handler to respond to every incoming HTTP request with a plain text message "Hello from Vert.x HTTP Server!".
❹ Listening on Port 8080: This block starts the server and binds it to port 8080. If the server starts successfully, it prints a success message. If it fails, it prints an error message with the cause of the failure.

Benchmarking Details

As web applications grow in complexity and user base, the choice of web server technology becomes critical. Different web servers offer varying capabilities in handling concurrent connections, processing requests quickly, and efficiently managing resources.

Benchmarking the performance of different web server technologies under realistic load conditions is essential. This section compares the throughput performance of two widely used web server frameworks: Jetty9 and Vert.x.

Tool Used:
WRK: A modern HTTP benchmarking tool capable of generating significant load when run on a single multi-core CPU.

Hardware Specifications:
Processor Name: Quad-Core Intel Core i7
Processor Speed: 2.3 GHz
Number of Processors: 1
Total Number of Cores: 4
Hyper-Threading Technology: Enabled
Memory: 32 GB

Software Specifications:
OS: macOS Ventura 13.0
JDK: OpenJDK 17

Purpose of Benchmarking

The primary goal of this benchmarking exercise is to evaluate and compare the throughput of two major categories of web servers:

1. Jetty9 HTTP Web Server
2. Vert.x HTTP Web Server

Throughput is a key metric indicating how many requests a web server can handle per second. Higher throughput implies better performance and the ability to serve more users concurrently.

Benchmarking  Command:

wrk -t10 -c100 -d30s http://127.0.0.1:8080/v1/ -s post.lua

• -t10: Number of threads to use (10 threads).
• -c100: Number of connections to keep open (100 connections).
• -d30s: Duration of the test (30 seconds).
• -http://127.0.0.1:8080/v1/: The URL of the service under test..
• -s post.lua: Lua script for more complex request scenarios (if required).

To reproduce locally checkout git repo

Results

Conclusion

Based on the benchmarking results, Vert.x HTTP Web Server is a better choice for high-throughput applications compared to Jetty9 HTTP Web Server. It can handle a significantly higher number of requests per second, making it suitable for applications requiring high performance and scalability.

These results provide valuable insights for making informed decisions regarding web server selection for high-performance applications.

Case Study: Building a MultiReactor WebApp for COVID-19 Global Dashboard

Overview: In this case study, we will create a web application to display COVID-19 statistical data reports by country, globally, and the latest coronavirus news using the  CollectAPI service. Our approach will include maintaining a local cache that is periodically refreshed for statistical data, while always delegating requests for recent news directly to CollectAPI.

Strategy and Implementation:

We will build a Spring Boot application with a Maven build, leveraging the Vert.x (NIO) toolkit to achieve high throughput without the overhead of blocking operations. The core application will deploy an initial Vert.x verticle as an HTTP web server listening on port 8080. The primary verticle handles all REST API requests and processes them according to the routes defined in the Vert.x router.

The secondary verticle receives all requests from the primary verticle and acts as a request dispatcher, routing requests to the appropriate service handler. We will implement two additional verticles to handle these requests:

1. Redis Manager: This verticle will manage dashboard query API requests and refresh the cached data periodically.
2. CollectAPI Manager: This verticle will act as a proxy, delegating requests to the backend CollectAPI service.

All verticles will communicate by sending and receiving requests/responses as events via the Vert.x event bus, ensuring efficient inter-verticle communication and maintaining a scalable, non-blocking architecture.

Sequence Diagram

MainApplication

An entry point sets up a Spring Boot application, initializes a Vert.x instance, and deploys a verticle named CovidDashboardService.

@SpringBootApplication
@ComponentScan("server")
public class MainApplication {

    @Autowired
    private CovidDashboardService serverVerticle;

    public static void main(String[] args) {
        SpringApplication.run(MainApplication.class, args); ❶
    }

    @PostConstruct
    public void deployVerticles() {
        Vertx vertx = Vertx.vertx(); ❷
        vertx.deployVerticle(serverVerticle, response -> {❸
           // Log response status 
        });
    }
}

❶ Running the Spring Boot Application: This line starts the Spring Boot application.
❷ Creating an Instance of Vertx: This line initializes a new instance of Vertx, the core of Vert.x.
❸ Deploying the first Verticle: This block deploys the CovidDashboardService Verticle. If the deployment is successful, it prints a success message. If it fails, it prints an error message with the cause of the failure.

CovidDashboardService

This class sets up an HTTP server with specific routes and handlers. It integrates with the Vert.x event bus to handle cached GET requests and deploy additional worker variables.

@Component
@NoArgsConstructor

public class CovidDashboardService extends AbstractVerticle {
   public CovidDashboardService(Vertx vertx) {
        this.vertx = vertx;
    }
    @Override
    public void start() throws Exception {
        super.start();
        final Router router = Router.router(vertx); ❶
        router.route().handler(BodyHandler.create());                   

        addCachedGETRoute(router, List["/cache1","/cache2","cache3"])   ❷ 
        addProxyGETRoute(router, List["/proxy1","/proxy2","proxy3"]) ❸

        HttpServer httpServer = vertx.createHttpServer()  ❹
   .requestHandler(router)
                .exceptionHandler(exception -> {//Log exception })
                .connectionHandler(res -> {//Log new connection });
         httpServer.listen(port);

    // deploy next verticle RequestRouteDispatcher 
        vertx.deployVerticle(RequestRouteDispatcher::new,  ❺
        new DeploymentOptions().setWorker(true).setWorkerPoolName("API-Router")
        .setWorkerPoolSize(20),result -> { //Log result status });
   }
private void addCachedGETRoute(Router router, List pathList) {❻
        Arrays.stream(pathList).iterator()
 .forEachRemaining(apiPath -> router.get(apiPath.value)
        .handler(this::handleCachedGetRequest));
}
private void handleCachedGetRequest(RoutingContext ctx) {❼      vertx.eventBus().request("CACHED", ctx.normalizedPath(), response -> {
            if (response.succeeded()) { // ctx.response().end("{result}")}
            else { // ctx.setStatusCode(500))}
      });
}
}

❶ Router Initialization: This line initializes a new instance of Router.
❷ Adding Cached Routes: Add all API paths for cached query to Router.
❸ Adding Proxy Routes: Add all API paths for proxy query to Router.
❹ Creating and Starting the HTTP Server: This block starts the server and binds it to port 8080. 
❺ Deploying the RequestRouteDispatcher Verticle: The next vertical for Request Dispatcher
❻ Adding and Handling Cached GET Routes: Method adding GET path to the router for cached query.
❼ Adding and Handling Proxy GET Routes: Method adding GET path to the router for proxy query.

RequestRouteDispatcher

This class sets up consumers for event bus messages related to caching and proxying and deploys additional verticles for managing Redis and data collection functionalities. Each consumer is responsible for handling specific types of requests, while the deployment methods ensure that related verticles are initialized and running.

@Component
public class RequestRouteDispatcher extends AbstractVerticle {

    @Override
    public void start() throws Exception {
        super.start();
   vertx.eventBus().<String>consumer("CACHED") ❶
        .handler(handleCachedRequest());                         
   vertx.eventBus().<String>consumer("PROXY")  ❷
               .handler(handleProxyRequest());

        deployRedisManagerVerticle();  ❸
        deployCollectManagerVerticle();❹
}

❶ Setting Up Event Bus Consumer for "CACHED" Requests: This line sets up an event bus consumer for messages with the address "CACHED".  It invokes handleCachedRequest whenever a message is received.
❷ Setting Up Event Bus Consumer for "PROXY" Requests: This line sets up an event bus consumer for messages with the address "PROXY". It invokes the handleProxyRequest method whenever a message is received.
❸ Deploying the RedisManagerVerticle: This method deploys the RedisManagerVerticle and logs the result of the deployment.
❹ Deploying the CollectManagerVerticle: This method deploys the CollectManagerVerticle and logs the result of the deployment.

Redis Manager

This class sets up an event bus consumer to handle caching requests by checking Redis and potentially delegating to another service if the data is absent. It also includes a mechanism to refresh the cache, ensuring the data remains up-to-date periodically. The Redis client is configured with options to manage pooling and handler waiting limits, optimizing its performance for high-load scenarios.

import io.vertx.redis.client.Redis;

@Component
public class RedisClientManager extends AbstractVerticle {
    private RedisAPI redisAPI;

    @Override
    public void start() {
      vertx.eventBus().<String>consumer("CACHED")❶
                     .handler(handleRedisRequest());  
      vertx.setPeriodic(5000, id -> refershCache());❷ //refresh every 5 sec

      Redis client = Redis.createClient(vertx, new RedisOptions() ❸
                .setMaxPoolSize(10)
                .setMaxWaitingHandlers(50));
      redisAPI = RedisAPI.api(client);
      refershCache();
    }
private Handler<Message<String>> handleRedisRequest() { ❹
    // Check if present in redis else delegate to proxy CollectAPI 
}

❶ Setting Up Event Bus Consumer for "CACHED" Requests: This line sets up an event bus consumer for messages with the address "CACHED". It invokes the handleCachedRequest method whenever a message is received.
❷ Setting Up Periodic cache refresh: This line sets up a periodic task that calls the refreshCache method every 5000 milliseconds (5 seconds).
❸ Creating Redis Client: This block creates a Redis client with the specified options and init the redisAPI object
❹ Handling Redis Request: This method checks if the requested data is present in Redis. If it is, it replies with the cached value. If not, it delegates the request to the "PROXY" address and caches the result.

Collect API Manager

This class sets up an event bus consumer to handle proxy requests to the CollectAPI. It uses the Vert.x WebClient to make HTTP GET requests to the API, adding necessary headers for content type and authorization. Upon receiving a response or encountering an error, it replies to the original message with the appropriate response or error message. This facilitates seamless integration and communication between different parts of the application and the external CollectAPI service.

@Component
public class CollectAPIManager extends AbstractVerticle {

    public static final String COLLECT_API_URL = "api.collectapi.com";
    private WebClient client;

    @Override
    public void start() {
    vertx.eventBus().<String>consumer("PROXY") ❶
.handler(handleCollectAPIRequest());
    client = WebClient.create(vertx, new WebClientOptions()); ❷
    }

private Handler<Message<String>> handleCollectAPIRequest() { ❸
       return msg -> {
          client
            .get(COLLECT_API_URL, msg.body()) 
            .putHeader("content-type", "application/json")
            .putHeader("authorization", "apikey your api key")
            .send()
            .onSuccess(response -> {
   msg.reply(response.bodyAsString());
            })
            .onFailure(err -> {
                msg.reply("Failed to connect CollectAPI" + err.getCause());
            });
        };
    }

❶ Setting Up Event Bus Consumer for "PROXY" Requests: This line sets up an event bus consumer for messages with the address "PROXY". 
❷ Creating the WebClient Instance: This line initializes a new instance of WebClient with default options. You can customize the WebClientOptions if needed.
❸ Handling CollectAPI Request: This method sends a GET request to the CollectAPI using the WebClient. If the request is successful, it replies with the response body. If the request fails, it replies with an error message.

Summary

This article provides an in-depth exploration of the evolution of I/O models, from blocking I/O (BIO) to non-blocking I/O (NIO) and asynchronous I/O (AIO), and their impact on modern software. The main problem addressed is the inefficiency and scalability limitations of traditional blocking I/O models in handling high loads and numerous concurrent connections, which is increasingly inadequate in the era of cloud computing, Big Data, and IoT.

The solution presented is a demonstration of reactive architectures, which are designed to be responsive, resilient, elastic, and message-driven. Vert.x, a toolkit for building reactive applications on the JVM, is highlighted as a powerful tool that offers features like the Multi-Reactor Pattern, Event Bus, and Verticles to handle high concurrency efficiently. Benchmarking results demonstrate Vert.x's superior performance to other tools, reinforcing its suitability for modern high-performance applications.

A detailed case study on building a MultiReactor web application for a COVID-19 global dashboard is provided. This application leverages Vert.x to handle high-throughput, non-blocking operations efficiently. It includes components like the Redis Manager and CollectAPI Manager, which manage cached data and proxy requests. Real-world implementations by companies like MasterCard and Dream11 illustrate Vert.x's practical applications and advantages in building scalable, high-performance systems. This comprehensive analysis underscores Vert.x's effectiveness in addressing the challenges of modern I/O and reactive application development.