Implementing Microservicilities with Quarkus and MicroProfile

Why Microservicilities?

In a microservices architecture, an application is formed by several interconnected services where all of them work together to produce the required business functionality.

So a typical enterprise microservices architecture looks like this:

At the beginning, it might seem easy to implement an application using a microservices architecture.

But doing so properly is not an easy journey as there are some new challenges that weren’t present with a monolith architecture.

Some of these are fault tolerance, service discovery, scaling, logging, and tracing, just to mention a few.

To solve these challenges, every microservice should implement what we at Red Hat have named "Microservicilities."

The term refers to a list of cross-cutting concerns that a service must implement apart from the business logic to resolve these concerns as summarized in the following diagram:

The business logic may be implemented in any language (Java, Go, JavaScript) or any framework (Spring Boot, Quarkus) but around the business logic, the following concerns should be implemented:

API: The service is accessible through a defined set of API operations. For example, in the case of RESTful Web APIs, HTTP is used as a protocol. Moreover, the API can be documented using tools such as Swagger.

Discovery: Services need to discover other services.

Invocation: After a service is discovered, it needs to be invoked with a set of parameters and optionally return a response.

Elasticity: One of the important features of a microservices architecture is that each of the services is elastic, meaning it can be scaled up and/or down independently depending on some parameters like criticality of the system or depending on the current workload.

Resiliency: In a microservice architecture, we should develop with failure in mind, especially when communicating with other services. In a monolith application, the application, as a whole, is up or down. But when this application is broken down into a microservice architecture, the application is composed of several services and all of them are interconnected by the network, which implies that some parts of the application might be running while others may fail. It is important to contain the failure to avoid propagating the error through the other services. Resiliency (or application resiliency) is the ability for an application/service to react to problems and still provide the best possible result.

Pipeline: A service should be deployed independently without any kind of deployment orchestration. For this reason, each service should have its own deployment pipeline.

Authentication: One of the key aspects regarding security in a microservices architecture is how to authenticate/authorize calls among internal services. Web Tokens (and tokens in general) are the preferred way for representing claims securely among internal services.

Logging: Logging is simple in monolith applications as all the components of the application are running in the same node. Components are now distributed across several nodes in the form of services, hence to have a complete view of the logging traces, a unified logging system/data collector is required.

Monitoring: Measuring how your system is performing, understanding the overall health of the application, and alerting when something is wrong are key aspects to keeping a microservices-based application running correctly. Monitoring is a key aspect to control the application.

Tracing: Tracing is used to visualize a program’s flow and data progression. That’s especially useful when, as a developer/operator, we require to check the user’s journey through the entire application.

Kubernetes is becoming the de-facto tool for deploying microservices. It’s an open-source system for automating, orchestrating, scaling, and managing containers.

Just three of the ten microservicilities are covered when using Kubernetes.

Discovery is implemented with the concept of a Kubernetes Service. It provides a way to group Kubernetes Pods (acting as one) with a stable virtual IP and DNS name. Discovering a service is just a matter of making requests using Kubernetes’ service name as a hostname.

Invocation of services is easy with Kubernetes as the platform itself provides the network required to invoke any of the services.

Elasticity (or scaling) is something that Kubernetes had in mind since the very beginning, for example running kubectl scale deployment myservice --replicas=5 command, the myservice deployment scales to five replicas or instances. Kubernetes platform takes care of finding the proper nodes, deploying the service, and maintaining the desired number of replicas up and running all the time.

But what about the rest of the microservicilities? Kubernetes only covers three of them, so how can we implement the rest of them?

There are many strategies to follow depending on the language or framework used, but in this article, we’ll see how to implement some of them using Quarkus.

What is Quarkus?

Quarkus is a full-stack, Kubernetes-native Java framework made for Java virtual machines (JVMs) and native compilation, optimizing Java specifically for containers, and enabling it to become an effective platform for serverless, cloud, and Kubernetes environments.

Instead of reinventing the wheel, Quarkus uses well-known enterprise-grade frameworks backed by standards/specifications and makes them compilable to a binary using GraalVM.

What is MicroProfile?

Quarkus integrates with the MicroProfile specification moving the enterprise Java ecosystem into the microservices architecture.

In the following diagram, we see all the APIs that comprise the MicroProfile specification. Some APIs, such as CDI, JSON-P, and JAX-RS, are based on Jakarta EE (formerly Java EE) specification. The rest were developed by the Java community.

Let’s implement API, invocation, resilience, authentication, logging, monitoring, and tracing microservicilities using Quarkus.

How to Implement Microservicilities Using Quarkus

Getting Started

The quickest way to start using Quarkus is through the start page by adding the required dependencies. For this example, the following dependencies are registered to meet the microservicilities requirements:

• API: RESTEasy JAX-RS, RESTEasy JSON-B, OpenAPI
• Invocation: REST Client JSON-B
• Resilience: Fault Tolerance
• Authentication: JWT
• Logging: GELF
• Monitoring: Micrometer metrics
• Tracing: OpenTracing

We can manually select each of the dependencies, or navigate to the following link Microservicilities Quarkus Generator where all of them are selected. Then push Generate your application button to download the zip file containing the scaffolded application.

The Service

For this example, a very simple application is generated with only two services. One service, named rating service, returns the ratings of a given book, and another service, named book service, returns the information of a book together with its ratings. All calls among services must be authenticated.

In the following figure, we see an overview of the complete system:

Rating service is already developed and provided as a Linux container. Start the service at port 9090 by running the following command:

docker run --rm -ti -p 9090:8080 
quay.io/lordofthejars/rating-service:1.0.0

To validate the service, make a request to http://localhost:9090/rate/1

curl localhost:8080/rate/1 -vv

> GET /rate/1 HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< www-authenticate: Bearer {token}
< Content-Length: 0

The status code returned is 401 Unauthorized because no authorization information was provided in the request in the form of a bearer token (JWT). Only valid tokens with group Echoer are allowed to access the rating service.

API

Quarkus uses the well-known JAX-RS specification to define RESTful web APIs. Under the covers, Quarkus uses the RESTEasy implementation working directly with the Vert.X framework without using Servlet technology.

Let’s define an API for the book service implementing the most common operations:

import javax.ws.rs.Consumes;
import javax.ws.rs.DELETE;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriBuilder;

@Path("/book")
public class BookResource {

   @GET
   @Path("/{bookId}")
   @Produces(MediaType.APPLICATION_JSON)
   public Book book(@PathParam("bookId") Long bookId) {
    // logic
   }

   @POST
   @Consumes(MediaType.APPLICATION_JSON)
   public Response getBook(Book book) {
       // logic

       return Response.created(
                   UriBuilder.fromResource(BookResource.class)
                     .path(Long.toString(book.bookId))
                     .build())
               .build();
   }

   @DELETE
   @Path("/{bookId}")
   public Response delete(@PathParam("bookId") Long bookId) {
       // logic

       return Response.noContent().build();
   }

   @GET
   @Produces(MediaType.APPLICATION_JSON)
   @Path("search")
   public Response searchBook(@QueryParam("description") String description) {       
       // logic

       return Response.ok(books).build();
   }
}

The first thing to notice is that four different endpoints are defined:

• GET /book/{bookId} uses the GET HTTP method to return the book information with its rating. The return element is automatically unmarshaled to JSON.
• POST /book uses the POST HTTP method to insert a book coming as body content. Body content is automatically marshaled from JSON to a Java object.
• DELETE /book/{bookId} uses the DELETE HTTP method to delete a book by its ID.
• GET /book/search?description={description} searches books by their description.

The second thing to notice is the return type, sometimes as a Java object and other times as an instance of javax.ws.rs.core.Response. When a Java object is used, it is marshaled from a Java object to the media type set in the @Produces annotation. In this particular service, the output is a JSON document. With the Response object, we have a fine-grained control on what is sent back to the caller; you can set the HTTP status code, headers, or for example, the content returned to the caller. It depends on the use case to prefer one approach over the other.

Invocation

After we’ve defined the API to access the book service, it’s time to develop the piece of code to invoke the rating service to retrieve the rating of a book.

Quarkus uses MicroProfile Rest Client specification to access external (HTTP) services. It provides a type-safe approach to invoke RESTful services over HTTP using some of the JAX-RS 2.0 APIs for consistency and easier re-use.

The first element to create is an interface representing the remote service using the JAX-RS annotations.

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

import org.eclipse.microprofile.rest.client.inject.RegisterRestClient;

@Path("/rate")
@RegisterRestClient
public interface RatingService {
 
   @GET
   @Path("/{bookId}")
   @Produces(MediaType.APPLICATION_JSON)
   Rate getRate(@PathParam("bookId") Long bookId);

}

When the getRate() method is called, a remote HTTP call is invoked at /rate/{bookId} replacing the bookId with the value set in the method parameter. It is important to annotate the interface with the @RegisterRestClient annotation.

Then the RatingService interface needs to be injected into BookResource to execute the remote calls.

import org.eclipse.microprofile.rest.client.inject.RestClient;

@RestClient
RatingService ratingService;

@GET
@Path("/{bookId}")
@Produces(MediaType.APPLICATION_JSON)
public Book book(@PathParam("bookId") Long bookId) {
    final Rate rate = ratingService.getRate(bookId);

    Book book = findBook(bookId);
    return book;
}

The @RestClient annotation injects a proxied instance of the interface, providing the implementation of the client.

The last thing is to configure the service location (the hostname part). In Quarkus, the configuration properties are set in src/main/resources/application.properties file. To configure the location of the service, we need to use the fully qualified name of the Rest Client interface with URL as key, and the location as a value:

org.acme.RatingService/mp-rest/url=http://localhost:9090

Before accessing the rating service correctly without the 401 Unauthorized issue, the mutual authentication issue needs to be addressed.

Authentication

Token-Based Authentication mechanisms allow systems to authenticate, authorize, and verify identities based on a security token. Quarkus integrates with the MicroProfile JWT RBAC Security specification to protect services using JWT Bearer Tokens.

To protect an endpoint using MicroProfile JWT RBAC Security, we only need to annotate the method with @RolesAllowed annotation.

@GET
@Path("/{bookId}")
@RolesAllowed("Echoer")
@Produces(MediaType.APPLICATION_JSON)
public Book book(@PathParam("bookId") Long bookId)

Then we configure the issuer of the token and the location of the public key to verify the signature of the token in the application.properties file:

mp.jwt.verify.publickey.location=https://raw.githubusercontent.com/redhat-developer-demos/quarkus-tutorial/master/jwt-token/quarkus.jwt.pub
mp.jwt.verify.issuer=https://quarkus.io/using-jwt-rbac

This extension automatically verifies that: the token is valid; the issuer is correct; the token hasn’t been modified; the signature is valid; it hasn’t expired.

Both book service and rating service are now protected by the same JWT issuer and keys, so the communication among services requires the user to be authenticated providing a valid bearer token in the Authentication header.

Having rating service up and running, let’s start the book service with the following command:

./mvnw compile quarkus:dev

Finally, we can make a request to get book information providing a valid JSON Web Token as a bearer token.

The generation of the token is out of the scope of this article, and a token has been already generated:

curl -H "Authorization: Bearer eyJraWQiOiJcL3ByaXZhdGVLZXkucGVtIiwidHlwIjoiSldUIiwiYWxnIjoiUlMyNTYifQ.eyJzdWIiOiJqZG9lLXVzaW5nLWp3dC1yYmFjIiwiYXVkIjoidXNpbmctand0LXJiYWMiLCJ1cG4iOiJqZG9lQHF1YXJrdXMuaW8iLCJiaXJ0aGRhdGUiOiIyMDAxLTA3LTEzIiwiYXV0aF90aW1lIjoxNTcwMDk0MTcxLCJpc3MiOiJodHRwczpcL1wvcXVhcmt1cy5pb1wvdXNpbmctand0LXJiYWMiLCJyb2xlTWFwcGluZ3MiOnsiZ3JvdXAyIjoiR3JvdXAyTWFwcGVkUm9sZSIsImdyb3VwMSI6Ikdyb3VwMU1hcHBlZFJvbGUifSwiZ3JvdXBzIjpbIkVjaG9lciIsIlRlc3RlciIsIlN1YnNjcmliZXIiLCJncm91cDIiXSwicHJlZmVycmVkX3VzZXJuYW1lIjoiamRvZSIsImV4cCI6MjIwMDgxNDE3MSwiaWF0IjoxNTcwMDk0MTcxLCJqdGkiOiJhLTEyMyJ9.Hzr41h3_uewy-g2B-sonOiBObtcpkgzqmF4bT3cO58v45AIOiegl7HIx7QgEZHRO4PdUtR34x9W23VJY7NJ545ucpCuKnEV1uRlspJyQevfI-mSRg1bHlMmdDt661-V3KmQES8WX2B2uqirykO5fCeCp3womboilzCq4VtxbmM2qgf6ag8rUNnTCLuCgEoulGwTn0F5lCrom-7dJOTryW1KI0qUWHMMwl4TX5cLmqJLgBzJapzc5_yEfgQZ9qXzvsT8zeOWSKKPLm7LFVt2YihkXa80lWcjewwt61rfQkpmqSzAHL0QIs7CsM9GfnoYc0j9po83-P3GJiBMMFmn-vg" localhost:8080/book/1 -v

And the response is again a forbidden error:

< HTTP/1.1 401 Unauthorized
< Content-Length: 0

You might wonder why we still get this error after providing a valid token. If we inspect the console of the book service, we see that the following exception was thrown:

org.jboss.resteasy.client.exception.ResteasyWebApplicationException: Unknown error, status code 401
    at org.jboss.resteasy.client.exception.WebApplicationExceptionWrapper.wrap(WebApplicationExceptionWrapper.java:107)
    at org.jboss.resteasy.microprofile.client.DefaultResponseExceptionMapper.toThrowable(DefaultResponseExceptionMapper.java:21)

The reason for this exception is that we are authenticated and authorized to access the book service, but the bearer token has not been propagated to the rating service.

To automatically propagate Authorization headers from incoming requests to rest-client requests, two modifications are required.

The first modification is to modify the Rest Client interface and annotate it with org.eclipse.microprofile.rest.client.inject.RegisterClientHeaders.

@Path("/rate")
@RegisterRestClient
@RegisterClientHeaders
public interface RatingService {}

The second modification is to configure which headers are propagated among requests. This is set in the application.properties file:

org.eclipse.microprofile.rest.client.propagateHeaders=Authorization

Execute the same curl command as before and we’ll get the correct output:

< HTTP/1.1 200 OK
< Content-Length: 39
< Content-Type: application/json
<
* Connection #0 to host localhost left intact
{"bookId":2,"name":"Book 2","rating":1}* Closing connection 0

Resilience

Having fault-tolerant services is important in a microservices architecture to avoid propagating a failure from one service to all direct and indirect callers of it. Quarkus integrates the MicroProfile Fault Tolerance specification with the following annotations for dealing with failures:

●    @Timeout: Defines a maximum duration time for execution before an exception is thrown.
●    @Retry: Retries execution again if the call fails.
●    @Bulkhead: Limits concurrent execution so that failures in that area can’t overload the whole system.
●    @CircuitBreaker: Automatically fail-fast when execution repeatedly fails.
●    @Fallback: Provides an alternative solution/default value when execution fails.

Let’s add three retries with a sleep timer of one second among retries in case an error occurs when accessing the rating service.

@Retry(maxRetries = 3, delay = 1000)
Rate getRate(@PathParam("bookId") Long bookId);

Now stop the rating service and execute a request. The following exception is thrown:

org.jboss.resteasy.spi.UnhandledException: javax.ws.rs.ProcessingException: RESTEASY004655: Unable to invoke request: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9090 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused

Obviously, the error is there, but note that there was an elapsed time of three seconds before the exception is thrown as three retries with a one-second delay are executed.

In this case, the rating service is down, so there is no possible recovery, but in a real-world example where rating service might be down for just a small amount of time, or multiple replicas of the service are deployed, a simple retry operation might be enough to recover and provide a valid response.

But, when retries are not enough when an exception is thrown, we can either propagate the error to callers or provide an alternative value for the call. This alternative may be a call to another system (i.e., a distributed cache) or a static value.

For this use case, when the connection to the rating service fails, a rating value of 0 is returned.

To implement a fallback logic, the first thing to do is implement the org.eclipse.microprofile.faulttolerance.FallbackHandler interface setting the return type as the same kind as the fallback strategy method is providing as an alternative value. For this case, a default Rate object is returned.

import org.eclipse.microprofile.faulttolerance.ExecutionContext;
import org.eclipse.microprofile.faulttolerance.FallbackHandler;

public class RatingServiceFallback implements FallbackHandler<Rate> {

   @Override
   public Rate handle(ExecutionContext context) {
       Rate rate = new Rate();
       rate.rate = 0;
       return rate;
   }
 
}

The last thing to do is annotate the getRating() method with @org.eclipse.microprofile.faulttolerance.Fallback annotation to configure the fallback class to be executed when no recovery is possible.

@Retry(maxRetries = 3, delay = 1000)
@Fallback(RatingServiceFallback.class)
Rate getRate(@PathParam("bookId") Long bookId);

If you repeat the same request as before, no exception is thrown but a valid output with the rating field is set to 0.

* Connection #0 to host localhost left intact
{"bookId":2,"name":"Book 2","rating":0}* Closing connection 0

The same approach may be used with any of the other strategies provided by the specification. For example, in the case of circuit breaking pattern:

@CircuitBreaker(requestVolumeThreshold = 4,
               failureRatio=0.75,
               delay = 1000)

If three (4 x 0.75) failures occur among the rolling window of four consecutive invocations, then the circuit is opened for 1000 ms and then back to half-open. If the invocation while the circuit is half-open succeeds, then it is closed again. Otherwise, it remains open.

Logging

In the microservices architecture, it’s recommended to collect logs of all services in one unified log for more efficient use and understanding.

One solution is to use Fluentd, an open-source data collector for a unified logging layer in Kubernetes. Quarkus integrates with Fluentd using the Graylog Extended Log Format (GELF).

The integration is really simple. First, use log logic as with any other Quarkus application:

import org.jboss.logging.Logger;

private static final Logger LOG = Logger.getLogger(BookResource.class);

@GET
@Path("/{bookId}")
@RolesAllowed("Echoer")
@Produces(MediaType.APPLICATION_JSON)
public Book book(@PathParam("bookId") Long bookId) {
    LOG.info("Get Book");

Next, enable the GELF format and set the Fluentd server location:

quarkus.log.handler.gelf.enabled=true
quarkus.log.handler.gelf.host=localhost
quarkus.log.handler.gelf.port=12201

Finally, we can make a request to the logged endpoint:

curl -H "Authorization: Bearer ..." localhost:8080/book/1

{"bookId":1,"name":"Book 1","rating":3}

Nothing has changed in terms of output, but the logline has been transmitted to Fluentd. If Kibana is used to visualize data, we’ll see the logline stored:

Monitoring

Monitoring is another "microservicilitie" that needs to be implemented in our microservice architecture. Quarkus integrates with Micrometer for application monitoring. Micrometer provides a single entry point to the most popular monitoring systems, allowing you to instrument your JVM-based application code without vendor lock-in.

For this example, Prometheus format is used as monitoring output but Micrometer (and Quarkus) also supports other formats like Azure Monitor, Stackdriver, SignalFx, StatsD, and DataDog.

You can register the following Maven dependency to provide Prometheus output:

<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-micrometer-registry-prometheus</artifactId>
</dependency>

The Micrometer extension registers some metrics related to the system, JVM, or HTTP by default. A subset of the collected metrics are available at the /q/metrics endpoint as shown below:

curl localhost:8080/q/metrics

jvm_threads_states_threads{state="runnable",} 22.0
jvm_threads_states_threads{state="blocked",} 0.0
jvm_threads_states_threads{state="waiting",} 10.0
http_server_bytes_read_count 1.0
http_server_bytes_read_sum 0.0

But application-specific metrics may also be implemented using the Micrometer API.
Let’s implement a custom metric that measures the highest-rated book.

Registering a metric, in this case, a gauge, is accomplished using the io.micrometer.core.instrument.MeterRegistry class.

private final MeterRegistry registry;
private final LongAccumulator highestRating = new LongAccumulator(Long::max, 0);
 
public BookResource(MeterRegistry registry) {
    this.registry = registry;
    registry.gauge("book.rating.max", this,
               BookResource::highestRatingBook);
}

Let’s make some requests and validate that the gauge is correctly updated.

curl -H "Authorization: Bearer ..." localhost:8080/book/1

{"bookId":1,"name":"Book 1","rating":3}

curl localhost:8080/q/metrics

# HELP book_rating_max
# TYPE book_rating_max gauge
book_rating_max 3.0

We can also set a timer to record the time spent on obtaining rating information from the rating service.

Supplier<Rate> rateSupplier = () -> {
      return ratingService.getRate(bookId);
};
      
final Rate rate = registry.timer("book.rating.test").wrap(rateSupplier).get();

Let’s make some requests and validate the time spent on gathering ratings.

# HELP book_rating_test_seconds
# TYPE book_rating_test_seconds summary
book_rating_test_seconds_count 4.0
book_rating_test_seconds_sum 1.05489108
# HELP book_rating_test_seconds_max
# TYPE book_rating_test_seconds_max gauge
book_rating_test_seconds_max 1.018622001

Micrometer uses MeterFilter instances to customize the metrics emitted by MeterRegistry instances. The Micrometer extension will detect MeterFilter CDI beans and use them when initializing MeterRegistry instances.

For example, we can define a common tag to set the environment (prod, testing, staging, etc.) where the application is running.

@Singleton
public class MicrometerCustomConfiguration {
 
   @Produces
   @Singleton
   public MeterFilter configureAllRegistries() {
       return MeterFilter.commonTags(Arrays.asList(
               Tag.of("env", "prod")));
   }

}

Send a new request and validate that the metrics are now tagged.

http_client_requests_seconds_max{clientName="localhost",env="prod",method="GET",outcome="SUCCESS",status="200",uri="/rate/2",} 0.0

Note the tag env containing the value prod.

Tracing

Quarkus applications utilize the OpenTracing specification to provide distributed tracing for interactive web applications.

Let’s configure OpenTracing to connect to a Jaeger server, setting book-service as the service name to identify the traces :

quarkus.jaeger.enabled=true
quarkus.jaeger.endpoint=http://localhost:14268/api/traces
quarkus.jaeger.service-name=book-service
quarkus.jaeger.sampler-type=const
quarkus.jaeger.sampler-param=1

Now, let’s make a request:

curl -H "Authorization: Bearer ..." localhost:8080/book/1

{"bookId":1,"name":"Book 1","rating":3}

Access to the Jaeger UI to validate that the call is traced:

Conclusions

Developing and implementing a microservices architecture is a bit more challenging than developing a monolith application. We believe that microservicilities can drive you to develop services correctly in terms of the application infrastructure.

Most of the microservicilities explained here (except API and Pipelines) are new or are implemented differently in monolith applications. The reason is that now the application is broken down to several pieces, all of them interconnected within the network.

If you are planning to develop microservices and deploy them to Kubernetes, then Quarkus is a good solution as it integrates smoothly with Kubernetes. Implementing most of the microservicilities is simple and requires just a few lines of code.

Source code demonstrated in this article may be found on github.

About the Author

Alex Soto is a Director of Developer Experience at Red Hat. He is passionate about the Java world, software automation, and he believes in the open-source software model. Soto is the co-author of Manning | Testing Java Microservices and O’Reilly | Quarkus Cookbook and contributor to several open-source projects. A Java Champion since 2017, he is also an international speaker and teacher at Salle URL University. You can follow him on Twitter (Alex Soto ⚛️) to stay tuned to what’s going on in Kubernetes and Java world.