In a microservices architecture, the application is formed by several interconnected services where all of them working together produces the required business functionality. So a typical enterprise microservices architecture looks like this:

Every service depends on other services and some of them backed by a database. Each service must have its own set of tests (unit, component, contract,...) to validate its correctness, for example after a change.

Let’s focus on one service that it is in the middle of the graph; if you look inside, you’d see something very similar to the next figure:

A service usually contains some (if not all) of these layers, which in summary can be described as follows:

• Resources: Acts as an entry point to the service. They unmarshal messages that come in the form of any protocol (i.e JSON) and transforms into domain objects. Also, they validate that the input parameters are valid, and are in charge of the marshaling process to transform domain objects to a protocol message for returning a response. For example, if your technology is Jakarta EE/MicroProfile, JAX-RS takes care of all these aspects, and the same for Spring Boot or any other technology.
• Business: It is the business logic, which implements the business rules of the service and orchestrates all the parts of the service such as Persistence and Gateway layer. Domain objects are part of this layer.
• Persistence: It is the part that saves domain objects to a “persistent” backend (SQL or NoSQL databases). If your technology is Jakarta EE/MicroProfile then this is the part that deals with JPA (in case of SQL databases).
• Gateway: So far, all previous parts are common in any other architecture, but gateways are typical layers of distributed architectures. A gateway encapsulates all logic to communicate from consumer service (where the gateway is implemented) to provider service (marshaling/unmarshalling operation between the underlying protocol and domain objects) and the configuration of the network client, timeout, retries, resiliency, … Typically if you are using Jakarta EE/MicroProfile and JSON as messaging protocol you likely be using JAX-RS Client as Http Client to communicate with another service.

You can test the top layers by mocking out the gateway layer. For example, the code using Mockito, for testing the business layer might look like:

   @Mock
   WorldClockServiceGateway worldClockServiceGateway;

   @Test
   public void should_deny_access_if_not_working_hour() {

       // Given

       when(worldClockServiceGateway.getTime("cet")).thenReturn(LocalTime.of(0,0));

       SecurityResource securityResource = new SecurityResource();
       securityResource.worldClockServiceGateway = worldClockServiceGateway;

       // When

       boolean access = securityResource.isAccessAllowed();

       // Then

       assertThat(access).isFalse();
}

But of course you still need to validate that the Gateway class (WorldClockServiceGateway) works as expected:

• It is able to marshal/unmarshal from the protocol specific to the domain object
• (Http) client configuration parameters are correct (i.e timeouts, retries, headers, …)
• It behaves as expected in the event of network errors

To test all these points, you might think about running the service that the gateway communicates with and running a test against the real service. This might seem like a good solution but it has some problems:

• You need to know how to start the provider service from the consumer service, and also all the transitive services that the provider depends on, as well as the required databases. And this looks like a violation of single responsibility, the consumer service should only know how to deploy itself and not their dependent services.

• In case that any service requires a database, a dataset must be prepared.
• Starting several services also implies that any of them might fail because of any internal/network error, hence making the test failing not because of an error in gateway class but because of infrastructure, making this test flaky.
• Moreover, starting all required services (even if it is only one) can take a lot of time, so your test suite cannot provide quick feedback.

One of the solutions that you might think is to skip this kind of tests because they are usually flaky and they take so much time to execute as they need to boot up the whole universe to run a simple gateway test. But the communication part in a microservices architecture is the central piece, it is exactly this part where any interaction with the system happens, so it is important to be tested to validate that behaves as expected.

The solution to this problem is service virtualization.

What is Service Virtualization?

Service virtualization is a technique used to simulate the behavior of dependencies of a service. Although service virtualization is commonly associated with REST API-based services, the same concept can be applied to any other kind of dependencies like databases, ESBs, JMS, …

Apart from helping in the testing of internal services, service virtualization also helps you on testing services that are not in your control, fixing some common problems that make this kind of tests flaky. Some of them are:

• Your network is down, so you cannot communicate with the external service.
• External service is down, and you get some unexpected errors.
• Limits on the API. Some public APIs has some limitations on rate/day. If you reach this level then your tests will start failing.

With service virtualization, you can avoid all these problems since you are not reaching the real service, but a virtual one.

But service virtualization can be used for more than testing the happy path cases, but many developers and testers find that its real power is in edge cases that are difficult to test against real services such as how the service behaves in case of low-latency responses or in case of unexpected errors.

If you think about how you have been testing components in the monolith architecture, you’ve been using something similar but between objects, and it is called mocking. When using mocks, you are simulating the behavior of an object by providing a canned answer to a method call. When using service virtualization, you are doing something similar but instead of simulating the behavior of an object, you are providing a canned answer of a remote service. For this reason, service virtualization is sometimes known as mocking for enterprises.

In the next figure you can see how service virtualization works:

In this concrete case communication between services is happening through the HTTP protocol, hence a thin HTTP server is used to be responsible for consuming the requests from the gateway class and provide the canned answers.

Running Modes

Generally speaking, service virtualization has two modes:

• Playback mode: Uses simulation data in order to provide a response rather than forwarding it to real service. Simulation data can be created either manually (in case the real service does not exist yet) or using capture mode.
• Record mode: intercepts communication between services and records outgoing requests and incoming responses from the real service. Usually, capture mode is used as the starting point in the process of creating the initial simulation data.

Depending on the implementation, they might contain other modules, but all of them should contain these two modes.

Hoverfly

What is Hoverfly?

Hoverfly is an open source, lightweight, service virtualization API simulation tool written in the Go programming language. It also offers language bindings that tightly integrate with Java.

Hoverfly Java

Hoverfly Java is a Java wrapper around Hoverfly abstracting you away from the installation of Hoverfly and managing its lifecycle. Hoverfly Java offers a Java DSL to generate simulation data programmatically and deeply integrates with JUnit and JUnit5.

Hoverfly Java sets the network Java system properties to use the Hoverfly proxy. This effectively means that all communication between the Java runtime and the physical network layer will be intercepted by the Hoverfly proxy. What this means is that although your HTTP Java Client might be pointing to an external site like http://worldclockapi.com the connection will be intercepted and forwarded by Hoverfly.

It is important to note that if your Http Client does not honor the Java network proxy settings, you will need to set it manually.

From this point, we are going to use Hoverfly and Hoverfly Java interchangeably to refer to Hoverfly Java.

To add Hoverfly to work with JUnit 5, you need to register next dependency on your build tool:

 <dependency>
       <groupId>io.specto</groupId>
       <artifactId>hoverfly-java-junit5</artifactId>
       <version>0.11.5</version>
       <scope>test</scope>
   </dependency>

Hoverfly Modes

Apart from Playback (in Hoverfly is called Simulate) and Record (in Hoverfly is called Capture) modes, Hoverfly also implements other modes:

• Spy: Simulates external APIs if a request match is found in the simulation data, otherwise, the request is forwarded to real API.
• Synthesize: Instead of looking for responses in simulation data, the request is passed directly to middleware (a defined executable file), which takes care of consuming and generating the required response.
• Modify: Requests are sent to middleware before forwarding it to the destination. Responses will also be passed to executable file before being returned to the client.
• Diff: Forwards a request to an external service and compares a response with currently stored simulation. With both the stored simulation response and the real response from the external service, Hoverfly is able to detect differences between the two. These differences are stored to be consumed in the future.

Hoverfly Example

To show how Hoverfly works , let’s suppose we have a service A (Security service) that needs to know the current time provided by another service deployed at http://worldclockapi.com. When you do a GET request to http://worldclockapi.com/api/json/cet/now next JSON file is returned:

{
   "$id":"1",
   "currentDateTime":"2019-03-12T08:10+01:00",
   "utcOffset":"01:00:00",
   "isDayLightSavingsTime":false,
   "dayOfTheWeek":"Tuesday",
   "timeZoneName":"Central Europe Standard Time",
   "currentFileTime":131968518527863732,
   "ordinalDate":"2019-71",
   "serviceResponse":null
}

The important field in the document is the currentFileTime which provides the full-time information.

The gateway class in charge of communicating to the service and return the current time looks like:

public class ExternalWorldClockServiceGateway implements WorldClockServiceGateway {
   private OkHttpClient client;

   public ExternalWorldClockServiceGateway() {
       this.client = new OkHttpClient.Builder()
           .connectTimeout(20, TimeUnit.SECONDS)
           .writeTimeout(20, TimeUnit.SECONDS)
           .readTimeout(30, TimeUnit.SECONDS)
           .build();
   }

   @Override
   public LocalTime getTime(String timezone) {
       final Request request = new Request.Builder()
           .url("http://worldclockapi.com/api/json/"+ timezone + "/now")
           .build();

       try (Response response = client.newCall(request).execute()) {
          
           final String content = response.body().string();
           final JsonObject worldTimeObject = Json.parse(content).asObject();
           final String currentTime = worldTimeObject.get("currentDateTime").asString();
           final DateTimeFormatter formatter = DateTimeFormatter.ISO_OFFSET_DATE_TIME;
           LocalDateTime localDateTime = LocalDateTime.parse(currentTime, formatter);
           return localDateTime.toLocalTime();

       } catch(IOException e) {
           throw new IllegalStateException(e);
       }
   }

}

The important part here is that the URL is not a parameter. I know that in a real example this information would come from a configuration parameter, but for the sake of simplicity and also because you can figure out that Hoverfly is proxying all network communications, the URL is hardcoded.

Let’s start to see some possible scenarios and how to test this ExternalWorldClockGateway class.

World Clock Service not developed yet

If WorldClockService has not been developed yet we need to use Hoverfly in simulation mode and provide a canned answer.

@ExtendWith(HoverflyExtension.class)
public class ExternalWorldClockServiceGatewayTest {

   private static final String OUTPUT = "{\n"
       + "   \"$id\":\"1\",\n"
       + "   \"currentDateTime\":\"2019-03-12T10:54+01:00\",\n"
       + "   \"utcOffset\":\"01:00:00\",\n"
       + "   \"isDayLightSavingsTime\":false,\n"
       + "   \"dayOfTheWeek\":\"Tuesday\",\n"
       + "   \"timeZoneName\":\"Central Europe Standard Time\",\n"
       + "   \"currentFileTime\":131968616698822965,\n"
       + "   \"ordinalDate\":\"2019-71\",\n"
       + "   \"serviceResponse\":null\n"
       + "}";

   @Test
   public void should_get_time_from_external_service(Hoverfly hoverfly) {
      
       // Given

       hoverfly.simulate(
           SimulationSource.dsl(
               HoverflyDsl.service("http://worldclockapi.com")
                   .get("/api/json/cet/now")
                   .willReturn(success(OUTPUT, "application/json"))
           )
       );

       final WorldClockServiceGateway worldClockServiceGateway = new ExternalWorldClockServiceGateway();

       // When
       LocalTime time = worldClockServiceGateway.getTime("cet");

       // Then
       Assertions.assertThat(time.getHour()).isEqualTo(10);
       Assertions.assertThat(time.getMinute()).isEqualTo(54);

   }

}

The important part is the simulate method. This method is used to import a simulation of the interactions between the test and the remote Hoverfly proxy. In this concrete example, it configures Hoverfly proxy to return a canned answer instead of forwarding the traffic out of the local host when it receives a GET request to the endpoint.

World Clock Service developed and running

If service is already running, you can use capture mode to generate the initial simulation data set instead of having to generate them manually.

@ExtendWith(HoverflyExtension.class)
@HoverflyCapture(path = "target/hoverfly", filename = "simulation.json")
public class ExternalWorldClockServiceGatewayTest {

   @Test
   public void should_get_time_from_external_service() {
      
       // Given

       final WorldClockServiceGateway worldClockServiceGateway = new ExternalWorldClockServiceGateway();

       // When
       LocalTime time = worldClockServiceGateway.getTime("cet");

       // Then
       Assertions.assertThat(time).isNotNull();

In this test case, Hoverfly is started in capture mode. This means that the request happens through the real service, and the request and the response are stored locally to be reused in simulate mode. In the previous test, simulation data is placed at target/hoverfly directory.

Once the simulation data is stored, you can switch to simulation mode so no further communication with real service happens again.

@ExtendWith(HoverflyExtension.class)
@HoverflySimulate(source =
   @HoverflySimulate.Source(value = "target/hoverfly/simulation.json",
                            type = HoverflySimulate.SourceType.FILE))
public class ExternalWorldClockServiceGatewayTest {

   @Test
   public void should_get_time_from_external_service() {
      
       // Given

       final WorldClockServiceGateway worldClockServiceGateway = new ExternalWorldClockServiceGateway();

       // When
       LocalTime time = worldClockServiceGateway.getTime("cet");

       // Then
       Assertions.assertThat(time).isNotNull();

HoverflySimulate annotation allows you to import simulations from file, classpath or URL.

You can set HoverflyExtension to switch between simulate and capture mode automatically. If a source is not found, it will run in capture mode, otherwise, simulate mode will be used. This means that you don’t need to switch from using @HoverflyCapture and @HoverflySimulate manually. This Hoverfly feature is very easy yet very powerful.

@HoverflySimulate(source =
   @HoverflySimulate.Source(value = "target/hoverfly/simulation.json",
                            type = HoverflySimulate.SourceType.FILE),
                            enableAutoCapture=true)

Detecting outdated simulation data

One of the concerns you face when using service virtualization is what happens if your simulated data is stale, so although all your tests are green, you may get failures when running your code against the real service.

To detect this problem, Hoverfly implements Diff mode which forwards a request to the remote service and compares a response with stored data simulation. When Hoverfly has finished comparing the two responses, the difference is stored and the incoming request is served the real response from the remote service. After that, with Hoverfly Java you can assert that no differences are found.

@HoverflyDiff(
   source = @HoverflySimulate.Source(value = "target/hoverfly/simulation.json",
       type = HoverflySimulate.SourceType.CLASSPATH))

Usually, you don’t want to run Diff mode all the time. In practice, it will depend on a number of factors, for instance, if you control the remote service or not, or if it is been heavily developed or not. Depending on the circumstances you will want to run tests in Diff mode once per day, once per week or once every time you are planning to do a final release.

A typical workflow for this verification task is:

1. Diff mode tests are run.
2. If there is a failure then remove the outdated simulation data.
3. Trigger a new service build job to force Hoverfly to re-capture the simulation.
4. Build might be failing because the newly captured data is different than the previously captured data. Then developers see that the service is failing and they can start fixing the code to adapt to the new service output.

Delaying response

Hoverfly allows you to add some delay before sending back the response. This allows you to simulate latency and test that your service deals with it correctly. To configure it you only need to use the Simulation DSL to set the delay to apply.

hoverfly.simulate(
           SimulationSource.dsl(
               HoverflyDsl.service("http://worldclockapi.com")
                   .get("/api/json/cet/now")
                   .willReturn(success(OUTPUT, "application/json")
                       .withDelay(1, TimeUnit.MINUTES)
           )
       ));

Verifying

I have mentioned that you can think about service virtualization as mocking for the enterprise. One of the most used features of mocking is that you can verify that a concrete method has been called during the test phase.

With Hoverfly you can do exactly the same, verifying that specific requests have been made to remote service endpoints.

hoverfly.verify(

           HoverflyDsl.service("http://worldclockapi.com")

               .get("/api/json/cet/now"), HoverflyVerifications.times(1));

The previous snippet verifies that gateway class has reached the endpoint /api/json/cet/now from host worldclockapi.com once.

More features

Hoverfly implements other features not shown here but useful in some cases.

• Request field matchers: By default, the DSL request builder assumes exact matching when you pass in a string. You can also pass a matcher so the matching is not strict (ie service(matches("www.*-test.com")))
• Response Templating: When you need to build a response dynamically based on the request data, you can do so using templating.
• SSL: When requests pass through Hoverfly, it needs to decrypt them to persist (capture mode) or to perform matching. So you end up with SSL between Hoverfly and the remote service, and then SSL again between your client and Hoverfly. To make this process smooth, Hoverfly comes with its own self-signed certificate which is trusted automatically when you instantiate it.
• Stateful simulation: Sometimes you need to add some state in responses depending on the previous requests. For example, after sending a delete request, you are likely to receive 404 error status error if the deleted element is queried.

SimulationSource.dsl(
           service("www.example-booking-service.com")
               .get("/api/bookings/1")
               .willReturn(success("{\"bookingId\":\"1\"}", "application/json"))

               .delete("/api/bookings/1")
               .willReturn(success().andSetState("Booking", "Deleted"))

               .get("/api/bookings/1")
               .withState("Booking", "Deleted")
               .willReturn(notFound())

In the previous code, when a request is sent to /api/bookings/1 using DELETE HTTP method, Hoverfly sets the state to Deleted. When a request is sent to  /api/bookings/1 using GET HTTP method then a not found error is returned since the state is Deleted.

Contract tests

Service virtualization is another tool that might help you with writing tests, but it is not a substitute for contract testing.

The main goal of contract tests is to validate that both consumer and provider services are going to be able to communicate correctly from the point of view of business, that both parties are following the contract they agreed to meet.

On the other hand, service virtualization can be used for:

• Testing services that are already created but they don’t have any contract yet.
• Testing integration with third-party services which we might not have the contract.
• Testing corner cases (delays, invalid inputs, …)
• Assisting you in writing integration tests where the client changes more often than the dependent service.
• Testing when the depending service is unavailable or it is expensive to do representative load testing.
• Spinning up services minimizing memory footprint on local laptop.
• Tests (and its data) are not hard to create or maintain as it happens with contract tests.

Conclusions

Service Virtualization (and Hoverfly) is another tool that you can use to test your services, specifically the communication between services, and you can do this using a somewhat “unit” test-based approach. I say “unit” because your test is not actually running the remote service, and so you are just testing one unit of the system. This allows you to test your communication layer without having to run the whole system, and you can do so without the time needed to initialize an entire stack. Notice that from the point of view of the consumer service under test, it knows nothing about provider service, and so a virtual service is no different than a real service. This, in turn, means that the consumer service does not know if it is living in a simulation or not.

With the advent of microservices architecture, service virtualization is an essential tool to avoid surprises when communicating with other services -- especially when working within an enterprise context with lots of dependencies. Service virtualization can also be used to remove the dependency on third-party services during your test phase, and for testing how your application behaves when it encounters latency or other networking issues. Finally, service virtualization can also to be used within legacy projects, in the case of having a monolithic application that requires access to a third-party service(s) that are challenging to otherwise simulate.

About the Author

Alex Soto is a software engineer at Red Hat in Developers group. He is passionate about the Java world and software automation and believes in the open source software model. Alex is the creator of NoSQLUnit and Diferencia projects, member of JSR374 (Java API for JSON Processing) Expert Group, the co-author of the book Testing Java Microservices by Manning, and contributor of several open source projects. A Java Champion since 2017 and international speaker, he has talked about new testing techniques for microservices, and continuous delivery in the 21st century. You can find him on Twitter @alexsotob.