When moving from a monolith to a microservices architecture, complexity implicitly hidden within the monolith becomes explicitly visible and the challenges for communication will grow exponentially, Michael Plöd explained in a presentation at GeeCON 2018, describing different strategies for communicating between microservices.
Plöd, principal consultant at InnoQ, started by noting that in his experience, teams currently too often see microservices as the default architecture. He emphasized that distributed systems are difficult systems; if you don’t need a distributed system you shouldn’t strive for one just for the sake of it. Well-built monoliths are often a better choice in such circumstances.
When the monolith isn’t enough and microservices are used, they must be integrated, and Plöd notes that this is not just a technical issue, it also has implications in other areas:
- Team communication: Teams often need to communicate to solve integration problems, which may lead to politics and governance issues.
- Coupling: It’s extremely important to achieve loose coupling between services, otherwise you will easily end up with a distributed monolith.
- Quality criteria: To find what's really needed regarding consistency, performance, scalability and robustness you should do an early assessment of your application.
- Technology: Although REST is commonly used, it’s not the only option for communication.
To help in team communication and manage coupling, Plöd notes that domain-driven design (DDD) can be of great help by finding boundaries at a business level – bounded contexts. These contexts commonly interact with each other in different ways, and to describe this a context map can be used. Patterns for these interactions include:
- Open host service, which specifies a protocol like REST for how a service can be used.
- Shared kernel, where two services share some code or a library defining the interaction.
- Customer / Supplier, where one service is a customer of another service and may thus influence how it’s implemented.
- Anticorruption layer, where a consuming service creates an adapter to minimize impact from another service that it interacts with.
Looking at the technical aspects of communication, Plöd first describes a more general categorization of communication – orchestration versus choreography. Using orchestration, one microservice knows about a process and actively calls other services in order to accomplish the task. Using choreography, a microservice publishes an event which other services react to, and act on. For Plöd this is an important distinction and he believes we should make an architectural decision about the preferred way of working within a system.
He also recommends thinking about both a macro and a micro architecture, and notes that the microservice architectural style is not about teams selecting anything they like, because then you will end up in total chaos. Instead, he recommends a macro architecture with rules that all teams must abide to. In the micro architecture, the teams have more freedom to select their own style of implementation.
Looking at the technical options of communication, Plöd notes four types:
- RESTful resources
- Messaging
- Domain events
- Feeds
Although REST is widely understood today, in Plöd’s experience very few teams implement it well, especially with regard to hypermedia and multiple representations. He notes that implementing a call to a RESTful resource is very easy nowadays but implementing a robust one that works in a microservices environment is much harder. There are quite a few pitfalls and challenges to be aware of, including:
- Service discovery. A way for a service to find the URLs of services it communicates with.
- Resilience, including how errors and downtime is handled. A graceful degradation of services is important to avoid one malfunctioning service taking a whole application down.
- Load balancing, to be able to handle increasing load. This can be achieved in different ways, depending on the environment the application is running in.
The next option is messaging, where microservices send and consume messages, commonly through a message broker. Here the challenges described for REST is not as relevant:
- Service discovery is not relevant anymore since services only know about the messaging system.
- Resilience is mostly handled by the messaging system. The main risk is an increase in latency due to errors or downtime.
- Load balancing is done by scaling up the messaging system or by increasing the number of message consumers.
Plöd’s third option is domain events. They represent facts about things that have happened in the past at a business level. Using them for communication leads to event-driven microservices, a very popular architectural style today. When using events for communication he describes a few alternatives for the payload used in events:
- Full payload, where the event contains all data needed for dealing with the event, for instance all data about a customer. This makes it easier for the consumer, but it also means a tighter coupling.
- REST URL, with just an URL to a resource that represents the event.
- Empty, with only data about the event itself. A consumer then must have other means of finding the data it needs.
- Mix, with for instance a small amount of data plus an URL for finding additional data. This is the preferred way for Plöd in most cases.
Plöd’s final alternative is to combine REST and events by using Atom Feeds. A service now publishes a feed with events which consumers can subscribe to and asynchronously read. Using HTTP, it’s very easy to communicate in most environments and it’s possible to leverage features like ETags, Last Modified, pagination and links. Another advantage is that the services publishing feeds is completely decoupled from the consumers. It’s solely up to each consumer to read the feeds as it sees fit and to keep track of events already consumed.
In order to offer feeds, the events must be persisted, and this is where event-sourcing comes into play. We can use these events as our main persisted model which also enables us to use CQRS to derive views that are optimized read models from those events. Plöd emphasizes that CQRS and event sourcing are solutions that only come into play in specific parts in a system, they do not represent an architecture used everywhere in a system.
To learn more about integration Plöd highly recommends the book Enterprise Integration Patterns written by Gregor Hophe and Bobby Wolf.