Microservices — the Letter and the Spirit

[My writings are personal opinions and don’t represent my company]

The Microservices pattern is powerful and popular and will continue to grow in popularity because it resolves many challenges of modern distributed solutions. Some developers find that Microservices bring so much complexity where they are in a constant loop of fixing what the pattern is throwing at them instead of gaining the benefits it promises. Having been in several Microservices projects, some that failed, I thought it might be helpful to the community to write about the practical lessons: the ‘characteristics of well and poor implementation of Microservices’ (which was the original title for this article). Looking back at each project to see if I can abstract the factors that contributed to its failure/success I found an interesting fact, teams that understood Microservices to be a pattern of ‘small services’ ended up with complex and unmaintainable solutions, and teams who understood Microservices to be a pattern of ‘decoupled services’ managed to get the best out of it (the underlying understanding of the pattern (‘small’ vs ‘decoupled’) forces developers to take certain design decisions that are consistent with these objectives). With that in mind, I found that it will be more beneficial to discuss the root cause of well and poor implementations: ‘small-services’ vs ‘decoupled-services’ or what I’m calling here the ‘Letter’ vs the ‘Spirit’. Along the way we will also discuss some important concepts about the pattern.

The Letter: Microservices should be small

Per Wikipedia, ‘micro’ is a "prefix comes from the Greek μικρός (mikrós), meaning ‘small’". This unfortunate choice of words of ‘micro services’ led to so much confusion about the pattern as a lot of developers made cutting out the solution into ‘small’ pieces the end goal of the pattern while in reality it’s just a means to achieve the goals of the pattern.

I’ve seen this misunderstanding first hand few years back when I cloned a git repo of a previous client and found that every method (like GetName, GetEmail) resides in its own class (C#) and every class resides in its own project – there were literally few tens of projects that you needed to scroll with the mouse wheel in the Solution Explorer (Visual Studio) to see all the projects. I later knew they did this under the impression that it was the right way to do microservices.

This might be shocking at the first glance but it should come as no surprise if you know that many definitions online include ‘small’ as part of what microservices are. After all, if we understand microservices to mean small-services then we are faced with the inherent question of ‘how small should it be?’ and you might be tempted to suppress the need for a clear answer by making your solution/code as small as it can get so you don’t miss out on any of the benefits of microservices.

It is true that this is an extreme example but this antipattern exists, albeit in a more subtle nuances, in many other projects so let’s get this straight – the prefix ‘micro’ here doesn’t indicate an ‘absolute small’; they might have initially called it micro/small in the context of a ‘monolith’ to just mean ‘part’ of a whole (given that any part is ‘smaller’ than the whole it’s part of), or maybe they called it micro because we’re dividing the monolith based on a singular functionality (‘single’ functionality still doesn’t necessary mean ‘small’ functionality) or whatever other reason one can think of for why it is called ‘micro’, I believe all these explanations don’t capture anything essential about the pattern. We can’t change the name of the pattern but we can stop using "small" as part of defining it, we have to reconcile the definition of the pattern with the desired implementation of it as a first step in making the pattern simpler and easier to put to practice.

Please note, the fact that I’m saying microservices are not small-services doesn’t mean that microservices should be big-services – all what I’m saying is that the size is irrelevant here and captures the wrong aspect of the pattern causing the implementation to be skewed in the wrong direction.

If microservices are not small-services, what are they then?

Wikipedia has a good definition for microservices: "structural style that arranges an application as a collection of loosely-coupled services".

The idea of the pattern is simple – different parts of the solution might have different needs in which case we decouple these parts to serve the needs of each part individually without affecting other parts. As we decouple these parts we standardize the way they communicate with other parts and abstract away their internal implementations so these parts become interoperable and reusable.

Let’s look at an example.  

Figure A shows a monolithic solution, all services (represented in gear icons) in this solution are tight together. Let’s take this monolith as a baseline to understand why/when we need microservices.

Note: I was tempted to give these gears different shapes/colors to indicate different service types/functionalities but then decided to keep them all the same because I want to emphasize that a monolith doesn’t have to be a mix of different service types to take advantage of microservices pattern, your monolith could entirely be composed of one service type (e.g. all batch jobs) yet still be partitioned if one or a subset of these services have specific needs different from the rest of the services (e.g. they have a different resiliency or scale objective). This is a subtle but important thing to understand about the pattern.

Back to our example, let’s say one of the gears in this solution is experiencing a heavy load and we have a requirement to scale it, choose any of the gears from Figure A and let’s call it ‘GearX’. GearX is part of the monolith so we need to scale the entire monolith to satisfy our requirement of scaling GearX:

Mission accomplished! Figure B successfully scaled GearX and our solution can now handle a bigger/more workload. However, notice that as we were working to satisfy our scaling requirement, we inadvertently introduced a problem (a big problem in fact!) – as you can see from Figure B, the side effect of scaling GearX in a monolith is that all gears were also scaled too which is a massive waste of resources! In the on-premises days this might not have been a big problem because we invested capital cost in our data centers so we usually didn’t pay extra money when scaling apps. However, in the Cloud world, with the consumption cost model, we cannot afford to scale all these gears and pay for their consumption if all what we intended was just to scale GearX. Welcome to microservices, we now have our first valid use case to implement the pattern.

To resolve this issue let’s decouple GearX so that we can scale it independently from other gears – figure C:

Now we can scale up/out GearX without wasting resources, we now can satisfy our initial scaling requirement while maintaining optimal cost of running the entire solution –

Note: Scaling in Capital cost vs. Consumption cost models is the underlying reason you often see microservices associated with Cloud even though microservices architecture long predates Cloud and it is not the only valid architecture for cloud.  

I bet you can figure out where this is going: if we can partition out one of the gears to optimize scale/cost, why not take advantage of this same methodology to achieve other goals? Let’s say for example, some other gear in the monolith is a ad-maker handled by a separate team that operates in a different part of the world, they have frequent release schedule to generate daily ad campaigns, instead of being dependent on the release schedule of our monolith we can partition out that service and give the advertisement team full control over it, they could use whatever language/DB/release schedule, we wouldn’t care since this service is now decoupled/independent.

As we are decoupling these services and standardizing the way they communicate, we should aim for making them stateless (whenever possible) so they are easy to scale and replace with no/minimal downtime.  

In summary, don’t cut your solution to small pieces upfront as this will most likely make your solution harder to maintain, the process is quite opposite: think of your solution as one whole (monolith) then look at the requirements to see if you can partition out some parts of this monolith to serve these requirements. If there is a vague requirement that *might* be resolved by partitioning out some piece of the solution, my recommendation is not to prematurely partition it, just write your solution in accordance with good practices (SOLID / 12-factor ... etc.) so you can easily extract away this partition in the future if the requirement becomes concrete. Avoid Figure E at all costs!

The Spirit: Microservices should be decoupled

Back to the Wikipedia definition of Microservices: "structural style that arranges an application as a collection of loosely-coupled services" – let’s unpack the ‘loosely-coupled’ piece as this is the spirit of the pattern. While there could be many things that contribute to making your services loosely-coupled (a dedicated storage for each service, dedicated src repo ... etc), I want to make sure we’re standing on a firm foundation and mention the two primary things that are essential (i.e. you can’t do without these two principles but you can add to them):

1. Event-driven interaction between the services
2. Versioning your services

Principle 1: Event-driven interaction between the services

Simply put, if a call to ServiceA triggers a chain of calls to ServiceB and C and D where all of them must succeed for ServiceA to return a response then you’re not implementing Microservices the right way.

Ideally, services don’t interact with each other directly. Instead, they use some integration service to communicate together. This is commonly achieved with a service bus. Your goal here is making each service independent from other services so that each service has all what it needs to start the job and doesn’t care what happens after it completes this job. In the exceptional cases when a service calls another service directly, it must handle the situations when that second service fails.

Here is our monolith turned into a typical microservices solution:

Principle 2: Versioning your services

Simply put, if the team of ServiceA needs a meeting with the team of ServiceB to be able to change something in ServiceA then you’re not implementing Microservices the right way.

Microservices presents us with an interesting challenge – on the one hand, the services should be decoupled, yet on the other hand all should be healthy for the solution to perform well so they must evolve gracefully without breaking the solution. Let’s take a simple example:

ServiceA is a service that generates a json message with a username field as a string, it places this message on a queue. ServiceB is a service that expects a json message with a string username field to lookup the user and do some processing. Now assume that there is a requirement for ServiceA to include the username and their email address as an array in the json message. ServiceB will now fail to process this json payload as it is expecting a string for the username field. The team of ServiceA needs to be agile and respond to changes but at the same time they need to be careful not to introduce bugs like this one.

It might be tempting to resolve this by coordinating the teams who are working on these services and create one release to all services (when both ServiceA and ServiceB complete their changes) but you will quickly find that this is not maintainable, it is actually a famous grandma recipe for frequent bugs. More importantly, if you take this approach it is worth revisiting the basic assumptions and ask yourself what was the benefit of decoupling these two services in the first place – bringing them home together might be the right thing to do when services can’t be loosely coupled.

Your best bet to resolve this challenge is to keep developing these services independently but version them so they become aware when a change happens. (Consequently, maintain some backward compatibility until all services stop using the legacy versions). In our simple example, ServiceA will keep generating a payload with username as a string (version 1) but will now add a version 2 with the username as array. ServiceB checks the version number and processes the messages it can handle (version 1) while working on developing their code to accommodate for version 2.

There are multiple ways to do versioning, any convention would do. I like the three digits semantic versioning 0.0.0 as it is widely understood by most developers and it is easy to tell what type of changes the service made by just looking at what digit of the three got updated. You can also have a look at how Microsoft versions NuGet packages if you need a more comprehensive versioning strategy.

Is versioning at day-one a premature optimization for microservices?

This is a healthy question to ask ourselves. I’m a skeptic of things we add at ‘day-one’, there is no exception in our case here, even though I’m recommending versioning be added at day-one I’m not recommending implementing microservices at day one unless there are clear benefits of doing so. Once we decided that microservices is the right design though, we automatically signed up for a set of known distributed-system challenges – partitioning the solution to different services necessarily means these services will evolve differently, we are resolving this challenge by introducing versioning. Moreover, it is hard to introduce versioning later in the development phase, you would save yourself a lot of effort if you introduce it at day-one.

Conclusion

Defining microservices as ‘small services’ might not be fully wrong but it captures the wrong aspect of the pattern and leads to incorrect implementation. ‘Decoupling the services’ is the spirit and the central idea of the pattern. There are two essential decoupling principles: 1) Services shouldn’t directly communicate with each other and 2) Services should be versioned so they can grow independently from each other.

About the Author

Alaa Tadmori is a Cloud Solutions Architect at Microsoft. He is an early adopter and enthusiast of the cloud computing model. Alaa believes in responsible IT, the solutions we’re building today are shaping our world and will be the world of our children so we have a responsibility and a choice everyday to make our world a better place. When not at work Alaa likes to spend time with his family and read nonfiction books.