For several years now we have been attempting to showcase lessons learned by various practitioners as they move to adopt microservices. Piotr Gankiewicz, a software engineer, embarked on a journey towards microservices and has now decided to share some of the lessons learnt on the way. Of course, as with all experiences, not all of it may be relevant to your own efforts but it's worth knowing about. As Piotr says:
Not so long ago, I’ve eventually decided to dive into the world of microservices. I did look for an opportunity to make use of this architectural pattern for quite some time and finally was able to do so. After three months of trying out the new things and learning stuff mostly on my own (the hard way) I believe it’s a good time to share some of my experience.
He begins by covering some core system design aspects, including API management (gateway), where he introduces the concept of a "service bus" without really defining it. He also discusses adding a Storage Service:
[...] in order to store objects in a read-only database (CQRS stuff here) you most likely need to subscribe to all sorts of the events like UserCreated, InvoicePaid etc. Then you need to talk to a specific (micro)service, fetch the data and save it into the database. In the former scenario your API would be responsible for subscribing the events, mapping the data and saving it into the database – is that wrong? Most, certainly not, however, I prefer the latter solution which is a full separation of the API from the microservices. In that case, there’s a so-called storage service that subscribes to the events, fetches the data from the (micro)services, knows how to flatten the objects etc. The API only needs to perform the HTTP requests to the Storage service in order to get the data – and it doesn’t really care whether it comes from some internal database or cache or some service that is located at the end of the world.
Finally before giving his lessons learned ("tips and tricks", as he calls them), he concludes the design aspects with a definition of Services, part of which includes:
Each (micro)service does possess its own domain models, repositories, business logic and so on. The only things that are common for the whole infrastructure are the service bus and a set of commands & events.
So on to the tips and tricks, only some of which we include here. First up is something for those who believe size is important for microservices: "Keep your services small".
It’s better to have many small microservices that focus on a single domain than a few bloated ones that perform totally different tasks & manage unrelated responsibilities under the same scope. The most usual examples would be: creating/validation the user accounts, sending the messages, managing products, handling the payments etc. and each domain would fit into a separate service with its unique entities.
Following on from what others have said about microservices, event sourcing and CQRS, Piotr believes CQRS is crucial:
Send the commands that have no return values, and perform queries that are idempotent – that’s all you need to follow the CQRS. If you stick to this pattern, you will quickly find out that it’s much easier to scale your applications simply by separating read & write operations.
Next up is back to data and how choosing a database for the service in question is critical, again similar to what others have discussed:
Each service (not a single instance, cause you may have many instances of the same service running on different nodes) should have its own database. Not only you will eliminate a single point of failure (a single, huge database for the whole system), but most importantly you will have a freedom to pick the best database for the particular tasks. You might want to use SQL for some financial operations which rely heavily on transactions or NoSQL database for storing billions of JSON documents.
Piotr covers other things such as request tracking (he gives an example of how he has approached this in his journey), using an asynchronous messaging approach (with HTTP), ensuring new services are easy to deploy (possibly an oblique reference to CI and CD) and writing end-to-end tests. The last one to mention is: "Include failover, service discovery and other useful mechanisms":
Whenever something goes wrong, you want to make sure that your whole system doesn’t crash, or at least part of it. Make sure you have some retry policies included (e.g. Polly), service discovery tools such as Consul and also keep your credentials in some centralized place e.g. using Vault, Azure Key Vault or my open source project Lockbox.
Much of what Piotr discusses is similar enough to what many others have spoken about over the past few years, so perhaps we are moving towards a common understanding of what works and what does not work when developing with microservices. However, it is important to note that although Piotr has discussed some of his lessons learnt, as yet there is no indication of how well any of the applications he has developed are behaving (scaling under load, resilience etc.) Perhaps this will come later.
