Self Contained Systems (SCS): Microservices Done Right

Everybody seems to be building microservices these days. There are many different ways to split a system into microservices, and there appears to be little agreement about what microservices actually are - except for the fact that they can be deployed independently. Self-contained Systems are one approach that has been used by a large number of projects.

What are Self-contained Systems?

The principles behind Self-contained Systems (SCSs) are defined at the SCS website. Self-contained Systems have some specific characteristics:

• Each SCS is an autonomous web application. Therefore it includes the web UI as well as the logic and the persistence. So a user story will typically be implemented by changing just one SCS even if they require changes to UI, logic and persistence. To achieve this the SCS has to have its own data storage so each SCS can modify its database schema independently from the others.
• The SCS might have a service API. If the logic implemented in the SCS should be used by other systems or e.g. by a mobile app, they can use this API. The API is optional. If the UI in the SCS is the only client for the logic implemented in the SCS, there is no need for an API.
• The SCSs should have no shared UI. The idea is to implement UI, logic and persistence in one system and to keep changes contained in one SCS. A shared UI contradicts this approach. It is likely that a lot of changes would require changes to the service and shared UI.
• SCS should shared no business code. Common business code would cause tight coupling between services. Also it means that some logic for a specific use case is implemented in more than one SCS.
• A SCS is owned by a single team. A user story will be typically be implemented by one teams changing one SCS. The decoupled SCS are used to to ensure that teams can work independently.
• To make SCSs even more less coupled, shared infrastructure should be minimized. So while a shared database or a shared messaging system can be used, care should be taken: The availability of the SCS will depend on the availability of the shared infrastructure.

How to split a system into SCSs

To keep the coupling between SCSs as low as possible, each SCS should implement a Bounded Context in the Domain-Driven Design (DDD) sense. The idea is that every system does not just have one domain model but in fact there are different models. Each is valid in a Bounded Context. For example, in the product search of an ecommerce system information about the product such as the current price, the description and size is important. To ship the product to a customer other information is needed: The weight of the product and the delivery address of the customer. Splitting a system into Bounded Contexts is a great way to find candidates for Self-contained Systems.

Finding Bounded Context can be done by grouping user stories together. So for example searching for products by full-text search, by categories or by recommendations might be part of the same Bounded Context. Of course the split is not clear-cut - depending on the complexity the search might be split into multiple Bounded Contexts.

Also a user journey might provide ideas about a split into SCSs. The customer journey describes the steps a customer takes while interacting with the system e.g. search for products, check-out or registration. Each of these steps could be a candidate for a SCS. Usually these steps have little dependencies. Oftentimes there is a hand-over between these steps: The shopping cart is handed over to the checkout where it becomes an order, and is then handed over to fulfillment.

It is important that an SCS does not just handle a specific domain object. For example a SCS for customer data does not make a lot of sense: There will be customer data in many different Bounded Contexts. So coming up with a single model for a customer and implementing it in a separate SCS is not possible. Even if it was, each system will use customer data and so there will be too many dependencies to that system. This is also the reason why the split into SCS should be motivated by user stories, Bounded Contexts or the user journey - this top-down-approach will lead to a set of decoupled systems.

While it might make sense to identify common parts later on, this should not be the focus. Common logic might be separated in a different system, but that means that the SCSs will have a dependency on the common system, which means that they not that decoupled any more.

Figure 1: SCS are Bounded Context and contain logic, persistence and a web UI.

Communication between SCSs

Communication between SCS can be done in a variety of different ways, but there is a clear preference:

1. UI integration provides a lot of flexibility. In the simplest case an SCS would just render a link. This is very easy and often enough - also it provides a lot of flexibility. The linked page can be changed entirely or even be replaced by a PDF instead of an HTML document. However, sometimes a web page has to be composed from elements that are created by different SCSs. So for example, a landing page might show information from many SCS on different parts of a page. Links are not enough to create such a page instead transclusion has to be used i.e. HTML has to be transcluded.
   1. One approach is to do the transclusion in the browser and load HTML snippets via JavaScript. Basically any JavaScript library supports this. But transclusion can also be done on the backend.
   2. Edge-Side Includes (ESI) is a standard implemented by HTTP caches like Varnish and Content Delivery Networks (CDNs). ESI defines HTML elements that would be replaced by HTML loaded from other server. Server-Side Includes (SSI) is very similar but implemented by web servers like nginx or Apache httpd. So if there is already a web server that all requests are routed through, e.g. for SSL termination, that server could also do SSI.
   3. UI integration can lead to a decoupled system: They just render a part of the UI. So for example no data schema need to be defined. Also resilience is built in. If the integration is done in JavaScript and the integrated system is not available, the code is simply not executed.
2. Next is asynchronous communication. That also provides loose coupling: Usually events are sent in this way. The receiving system can decide how to handle an event, and so a change in the logic might only be implemented in the receiver. Resilience is also ensured - if a system is not available, it will eventually come up again and then process the messages. In other words, only latency is increased, but asynchronous systems need to deal with latency anyway.
3. As a last resort systems can communicate using synchronous calls. This is not forbidden, but strongly discouraged, because it provides tighter loose coupling and also systems must deal with other dependent systems being unavailable in order to avoid error cascades.

It is important to point out what synchronous and asynchronous communication mean in this case. Synchronous communication means that an SCS would call another system and wait for a response while it is serving a request. Asynchronous communication would mean that communication takes place while no request is served, or there is no need to wait for a reply.

It is possible to use synchronous protocols like REST for asynchronous communication. For example, data can be replicated while no requests are served using REST - this would still be considered asynchronous communication. It is also possible to send out events using REST, i.e. the Atom Web standards are often used for blogs. Subscribers can also poll a resource at a specific URL to find new blog posts. Of course this can also be used to find new events like new orders or the like. An alternative to REST are messaging solutions that provide asynchronous communication as a default.

Figure 2: SCS can be integrated on the UI layer, with synchronous or with asynchronous communication.

SCS vs Microservices

Self-contained Systems share a lot of characteristics with microservices: They both can be independently deployed and aim for decoupling. However, SCSs are defined much more precisely. Microservices might use any type of communication, they might or might not provide a UI and a specific user story might be implemented by several microservices working together. SCS are much more restrictive.

SCS focus on the strength of microservices. Decoupling and independent deployment mean that changes to a microservice do not influence other microservices. Therefore it is easier to change a microservices. SCS make sure that a feature will be implemented in just one SCS, and can therefore be brought into production with a single deployment. It makes little sense to invest in microservices and gain independent deployment if most of the changes require the deployment of more than one microservice anyway.

However, SCS are rather coarse-grained. Each SCS will typically implement a Bounded Context as defined by Domain-driven Design (DDD). That ensure that each business requirement will be implemented in one SCS. However, it might make sense to also include fine-grained microservices. For example, let's assume that the last step in an order process does a lot of calculations and should therefore be independently scalable. While all of the order process might probably be one SCS this last step could be implemented in a separate microservices. Then multiple instances of that microservice can be provided to ensure that the load can be handled.

So an SCS is at least one microservices. It is a web application and therefore independently deployable. But an SCS might contain multiple microservices - typically for technical reasons such as scalability or security.

Typical technology choices for SCSs

Every SCS might be implemented with a different technology. This is one of the benefits of microservices and also SCS - you can choose the best technology to solve the problem at hand. Because SCS are web applications, the SCS will probably be implemented with a web framework. Technologies in this area are well-known and in no way SCS specific.

However, as mentioned earlier SCS should not share a common UI. This rules out Single Page Apps (SPAs) that use multiple services. SPAs are very popular and implemented with frameworks like Angular. However, if one SPA is used as a frontend for many services, it is a shared UI. The architecture would therefore not conform to the SCS idea. The SPA approach means that changes will probably require a change to a service and the SPA. It is unlikely that a new feature will just modify the UI or just the service.

It is possible to combine SCS and SPA: Each SCS would have its own SPA. However, then switching from one SCS to another means leaving an SPA and starting another one - which might not be very quick and lead to a bad user experience. Note that SCS can of course still use JavaScript to provide a better user experience. However, the JavaScript code should be limited to UI enhancement and it should not be a shared UI. As SPAs are quite popular these days and hard to combine with SCSs, it sometimes difficult to make the case for SCS even though SCSs have many advantages.

The communication between SCS can be implemented with REST. In that case the infrastructure is simply just HTTP based - which will be needed for the web pages anyway. An alternative is a messaging system. If a messaging system is used, this must be highly available and scalable enough. If it crashes the whole system might become unavailable.

Challenges when implementing SCSs

Each SCS will bring its own UI. So to provide a common look and feel care has to be taken. Each SCS provides its own web interface and might therefore have a different design, look and feel. It might be enough to provide a style guide. The style guide defines the UI elements. It is not just limited to the visual design but also talks about the user experience and usability. Without a style guide it is very hard to create a UI for a complex system - whether it is implemented as a deployment monolith or as a set of SCS.

The style guide can require assets such as fonts, icons, CSS to style HTML or JavaScript to implement more advanced UI elements. In a deployment monolith all of this would just be integrate in the system like all other code. However, each SCS renders its own web UI and should therefore have access to these assets. An easy way to achieve this is an asset server that provides these assets. An alternative is an asset pipeline - each project would include the pipeline just like any other code dependency. This is an exception from the rule that SCS should not share code, but it is the price that needs to be paid for a cohesive across SCS.

Even though the look and feel of all SCSs should be as similar as possible a change to the look and feel and the assets should not be pushed out to the SCS. Instead each SCS should pull the new version as soon as possible. That ensure that each SCS is tested with the new version of the assets and actually works with them. Therefore the assets have to be versioned and older versions must be available for some time.

The integration on the UI level might seem easy, but it has some consequences. For example, running JavaScript code in the context of a different web page might not be possible because, for example, a different version of a JavaScript library is used in differing pages page. So JavaScript should ideally not be shared across SCS. Also care must be taken that the layout of included HTML code does not break the layout of the page.

While UI integration might seem easy in fact there are some considerations concerning the interfaces that need to be taken care of. Implementing SCS requires some skills in frontend development. In a way it solves the architectural challenges of splitting a system into SCS by relying on UI technologies, which in turn requires a specific mix of (cross-functional) skills in the team.

Developer Productivity vs System Complexity

Microservices usually come with additional complexity - more systems must be deployed and operated. SCS are microservices and therefore have some of that complexity too. However, SCS are coarse grained so the number of deployable artifacts will not be large.

In essence, SCS rely on the same infrastructure that web applications use. The complexity of the infrastructure is therefore not very high - if you can run web application you can also run SCS. It will just be more web application than before. This is one of the advantages of SCS - there is less need for advanced technologies. However, SCS still provide most of the benefits of microservices. Even an integration with REST or messaging is not always needed. It is interesting to see how much you can achieve with integrating systems just with links.

As SCS aim at containing every change in a single SCS the developer productivity should be quite high - changing and deploying one single SCS should be enough to handle most of the changes. Therefore, SCS provide a low technical complexity. In fact they are just web applications that we have been building for a very long time. At the same time they provide significant benefits.

When to use SCS

Obviously, SCS are only a solution for web systems. However, for other systems such an architecture based on the creation of not too many clearly separated systems each with its own Bounded Context, database and logic might also be a good option. Projects that build a portal without persistence and little logic can often benefit from the concepts of SCS i.e. the focus on UI integration.

It is worth noting that this architectural style relies on the concept of DDD Bounded Contexts, and such an architecture might be hard to migrate towards if the existing architecture relies on different approaches. Migration is an important factor that is often understated - a great architecture is worth nothing if it cannot be implemented. Finally, we have to mention that SCS require a good understanding about web applications. This should be commonplace, but with the rise of SPAs some of the fundamental ideas of web development are now not so well understood any more.

From our experience, SCS solve a lot of problems in complex web applications, and is often a very good starting point for the architecture. Not surprisingly there are a lot of projects using SCS - among them Otto, one of the largest eCommerce companies in the world.

Conclusion

Self-Contained Systems (SCS) are an architecture approach that takes ideas from microservice and combines them with approaches from classic web application development to gain a set of highly decoupled systems. Because SCS are essentially just web applications, most developers are familiar to the fundamentals of the approach. This way SCS provide a well-proven architecture in particular for web applications, and it might also be a good inspiration for other types of applications.

About the Author

Eberhard Wolff has 15+ years of experience as an architect and consultant - often on the intersection of business and technology. He is a Fellow at innoQ in Germany. As a speaker, he has given talks at international conferences and as an author, he has written more than 100 articles and books e.g. about Microservices. His latest book is about Microservices His technological focus is on modern architectures – often involving Cloud, Continuous Delivery, DevOps, Microservices or NoSQL.