Reviewing the Microservices Architecture: Impacts, Operational Complexity, and Alternatives

Wes Reisz moderated an InfoQ Live roundtable on the impact of microservices, dealing with operational complexity, and alternatives to the microservices model. The participants were Leif Beaton (NGINX senior solutions architect), Yan Cui (independent AWS and serverless consultant), and Nicky Wrightson (Skyscanner principal engineer).

Peter Rodgers first introduced the term "micro-web-services" during a presentation at the 2005 Web Services Edge Conference, going against the conventional thinking at the time. The SOAP service-oriented architecture peaked in 2005 and Rodgers was arguing for RESTful services. He described in his presentation how a well-designed micro-web-services platform combines the underlying architectural principles of the web and REST services in Unix-like scheduling and pipelines to provide flexibility and simplicity in service-oriented architectures.

After six years of innovation and thought about this problem, a May 2011 workshop for software architects coined the term "microservice" to describe what participants were seeing and employing. By the next spring, the community embraced "microservices" to describe this architectural style.

James Lewis and Fred George gave presentations. Adrian Cockcroft was actively developing these systems at Netflix and described these as "fine-grained service-oriented architecture".

Other thought leaders in this new space included Sam Newman, Evan Bottcher, Martin Fowler, and Graham Tackley. Some of the very first presentations on microservices, even before the name evolved, took place at QCon.

Microservices found its stride with the introduction of DevOps. It has become more popular for how teams build systems, particularly continuously deployed systems.

However, it does come at a cost, and we're going to focus on those costs and whether or not microservices is worth the tradeoff.

Wes Reisz: One of my favorite microservices definitions is Adrian Cockcroft’s finely-grained service-oriented architecture in a bounded context. How do each of you define microservices?

Wrightson: I think I've struggled with this over the years because I didn't actually understand anything to do with DDD for the first three years I was working in microservices. Nor did our architects, which led me to a very specific, "smallest, most independently deployable piece of the code". It was wrong. I like to bring it back to that bounded context idea. It's not the same idea, but it's certainly talking about that context in that domain, and not having dependencies out. It's completely isolated.

Cui: My view of microservices is very similar, around bounded context. In fact, often I find myself using microservices and bounded context interchangeably. Certainly, I think, Lambda functions tick all the boxes of what you have considered as microservices, from an infrastructure point of view. One function doesn't do everything; it’s one small piece of a bigger machine and everything has to work together in perfect synchrony for something to happen. The thing that actually happens is what you want from the business point of view. Again, when I think about microservices, I think about a scope of responsibilities that are self-contained. Within that scope, one team or one person can own everything, and can consider, move, and reorganize things however they like. Outside of that, everything should be done via messages. There should be as few direct dependencies as possible because otherwise you create those hard couplings that create cascade failures in more complicated environments.

Reisz: It has been about eight years since microservices spun up. Considering the technology, early adopters, early majority, late majority, laggards, etc., where on the adoption curve do you think microservices are today?

Wrightson: Surprisingly, I think that people are not as far along as I always expect. We've jumped over that chasm, but there are many early adopters out there. I think they're going to struggle still with the rest of it. It's not because of microservices. It's all that goes along with it. You need fundamental organizational shifts and mind shifts. You've got a lot of organizations out there that are still on-premises. They're not even thinking about the cloud and the technologies that go along with that. They're not thinking about DevOps and continuous delivery.

One more thing about microservices: it solves an organizational problem. It doesn't solve a technical problem. You've got to be able to fold in that organizational shift before you even go down this road. That's what I feel stops late adopters, because you've got so many people out there running very complex monoliths in very traditional organizations.

Cui: I would say that we are still in the early-majority phase as well. I work with a lot of customers who are moving straight from on-premises or at least monolithic systems to serverless, and who have to learn about microservices, how to decompose the problem domain. Many challenges come with microservices from the monitoring, debugging, and observability angles. Certainly, there are many more of those companies out there than I thought a few years ago, until I started working with more companies from all walks of life. Then I realized that not everyone's doing microservices yet.

Reisz: Nicky, you hit on "organizational". When I was first trying to understand the idea of microservices, I found a blog post by Martin Fowler. I remember this picture that vividly sticks in my head that said, "You must be this tall to be able to operate or run microservices." Where's that bar today? What do you need to do to be able to operate microservices successfully? And is that bar higher or lower than it was?

Wrightson: I think what I was alluding to earlier is that, organizationally, we have microservices so that we can deploy discrete changes quite rapidly. The point at which that becomes an issue is when your company has grown to a certain size and you want to have teams and people with that granular autonomy. What size organization? That still sticks.

I think, in some ways, the landscape has gotten more complex. For every complex problem, there have been many simple solutions rolled out. If there have been 12 simple solutions rolled out for that complex problem, you then have to understand all 12 of those products to solve that problem. It goes on and on. We're putting a lot of tooling out there. We're greatly increasing the amount of understanding that you have to have about your landscape. I think that bar is still hovering in a similar place. The breadth of what you need to look at is going slightly wider.

Cui: I'd like to echo what Nicky said about the tooling. I'm certainly not a fan of where the DevOps ecosystem is in terms of tooling and how obsessed everyone is about which tool to use. That really bugs me to hell. The other day I saw a promo for KubeCon, and there was one-and-a-half minutes introducing every single tool to use in your Kubernetes cluster for monitoring, controlling, and all the options.

Reisz: The options and choices you have to make.

Cui: Yes, exactly. I think a lot of the technology that's come along, that's supposed to help you build microservices architectures, has actually made it a lot more complicated. The learning curve has gotten a lot deeper. At the same time, I also think that we are building more complex systems as well. Especially in the serverless world, I see a lot of very interesting and very complex event-driven systems, which brings in a lot of challenges that even API-centric microservices might not have to solve in terms of tracing in locations across all kinds of things like SNS, Kinesis, and DynamoDB streams, which are important. There are tools that are getting better at doing that now. It's still, I think, certainly from the observability side of things, more interesting now because people are building more complex solutions to more complex problems.

Reisz: Leif, how tall do you need to be to use microservices?

Leif Beaton: How tall do you want to be? There is a cost of purchase to microservices, in terms of a fundamentally different mindset about developing applications from a microservices perspective versus the old monolithic process. Things need to be a little bit more clearly defined up front than you can do with a monolith. As a small startup, you can start up with a monolithic approach and see where it leads you. With microservices approaches, that tends to become a little bit of a spaghetti soup. That's not to say that you have to be a conglomerate in order to avail of microservices. As long as you do the planning up front and decide on this scope for this part of the application and on that scope for that, then there are benefits for the smallest of companies and the largest one. I hasten to add that there are certain applications that should stay monolithic, though.

Reisz: You’ve started to hit on some of the problems with microservices, one of which, as Nicky says, is complexity. I have this vision of the Cloud Native Computing Foundation (CNCF) landscape: it is a monster. There are so many different possibilities out there. Just take Container Network Interface (CNI) with Kubernetes: you've got Canal, Flannel, Weave, Cilium... — all these choices on how you might do container networking. Complexity is obviously a factor in microservices. What are some of the other challenges?

Beaton: Overall governance is one. You touched upon something a little while ago about monitoring, tracing, and stuff of that nature. That's, of course, critically important when you start moving into this, because communication throughout the application is not a function call within memory and runs on the same application. It's a network call, which may or may not be locally on the system that you're talking to at the moment. Being able to monitor, track, and trace communications throughout an application is absolutely critical. Without that, you're essentially trying to find a black cat in a dark room while wearing a blindfold.

Reisz: Nicky, you talked about organizational maturity. Why is organizational maturity one of the challenges with microservices?

Wrightson: It drives, I think, being autonomous as a team and being able to take ownership. That I think is needed because you need to take the operations of that set of microservices or your domain as well. You need to be able to do that. Organizationally, it means that you can own the purpose of that domain. Microservices means that you don't need to ask the payments team if you can deploy it, because you no longer have those dependencies. It's quite a hard thing, culturally, to do.

Reisz: Yan, you and I talked a bit recently about orchestration, about some of the challenges on making things talk together when you're in a microservices environment. Can you talk a bit about some of the challenges around orchestration with services?

Cui: I think that our conversation was around how to implement the business workflows using orchestration versus choreography, where the choreography is the name of the game, especially in the serverless world where event-driven has become almost the norm. Certain problems are just not worth the cost that you have with event-driven architectures where all the tracing, all the monitoring becomes a bit more difficult. There's no central point where you can say "Okay, that's how the workflow works," because it's never source-controlled anywhere. These all just exist in someone's mental model of how a system works.

Certainly, within microservices, when you're inside a bounded context of one microservice, orchestration is absolutely fine, using tools like Step Functions that give you the orchestration engine that solves a lot of problems and makes a certain class of problems very easy to implement, monitor, debug, and trace. For things like payment or any business critical workflows, I prefer to use something like Step Functions to capture the actual workflow itself and make sure that it's source-controlled and everything.

Reisz: As you've mentioned event-driven architectures, I want to ask you how  event-driven architectures are related to microservices. Can you take one step back now and talk a little bit more about event-driven and microservices?

Cui: I think when people think about microservices in their early days, they're thinking about one service calling another one via some RESTful API, and then waiting for a response. That's very much the request-response pattern. They very soon realize that, actually, there are some problems because they're waiting for something to respond but they've got things down — what can they do? Chances are, you probably are going to go down yourself. You have to do a lot of other things to prevent those cascading failures.

Also, many things just don't need to happen synchronously. Event-driven systems take a lot of those synchronous processes and try to drive them off of some event bus or some event queue instead. Instead of me doing my thing, I just process an order. Now I have to tell the promo-code system to generate a promo code and then tell some other system to do its thing. Instead, I just publish the event to some centralized event bus that says, "An order has been processed successfully." Then the payment system or the promo-code system can listen to the events that they care about and they can do their own thing.

Microservices and event-driven architectures are a good mix, because we talk about microservices being self-sufficient, being self-contained, and you have to constantly react to API calls from other systems or you have to call other API systems, which creates tighter coupling with other systems. Whereas, if everyone just listens to events and you publish events for things that you've done, then everyone can be much more self-contained and much more loosely coupled. It's a great way to build microservices, especially in terms of the communication between different services.

Reisz: We've talked about organizational and complexity problems. We've talked about problems around choreography and orchestration when it comes to services. Given all of these problems, given all the choice and all the complexity involved in microservices, should we just be building monoliths?

Beaton: There are monoliths that should stay monoliths, that shouldn't be decomposed and exploded out into microservices. That being said, there's a tremendous array of benefits to a microservices approach: most obviously, having more control over what a specific component is doing, a more narrow scope of what a specific component is doing. Quality control over that becomes so much easier. Quality control over the overall application becomes easier too because when you update the search system or the payment system or whatever, you don't have to do full quality assurance on everything. You just need to focus your efforts on the component in question.

There are also less tangible or perhaps less obvious benefits. The freedom of choosing whatever technology or language is appropriate for the problem at hand is brilliant. It allows you to mix and match technologies to a much greater extent than would ever be conceivable in a monolithic world. That, of course, also means that sourcing talent becomes less of a challenge, because if you have two equivalent tools or languages for a specific problem, you double your portfolio of developers.

Wrightson: I think there is no right or wrong answer for this. I don't think a company needs to be one thing or another. They can actually have different areas of the business operating in different manners that suit them. It comes back to that organizational factor. Can you deploy rapidly? Can you iterate rapidly? Because sometimes you can end up having a beautiful microservices world and not being able to do that continuous deployment and continuous delivery that you're actually aiming for. Often, that's when you think you could do it quicker with a monolithic application. As long as you've got contracts between them and you’re leveraging messaging as much as possible, you can have monoliths living with microservices. I think you need to be very thoughtful about what you expect from going to a microservices world. I think people like to get on the new thing. I've seen people jump with both feet into the deep end of microservices and the teams end up spending 95% of their time trying to wrangle infrastructure and 5% of the time developing features. It can flip-flop easily.

Reisz: I think Sam Newman said something along the lines of "Build monoliths until you can't." When you have a need to move beyond it, then start to do that. You don't need to introduce complexity for no reason. Yan, what are your thoughts?

Cui: We talk about the right tool for the job. Microservices, all these other things — they're just tools. Every tool has some tradeoffs, some pros and cons. You’ve got to weigh them against your specs and see whether or not the cons are worth your while. If you're a large company and want to move fast, you want to reduce the number of failures one team can impose on other teams. Here, microservices is probably a good fit for you as you can isolate the teams so that one service going down is not going to bring down your whole empire. Or you can allow more people to work on the system while being isolated from each other.

If your staff is two people working on a system, you probably shouldn't even think about microservices, not for a very long time. Just do everything monolithically and deploy everything monolithically until you need to scale up the organization. I think Nicky said at the start that microservices allows you to solve organizational problems, not technical problems. People have often told me that if you need to scale, you have to use microservices, which is just not the case. Look at something like Supercell: every game they make is going to have 100 million daily active users and it's all running off of one JAR. It's all one service, one JAR. You can absolutely scale monoliths. In fact, if you don't know how to scale a monolith, you're not going to know how to scale microservices. For the right context, microservices is a great tool.

Reisz: We talk about absolutes, microservices and monoliths, but there are middle grounds, Let’s discuss modular monoliths. Modular monoliths can, for example, be multiple JARs that you might pull together. Yan, what are your thoughts on a middle ground, like modular monoliths? Is this just a natural evolution or do we just go straight to microservices?

Cui: I used to build a lot of pretty big back ends for games with lots of users. It was all monolith but at the same time was well-modulated in the core level. What we're talking about here is modularization. You can get modularization at so many different levels. You don't have to have separate deployables, especially if everything has to get changed together. We talk about microservices but if every time you change something, you have to change three of them and deploy all three of them at the same time, then they're not really independent anyway. I certainly think that you don't have to go all the way to the purest version of microservices, where there's absolutely no shared database and everything is independently deployable, scalable, and all of that. Practically, there are many hurdles you’ve got to jump through.

Personally, I'm very much in favor of middle grounds, at least for now. As long as you understand the characteristics you want to achieve for your architecture you can choose the best way to get there. How you get there is maybe one step or maybe it's through some middle ground first. You have to make that pragmatic decision. Otherwise, you can put yourself into a real rabbit hole without getting anything out of your technical decisions.

Wrightson: I totally believe in pragmatism running the design. I think sometimes we lose track of that in our desire to do beautiful engineering work. I'm seeing people that don't recognize that they are writing microservices, because there's a whole breed of people that have come out of university, who are way younger than me, that don't realize that they're writing microservices when they're doing these complex distributed systems. The way that they're writing them is starting off with quite a modular baby monolith, a macroservice, and then pulling it apart when it starts to become unwieldy. There is that side of things. I've also seen these attempts at modular monoliths, but their trouble was the pull requests were stacking up, so the time-to-release cycle was about a month. That defeats both objects. You've ended up with the worst of both worlds.

Reisz: As Yan said, use the right tool for the right job. And let's assume you work with an application that gets lots of spiky traffic. How do you deal with having just milliseconds to scale in a microservices world?

Beaton: That's the Holy Grail right there. Ultra-low latency is what we're all trying to achieve these days. Our customers range from healthcare, to FinTech, to trading, to all these industries where ultra-low latency is absolutely critical. How do you combat that in terms of things spinning up and going down? You need a data plane that is configurable and has a lot of intelligence built into it, in terms of being able to determine not only that a system is up and running but also that the application code within that system is responding both predictably and also timely, and to phase it out if such is not the case. Having a data plane that is very, very flexible is critical if you want to start mixing and matching technologies as well, which we've been talking about with having monoliths coexisting with microservices. Perhaps introduce a triangular pattern and explode out some microservices to take over certain responsibilities from a monolith, and all these things.

Reisz: Nicky, Skyscanner uses thousands of microservices. How do you deal with latency?

Wrightson: We balance over different areas. When we're seeing one that's going a little bit wrong, we can send it quickly to another area. We're moving over to Kubernetes. We were on ECS prior to that. Trying to scale up quickly is just not easy. We try to make sure that we've got enough resilience in the overall picture to be able to take the load. We're constantly doing load tests against all of our stuff as well. We work in the travel industry, so you see a very weird spike. We have a very strange traffic pattern going on. It is very different from when I worked in content, but trying to constantly have load tests against that gives us some good understanding. We do have resilience in other regions.

Reisz: We've seen these talks and read papers and things about people decomposing the monolith. They say things like velocity or teams stepping on each other are the smells that lead us to decompose a monolith and go to microservices. But we're seeing Istio with istiod, for example, going back to more of a monolithic architecture. What are some smells that might indicate to you that maybe it's time to step back and start looking at a monolith?

Wrightson: When I was at the Financial Times, we took a load of our services and meshed them up together just because maybe we hadn't got DDD right. That might have been one thing. I think what I've seen elsewhere, as well, is the toil level of being able to roll out any form of change across these. If you've got one team supporting 89 services, and you keep rolling out features and each of those has a security patch, you end up spending more time rolling out security patches than actually running a business. I think that's a real smell when you're spending more time maintaining the code than building new features. I think you need to rethink things. That's the biggest smell for me.

Beaton: I just want to re-emphasize something Nicky mentioned earlier, which is one of the key components of doing it wrong, so to speak: this whole attitude of it's new and shiny and therefore must be better. That's not always the case. As a matter of fact, that's hardly ever the case. It’s true sometimes but, generally speaking, you need to have a reason to adopt a new technology beyond just that it's new. That's not good enough. If it solves a problem for you — sure, I'm on board with that. If you're adopting it simply because it's version 2.0 of whatever, that's not good enough.

Reisz: Let's wrap with final thoughts. I think Leif started us there. If it's just new and shiny, that's probably not a good enough reason to use it. We need to be solving real problems. Nicky, any final thoughts?

Wrightson: I don't think that there is an all-or-nothing answer to this. There will be cases when it's always worth it. The great thing now with executing and microservices is that it's cheap to take for a test drive, at least, to try to work out whether you can leverage some of the benefits. Don't underestimate the cost organizationally and culturally.

Reisz: Yan repeated it, but microservices solving an organizational problem and not a technology problem still resonates with me. Yan, what are your parting thoughts here?

Cui: Always aim for a desirable business outcome, not the technology itself. I don't want to say technology is not important but it's not why we're doing anything we're doing. It is about creating business value. You need to start from that and then work backwards to decide on the best thing for you to do to create the most business value.

About the Authors

Wes Reisz is platform architect at VMware and former VP of technology at Section (an edge-computing platform). Reisz also chairs the San Francisco edition of QCon. Before VMware, Reisz served as the product owner for all of the English-speaking QCon conferences worldwide, was a principal architect with HP Enterprise Systems, and, for over 13 years, taught as an adjunct professor at the University of Louisville (Go Cards!). In addition to his current roles, Reisz co-hosts of The InfoQ Podcast.

Leif Beaton is a Senior Solutions Architect at NGINX based in Cork, Ireland. Having worked in the IT sector for 20 years he has built a diverse background encompassing Security, Networking, Hardware and Software architecture, as well development. Leif interacts daily with NGINX customers, helping them translate their requirements into modern architecture.

Yan Cui is an experienced engineer who has worked with AWS since 2009. He has been an architect and lead developer in a variety of industries ranging from banks and e-commerce to sports streaming and mobile games. Cui is known for his serverless articles on Medium as well as his blog theburningmonk.com. Some of his work even made its way into the Serverless Well Architected whitepaper by AWS. He counts C#, F#, Scala, Node.js, and Erlang among the programming languages that he has worked with professionally.

Nicky Wrightson has extensive experience delivering large scale cloud native architectures previously at the Financial Times and now at Skyscanner. She passionately promotes operability as a first-class concern in developing these large distributed systems. She works on the data platform at Skyscanner where the huge scale means a whole different set of problems to solve while still striving to be operable, cost effective, and maintainable.