Transcript
Losio: We are going to talk about maintaining software quality with microservices.
I would like to just give a short introduction of the topic, why we discuss microservices, and also why we discuss about software quality with microservices. In the last, probably 10 years now, we've been discussing microservices. We have had a great adoption and greater adoption as well in enterprise. One of the major challenges in the migration, in moving from a monolith to microservices as well is keeping the software quality and making sure that you're not afraid of what you're going to do, and make sure that you're not going to be held back by fears or by the challenge you have ahead, and not do it.
Background, and the Microservices Journey
My name is Renato Losio. I'm a software architect. I work with microservices. Definitely, here are four experts. I'm not one of them. I'm an editor in InfoQ. I would like to introduce our four panelists. If you can just give a very quick introduction about yourself and where you are in your journey with microservices, and what you do.
Kaiser: I am an independent tech consultant from Hamburg, North Germany. My encounter with microservices started back in the early 2015, 2016, when I was at that time, a startup CTO, transforming a monolith to microservices. Today, I'm focusing more on socio-technical systems that are aligned to business strategies, but this journey of socio-technical systems itself that started with microservices and its challenges to find the right boundaries around your business domain and also the team boundaries. Currently, it's not prevalent, the microservices question that I have day-to-day, but they are part of my day-to-day work as well.
Kruger: My name is Nico Kruger. I'm a Senior Director of Customer Engineering at Rollbar. I spent the last 17 odd years working specifically with customers in the quality space, so helping people be it in the monolith or microservices, if you're doing medical device, if you're doing a high tech software or a Fintech software, helping them build the process around delivering quality software at the end. Really, part of my journey has been understanding how people have been transitioning to microservices and changing their approach to it. More importantly, is the end product still something of a very high quality, which our customers demand?
Abeysinghe: I'm Asanka Abeysinghe, Chief Technology Evangelist at WSO2. I'm coming from a strong product engineering and product architecture background. When it comes to microservices as a product company, we are building our products based on microservices concepts, because early days we used technologies like OSGi and componentize the stack. It was a smooth move for us from that type of an architecture into API led microservices. That's one part of my journey. The other part is products are ready to build applications using microservice architectures. As a consultant, I'm helping many organizations across the globe to build microservice architecture with our components, as well as various other technologies available in the industry.
Van Der Merwe: I'm Rehan Van Der Merwe. I work for a company here in South Africa. We're doing consultancy. The company's name is Swipe iX. I'm also an AWS hero similar to Renato. Then, I've worked on enough projects around microservices, and one of those, know how not to implement microservices. I blog and do presentations. I just have a passion for microservices, and especially serverless, and everything that AWS has to offer.
The Biggest Challenge in Ensuring Quality with Microservices
Losio: What do you think is the biggest challenge in keeping quality with microservices? That is basically the core part of our conversation.
Van Der Merwe: I think one that's overlooked a lot is actually not to do with the technology itself, but with the communications. When you take on microservices or decide to implement microservices, you have to ask yourself a few questions, like how many people are on the project? Are they going to be more than one team? That determines if you actually should use microservices. Because if it's only going to be one team and a few people inside that team working on all of the microservices at the same time, you might be better off actually not doing microservices. I think one of the key things that must be focused on is the communication between the teams, because now there's different types of schemas with everybody and communication between them, to communicate and get those services to work correctly. There's many other technical problems as well.
Kaiser: Related to the team perspective or team aspect, in terms of you were asking, biggest challenge of managing software quality with microservices. I have one point to add, that is also that it should not exceed the team's cognitive load. The amount of information the members of a team can hold in their working memories. Otherwise, if largely exceeding the team's cognitive load, it will lead to quality issues. That's also one team aspect. Also, another challenge related to software quality is the challenge of testing, in terms of integration testing, because now you have more moving parts than in a monolith. You have to integrate with your services, over the network with each other. Everyone has its own contract. How can we provide automatic testing with test automation? Having them in place, integrating the contracts of other services without breaking the entire system, for example. How much unit tests, integration tests, or end-to-end tests? How to balance for example testing with detecting issues faster in production, and something like that. That's all related towards testing and the challenge that come with microservices related to integration tests.
Kruger: I was working on a large project in Australia many years ago, and they were moving from a monolith into lots of microservices. One of the things that was striking to me was a couple of months down the line, the biggest problem was in fact that they had so many microservices and this contract between the services were constantly being broken. You constantly have stuff failing, not working, because so many changes and I don't know about it. I agree with lots of different teams, lots of moving parts and this contract between them as they communicate is often where stuff goes horribly wrong. That actually slows people down, because now all of a sudden to complete a process, be it at checkout, or whatever, you're interacting with 10, 20 microservices. It takes one of those to break. I do believe the communication and setting up that contract between them is going to help you with the first hurdle, I think, into getting the base set up correctly so you don't create more hurdles for yourself down the line.
Losio: I like that you mentioned going back a few years, because I think from an external point of view what I see, at least the last few months, is a lot of discussion about, it is over 10 years that we have been working with microservices, that's great. The other side is a bit of a rejection, a lot of noise on Twitter, or wherever else about, "No, it's time to go back to the past. That doesn't work." It's a confusing moment in terms of that side.
Kruger: I think it's a very good point. The problem is we take a concept and we run with it. I was speaking to a customer on AWS the other day, they have 150 serverless things living. We take this concept too far. I think sometimes you got to stop a bit, because people just take a technology and run and say, what is the problem I'm trying to solve? What's the best tool in my bag to solve that problem? It may not be a microservice, it might be a thing that's a bit bigger, but is it solving the problem. Because at the end of the day, we're building stuff, solution software to solve business problems. I think that's important. Figure out the problem you want to solve first, and then go pick the right thing in your tool bag to go and address that problem.
The Place of Microservices in the Future
Losio: We said, where we started in the past. What the challenge is now, in this discussion, is maybe we have to go back or maybe we have to go forward. Do you see that topic still growing as a discussion in 5, 10 years, or do we expect that in the future, basically, microservices will be part just of our life, and we stop simply discussing about that?
Abeysinghe: This is something everybody is thinking, where these things are going to be. I believe it's the future, because if you look at it from two sides, one is how the teams are organized. I think we talk about teams a little bit, these autonomous two-pizza teams, distributed. With the pandemic, it's more distributed now. That's one side of the story. Then there's technology-wise, I think, even since we didn't do SOA properly, microservices came as a concept. Then again, technology-wise, containers, container orchestration systems, service meshes, all these things increase the ability to build proper microservice architecture. I think the world is moving in that direction. Again, if you look at it, why we are doing this thing, end of the day, we are building a digital experience for our consumers. Consumers are looking for a better experience, so we need to quickly deliver a better experience for the consumers. Having this componentized microservice based architecture is the only way to handle it. I think that is the future and it will improve.
Some of the things that we discussed here, like dependency management, observability, all these things will get solved. I think even with the current technology stack, we can do a certain amount of work, but it's about the planning, communication, and then having automation up to a certain level, as well as stick to basics because now we have a unit. A unit is a microservice. Then based on that, we can build all the fundamental stuff that we used to do in software engineering, properly. I think that's the way the technology is moving. We have to stick to that as well as we have to improve and fix these little issues that we are having in the architecture, as well as in the development lifecycle.
Migration to Microservices, and its Correlation to Public Cloud
Losio: One thing I noticed in all your feedback is out of nowhere without asking anything, I already heard the different names of cloud providers or mentioning public cloud. I wonder how much moving to microservices is as well. I'm not saying part of the journey to public cloud or whatever, but how correlated is that? Do you need to go to public cloud, whatever it is: Amazon, Google, Microsoft, or something else, or it's just a common thing, or it's basically two parts of the same equation?
Kaiser: What I encountered was when starting the journey to microservices, it comes with a lot of infrastructure and operational complexities. It enables us to offload some of the infrastructure complexities to the cloud providers themselves. Public cloud services make the transition to microservices easier because it lets us offload and differentiating heavy lifting, and let us focus on our core domain and provide solutions to solve our users' needs instead. Shift our focus back on what terms we are providing competitive advantage to our customers. I think that the adoption of cloud services is helpful. It depends on what your competitive advantage is. If your competitive advantage is, we are providing our solution as on-premises running on client's infrastructure, then it might be a different perspective. It lets you focus back on your differentiating aspects, and then using cloud services for the infrastructure components.
Kruger: From our perspective, what we're seeing is people are doing exactly that, offloading the work to a cloud provider, because that's not what they're good at. No software engineer ever wants to manage infrastructure. They don't want to do that stuff. We are seeing a big gap of where everything is now in code, all these microservices Lambdas, and it's getting harder to actually figure out what's going on. Because all of a sudden you have so many of these microservices and to know when something is going horribly wrong, and to know it quickly is actually becoming a bit of a problem. Getting that quality right up front, that could slow you down a bit. You've got to figure out, what's your tolerance for risk? What level of testing do you want to do before you push live? What's your tolerance for risk, if something does go wrong? How does your architecture then allow you to either roll back quickly or turn things on or off in semi real time, or real time.
That's definitely what we're seeing from our customers is they need that confidence when they're pushing multiple microservices live, oftentimes 10, 20, 30, 50 times a day. The release cycle is so hectic, and so large, that the amount of change is just tremendous. We're seeing a big need, like I don't want to do infrastructure, but now the code is doing everything, so can it help us with managing the quality of that piece? Yes, definitely we believe that nobody here wants to manage infrastructure. Yes, definitely going to cloud and giving us that infrastructure is helpful.
Van Der Merwe: One of my points as well I think, is that the cloud makes it much easier, because it allows you to lift and shift your home into the cloud. Then from there, you can start easily breaking off your monolith and start to create a microservice architecture, because you can't just move directly, from the one to the other. It needs to be a gradual process. Otherwise, you're going to have to disrupt your business and your core logic of your applications that you're delivering. I think with that then also, you would either provision that lift and shift monolith, and then as you're strangling and as you are moving services out, you would see that now I can shrink my EC2 server, for example.
That's only something that you can do within the cloud, because you don't have any fixed contracts. You can do that on-demand for quite a few years, if you wanted to. You can even do then later reserved instances, if you see that, ok, now there's only a small part of this monolith that I left, and that becomes a service. Before you know it, you then have microservices. Also, the clouds have lots of cloud native services that help you implement microservices, things like EventBridge on AWS, or SQS, or SNS, all those asynchronous platforms, like you can manage them that you don't want to manage. All of that is being done for you by these providers. You can almost then just focus on your business logic.
The Lift and Shift Approach to Cloud Migration
Losio: You convinced me that I want to move to microservices. I have my beautiful, big monolith sitting in my data center, or in my rack. If I understand, from your point of view, the first step should be, first I do a lift and shift. I'm not trying to reengineer anything yet. I first move it there where I have a bit more space to then take little pieces one by one away, and take advantage of the cloud. I don't go immediately to microservices in-house or try to do any hybrid scenario. Anyone disagrees on that? Anyone has any other approaches?
Abeysinghe: I think I can start with that, because I see that with many of the customers that I am working with as well. The cloud migration is a two-step process. First is moving to the cloud, and then application modernization. The first step is the hardest step to take. My advice is do the lift and shift, and then look at the application modernization. The most important thing is the gap in between. How long you will take to do the application modernization is key, because you can't delay that. If you delay it, then you will be paying a lot to your infrastructure provider, because monolith is not designed to use cloud infrastructure. As well as you will not be utilizing the platform services underneath provided by the cloud infrastructure as well. You have to do it wisely. I think to start, lift and shift is a great way to get started and get moving. Because the fundamental problem is cloud, sometimes based on the domain that you are in, you are scared to run some processes inside the cloud. You are scared to move some data into the cloud. I think first, as technologists, we need to break that barrier, and then tell the business, cloud is a viable solution for us, and then do the application modernization with that.
Limiting the Risk of Cloud Vendor Lock-in
Losio: I have two questions about it. One is, basically the way you talk sounds like it can be actually one of the major challenges in going to this process is actually the initial lift and shift. Not even just the modernization that once you've done that step you can take. The other one that goes back as well to the service mentioned before in terms of native services or whatever, is one of the pushback I usually have after a discussion is, ok, I want to have microservices. I want to just think about my product, whatever it is, but I don't want to be entirely coupled with a service provider because I have that feeling that I have my beautiful monolith in my data center. It's a little baby that is there and that I control. Now I split it in 20 different amazing services that cost me less, but I'm vendor locked in. Is there really an issue, first of all? Second, do you see how to limit that risk?
Abeysinghe: I think we are lucky today because we have an abstraction called Kubernetes. Then we have a deployable unit called Docker. With that, I think there's no such vendor lock-in.
Kruger: I think you got to be careful. History shows us that unfortunately, probably in this industry, we've had a trend of that, you can go back pace 400s, there's always going to be, vendors want to lock you in and try to make some money from that process. I think you're right, the fact that we can break these things down to the code level, as long as that code is not proprietary, it can be lifted and shifted to any cloud vendor: Azure, Google, whatever it is. As long as you can do that, then you avoid that situation where it's locked in. As long as that code is not using something that's so proprietary that you cannot move. I think we're getting to that point where in the cloud space people make that risk management decision to say this stuff has to be able to move if I want to service customers in the U.S., in a federal environment, I might need to run Azure. I couldn't do that with some of the other cloud providers. You have to have that flexibility to deploy your service maybe on their private instance in whatever cloud provider. I think we're getting to a point where we likely don't have it, but we got to be very aware of that. It is often a history that comes with our industry, unfortunately, in the past as well.
Losio: I actually want to ask Susanne, as I'm aware that there are countries like Germany where the topic is quite sensitive, the topic of public cloud. What's usually your advice in this sense? It is something that customers usually have against microservices, in general, the fact that they usually go into a public cloud as part of the deal.
Kaiser: Yes, in terms of like vendor lock-in?
Losio: Yes.
Kaiser: The thing is it comes with a cost if you try to, because that requires an abstraction layer on top of it. If you would like to mitigate the risk of vendor lock-in that means that you have to add an abstraction layer. Introducing this abstraction layer is also costly. What are the risks that you're really facing? I would like that to be my first question towards clients, if they say, vendor lock-ins. Then I'm usually asking this question, what is your fear? What risks are you encountering? Yes, then I can't move to another cloud provider. Why do you want to move to another cloud provider? Yes, they might increase the prices. When did they increase the prices lately? It's a chain of questions that I raise, in order to try to measure the risk. Also, to try to measure the cost that comes with abstracting it. You have to be aware of, and make a mindful decision regarding weighing out costs, because it's nothing for free.
Sometimes I'm also advising, for example, specifically with the microservice architecture, to go into the hexagonal architecture, where you have loosely coupled software components that are loosely coupled with your software environment. Where you can replace, for example, your adapters, and connecting your business logic remains the same. You don't have to readjust it for example when you go serverless. Where you have a deeper integration, compared to running on a Kubernetes cluster, for example, because you have to implement the serverless functions in AWS or in the Azure cloud itself differently. When they are really very afraid of vendor lock-in, I can suggest to go then with a hexagonal architecture where you can easily replace your adapters that are connecting your business logic with the outer environment itself. It introduces more objects if you go into object oriented architecture, so it comes with a cost.
Steps to a Microservice Journey, While Maintaining Software Quality
Losio: I would like to go back to both the topic of serverless and Kubernetes in terms of microservices. Basically, I want to go into my microservice journey, what are the steps? What do I need to do? Which processes do I need to have in place to maintain software quality?
Kruger: I think one of the things people forget is when you write software, before you write a line of code, figure out what it is you're building. I think sometimes we've lost this art of, we used to call it requirements building and all that stuff. That is so important, because many of the problems as we've learned in history are solved before writing a single line of code. It is important to know what you're building and why you're building it. I think once you get that process right, that's 80% of your battle won right there. I think that's the biggest part of it. Know what you want to build and design those endpoints, whatever they are, from a microservice, so how things should be tested. If you want to do some test-driven development, that could be an option as well. Put these things that you should be doing in place. I'm not saying over-engineer them, just enough, to get the service out in the right amount of time. If I can offer any advices, before you write a single line of code, make sure you know what you're writing. Why? Because I see a lot of mistakes where people rewrite the same thing three or four times, because they didn't truly understand in the first place what was supposed to be delivered as an end product.
Abeysinghe: I completely agree with Nico. I would like to connect it with something. Susanne talked about hexagonal architecture as well. My advice is, start with domain-driven design. This is a really good concept, a concrete concept to define the granularity of your microservices, identify the purpose, and divide it. A hexagonal architecture is great. I authored something called cell-based architecture, a little bit outside from the object orientation. I looked at the functional programming and then API-first approach. It's about that. Then you can design these microservices properly. Then, again, you can have a nice team concept around that as well, because domain creates a nice boundary for the team, as well as for your microservices. My advice for the developers, start designing from there, and then take it to the code level before writing code.
Tests to Maintain Microservices Quality
Losio: Which tests do you suggest using to keep good quality for microservices?
Van Der Merwe: From my personal experience, like within the service, you have to do unit tests. That's been easy at least. With these unit tests, you actually have to mock the other services, to have a test that isn't influenced by any of the architectures and boundaries. What gets difficult is that you can't just do unit test, you have to do end-to-end testing, like you have to test the whole process, through and through. Those get more difficult, especially when working with different teams and microservices, because your dev environment, for example, has to talk to the prod environment or dev environment. There's communication, and there's always a bit unsure about. It just makes that whole testing process more difficult than if it was something like that with a monolith. With that, I would definitely go with unit tests first, within the service. Try to mock something if you can. Then also do a second set of test, end-to-end test, usually in your pipelines, because each service being deployed has to be in a pipeline. Otherwise, you're going to be underwater to move very slowly. That's my advice.
Service Management by Teams at Different Size Companies
Losio: I actually have a follow-up, where we mentioned talking about different dev team or different teams or testing your whole component. I was wondering how the process is different in terms of managing microservices. I'm thinking about two entirely different scenarios. The large enterprise with a certain amount of microservices, but a team size and engineering size that is significantly larger than the number of services that you maintain. Versus maybe the small startup, the small company that is leveraging a significantly smaller number of services, but with a very small team. I was wondering if there's signals that different teams manage different services, so it's more like you're managing your own product in a large company, in your own team, and talking to external services. Or, do you see a different approach, or at the end of the day it's still the same from a quality perspective?
Kaiser: First of all, we also touched the topic about team boundaries, and as a team that we are owning one service only, so that we are not sharing services across multiple teams so that we have a very clear responsibility addressing. One team can own several services, but one service is not supposed to be shared across several teams. The services that a team owns, that they have the responsibility to fully test it. In terms of, as we already described with unit testing, with service testing, but also with stubbing, mocking, or using simulations for testing purposes itself. Or also consumer-driven contract testing, where we test the contract itself, from the consumer and the provider side itself.
When it comes to end-to-end tests, it's challenging. What version do you use from the other service, from the teams that are responsible for the other services that you integrate with? What version do you use? Does every team write their own end-to-end tests, or is it a fan-in pipeline where you run your unit, your service integration testing your own pipeline of your own team. Then at the end, there is an end-to-end test that every pipeline is fanned in, and run this, execute this end-to-end test at the end. Then it's a bottleneck that you introduce who's responsible for this end-to-end test. End-to-end tests are very brittle. They can fail quite easily. End-to-end tests become more brittle if you have so much services and so many teams trying to implement end-to-end tests. I would go further down to contract testing instead of end-to-end tests, because the contract itself can be then owned by a team itself.
End-to-End Tests and Service Coupling
Losio: End-to-end test sounds to me like expecting a certain coupling between the services.
Kruger: That's fundamentally one of the reasons Rollbar exists is because when we put the stuff into production that is when it breaks loose. Stuff breaks when one contract is being broken by somebody else. It's in that flow as a customer, I'm checking out. I'm doing a set of things that involves many microservices, and that's where the wheels come off. That's where it breaks. What people use Rollbar for is to mitigate that risk. You can't test everything. Can I know in production when that contract is broken, and why? What's that exception? What was the package it was expecting? It was not returning the right thing. That to me is a spot-on comment. That's where you can really solve those problems. If you actually know when these contracts are broken, meaning the return or what I was expecting is not of what it was coded to do in the first place.
Abeysinghe: I have an analogy for this, I feel like an environment with microservices is like a house with a bunch of teenagers, because you are in your room and then doing your own thing, but you're doing it better. The dinner table is the build system. You get together in that dinner and then you discuss. Discussions are the APIs. This unit should have a proper API that you can connect during that build system, and then start communicating with each other so that all the test systems should be incorporated with that build system. Then you can bring some discipline, because we are ahead to bring governance back again, at the level we used to have. Then again, we need to incorporate some governance and some discipline into the system. I think an automation build system is the solution for that. We need to plan that while we are planning the microservices on the development side. Having this switch DevOps, TechOps, practice build, provide a solution for that. That's how I treat it.
Cloud Providers and Testing
Losio: I was wondering if a cloud provider as well can help in that sense. For example, on AWS, is there anything that you can use to leverage a bit? I have my microservices on AWS, I use some managed and some not managed services, I need to start to plan out to do some testing. Is there anything that the cloud provider can help me, or it's just basically more third party, or more down to me, or more to us and partners?
Van Der Merwe: AWS specifically, that's my public cloud, but they have a service called AWS EventBridge service discovery. What I'm thinking now is if you have this home in production or something, and if two services do communicate with EventBridge, then that schema and contract that they are communicating with will be recorded in that registry. Then you can, for example, before going live or before doing the acceptance on this new deployment, you can have some alerts that check if this version or event is different from the previous one. If it is, then that schema might have changed, and you might not have known about it, or maybe that other program will now break because the schema is different. You can even put some alarms if you have a central event bus like that, and that just monitors if events change. Then maybe even notify everybody of that change. I think there are ways to mitigate those contract breaking changes.
Circuit Breakers and Observability
Losio: Quality must be enforced by each service, so you need to implement circuit breakers and add observability in place. That's part of it.
Abeysinghe: That is about unit testing. It is important. Then again, you have to do that integration testing as well. I think one part we missed entirely is the test data. Even if we have a good set of tests, if you don't have a rich number of test data, we will not cover certain use cases. I see, most of the organizations are not paying much attention to test data. I am proposing, like have this role of a business architect who has the depth of technology as well as knowledge about the domain, who will start working on the test data and then cover as much as possible. That is one success path that I have seen with many microservices based implementations.
Kruger: We have a lot of customers around the world, and some of them are at the bleeding edge, the leading edge. What we're seeing some of them doing is maybe a bit less testing. As soon as they pick up a problem in the code itself, they will automatically either roll back, turn a feature flag on or off, roll back if you do canary rollouts, or if X number of errors are happening on that rollout, return back or stop the deploy. We are seeing some organizations moving to that as a method to say I'm going to take on that risk, but how do I mitigate that risk of pushing out a large number of changes at scale? That's one of the ways they actually do that, if you don't have all that testing that you can do up front, as an example.
Steps to Improving Quality, when Moving to Microservices
Losio: I'm a developer, I'm an architect, I follow this topic, and you convince me. That's great. What's my first step to improve my quality while moving to microservices? I want to start tomorrow. We said on the deployment side, we start with maybe a lift and shift if we are not in the public cloud, but where should I start then, to developer, write contracts? Define how many microservices I have?
Kaiser: Understanding the goal, of course, like why would I like to introduce microservices. Also, identifying good candidates for microservices. Coming back to domain-driven design, for example, following the key principles of high cohesion within a service, and also loose coupling between the services, and trying to find the good candidates for microservices and also suitable team boundaries.
Abeysinghe: Regardless of whether we write microservice or monolith, each line of code that we write is impacting somebody's life, because we are writing mission critical systems. We have to think about it. I think if you think like that as a developer, the quality will be the main focus. That's the advice that I would like to give for every developer.
Kruger: I think my final advice would be, most microservices, if not all are written in code. Focus on the code, that's where the problems will occur. That's where the bugs will occur. Get back, focus on the thing that you're delivering, which is that code. Focus, then know what you're going to build before you want to build it. That's really where all the testing needs to happen. That's where the problems will happen, but also where success is. Focus back on the core.
Van Der Merwe: My piece of advice would be, do some research first, that you need to learn about domain-driven design and event-driven design, because synchronous communication between services mustn't be your first go-to. You must, I'm thinking like asynchronous communication between services to make that IDE resilient and available.
See more presentations with transcripts
