Software Evolution with Microservices and LLMs: A Conversation with Chris Richardson

Transcript

Michael Stiefel: Welcome to the Architect's Podcast, where we discuss what it means to be an architect and how architects actually do their job.

Today's guest is Chris Richardson, who is an architect who helps organizations modernize their architectures. He's the author of POJOs in Action and the founder of the originalcloudfoundry.com, an early Java PaaS for Amazon EC2.

Today, he is a recognized thought leader in microservices and speaks regularly at international conferences. Chris is the creator of microservices.io, a pattern language for microservices, and he's the author of the book, Microservices Patterns. He is the founder of Eventuate, an open source microservices collaboration platform.

How Did You Become An Architect? [01:24]

It's great to have you here on the podcast, and I'd like to start out by asking you, were you trained as an architect? How did you become an architect? It's not something you decided one morning you woke up and said, "Today I'm going to be an architect".

Chris Richardson: Gosh, I didn't know you'd open with such a tough question. It's like, "How did this happen?" But my first real job out after college, which was an incredibly long time ago, I mean, actually it's like 39 years ago, if I remember correctly, I joined what you would call a startup. I don't even know if that term was even used back then. The goal of the startup was to build a LISP system. This was back in the mid 80s when AI was a hot technology.

Michael Stiefel: I remember those days, Thinking Machines and all that.

Chris Richardson: Yes. And the primary programming language for AI back then was LISP. But LISP primarily sort of ran on custom hardware, like Symbolics machines and so on, because they actually provided hardware support for garbage collection.

Anyway, our mission was to build a rich LISP environment on standard Unix machines. And me and some of the other folks, we were fresh out of college. And then there was going to be some senior academics. Then they were going to work on the hard part of the system, the actual LISP engine, and we were going to do the easy part and build the user experience. But then they left. So here we were, a bunch of newbies having to figure out how to build a state-of-the-art LISP system.

And I guess that was sort of when I got ... Technically, it was my second job after college, but the first one was just a continuation from what I did in college. But it was right at the beginning of my career here I am forced into having to make big decisions around what ... I didn't even use the term back then, architecture. And in this case, it was like, how do you build a state-of-the-art LISP system on mainstream hardware? I think I might've had a fancy title like chief software designer. I can't even remember what it was called back then, but I guess that's when I just had to make some important design decisions.

Michael Stiefel: So it was sort of a trial by fire?

Chris Richardson: Yes.

Michael Stiefel: And that gave you the idea that you actually liked making these kinds of decisions, these trade-offs?

Chris Richardson: Yes, it's kind of interesting. I've just always wanted to work on interesting things and solve interesting problems.

What Does an Architect Do? [04:14]

Michael Stiefel: I can empathize with that because that's sort of what got me into architecture too, because there were interesting problems to solve and the architect was very often, especially when you're building a solution, that's very much where all the pieces come together.

From my point of view, anything that can't fit into a use case belongs to the architect because you can't write a use case that says, "Make this system scalable.” Make the system secure". Somebody has to be responsible for all those emergent properties because if no one's responsible, they won't get done. And that's how I see the role of the architect.

Chris Richardson: Yes. I mean, it is funny, right? There are various definitions ranging from the big decisions that are hard to change later to ... I often talk about how the goal of architecture is to satisfy an application's non-functional requirements, like scalability, performance, security. And of course today, given the importance of fast flow rapidly delivering software, the development time attributes like maintainability and testability and deployability become absolutely essential as well. But then if you look at the research, it's like even the distinction between functional and non-functional requirements is a little blurred.

Michael Stiefel: Yes. I mean, you're sort of saying it, to be flippant about it, there was a justice on the US Supreme Court when asked to describe what pornography was. He said, "I can't define it, but I know it when I see it". And that's sort of like architecture. You may be a little hard to define, but you know it when you see it.

Chris Richardson: Yes. It is all about making these impactful decisions.

Michael Stiefel: Those are non-functional requirements that don't fit in use cases, those things that often fall through the cracks.

Modernizing Legacy Systems [06:15]

So one of the things that you have talked about is modernizing legacy systems. We have all these legacy systems out there, and there are two thoughts that come to mind when I think about this. One is how do you understand those legacy systems and how do you evolve them? And does artificial intelligence have any role in understanding a current system or giving you any clue on how to do this?

Chris Richardson: Well, looking at it from the context of microservices, which I've been interested in for a long time, adopting microservices usually means modernizing a legacy application. In other words, migrating from a monolithic architecture to a microservice architecture.

And I sort of joke that enterprises have already written all of the software that they need. They just need to keep evolving it, right? But I mean, obviously greenfield development does happen as well. And then it's like, "Well, why do you do that?" And one reason is the underlying technology platform that you're using is just absolutely archaic. When you have companies who've written software for hardware that doesn't exist and now it's running on an emulator on the cloud, there's sort of a whole bunch of technology governance things there. And then more generally, just keeping your technology stacks current.

I remember working with client where I think their core business system for a really well-known company was written in some cobalt dialect that ran on back in the '60s, and that's 60 years later that was powering a business, and they were in a kind of crisis mode because I think the people who built that software and really understood it were retiring, right? And then on top of that, I mean, obviously applications need to evolve as their non-functional requirements change as well.

Michael Stiefel: One example that comes to mind is in those days they did batch processing. And now when you move to 24-hour a day processing, you can't just adapt that software to the new world. You have to actually take a fresh approach to it.

Chris Richardson: Yes. I mean, it's sort of like we've gone from a world where software just did this batch stuff in the background, to, "Oh, well, we've got newfangled web technology", and that's how customers interact with things. And then as you say, it's real time, it's streaming and all kinds of drivers like that. Although I think batch is alive and well, right?

Michael Stiefel: One example that comes to mind is I remember there was a brokerage house, whose name I will not mention because it would be well known. Before there was 24 hour a day trading, they had a batch system. And you could say, "At night, we'll run all the trades and clear all the trades". That just won't work in the modern world.

Chris Richardson: Yes. Any other examples where the night is not long enough to process all of the data.

Michael Stiefel: Yes.

Chris Richardson: I think that's probably increasingly common.

Michael Stiefel: Yes.

Rapid Software Delivery [09:44]

Chris Richardson: So modernization is clearly important. I come at it from the perspective of microservices as an enabler of fast flow, rapid software delivery. And given that volatile, uncertain, complex, ambiguous nature of the world and the need for businesses to be very nimble, architecture has to evolve to properly support fast flow or concretely support DevOps as the development methodology and, say, team topologies as the organizational structure.

Michael Stiefel: Which says to me that you have to evolve your architecture so that you can deliver small pieces at a time and you can deal with a very small piece and turn pieces off when you use feature flags. You have to be able to piecemeal, evolve your architecture, and that's where microservices come into play.

Chris Richardson: It's interesting when you talk about evolution, right? One of the really distinctive aspects of microservices is that each service can have its own technology stack. So you can incrementally modernize the technology stack based on cost benefit analysis rather than having to do a big bang next generation technology refresh.

Michael Stiefel: But you do have to get from that big ball of mud to that microservices architecture.

Chris Richardson: Yes, which is a very unpleasant experience, which can take years to actually do.

Michael Stiefel: So how do you propose to do this? You have a client that says, "We've got to modernize. We'd like to have a fast flow". We see that you understand team alignment, you have these patterns for the microservices architecture. How do we get there?

Chris Richardson: I mean, fundamentally, it's much easier to describe than to actually do, but it's basically carving off pieces of the monolith and turning them into services, which sounds straightforward. But what that in reality means is identifying a module inside a monolith that would be a candidate to be a service and then untangling it from the rest of the monolith so that it can be a service and interact in a sort of course-grained way.

Data Model and Transaction Consistency [12:18]

Michael Stiefel: What happens when the data model makes it difficult to do that splitting apart? Do you see that sometimes you have to look at the data model first and evolve that before you have the services? Because it's very difficult to have a service that has a single source of truth if there's several services that are mucking around with the same data structures.

Chris Richardson: Yes. I would say the vast majority of the conversations end up on, "Well, what about the data?" Essentially, a module is... Well, a chunk of code, right? And it's a slice of your database schema as well. So it's a subset of the database tables, and it might even be a subset of the columns of some of those tables. And so when you extract it out, you're refactoring both the code and the data at the same time, which can be really, really complicated to do. Enterprises tend to have really complex schemas that are difficult for people to wrap their heads around, right? So it's a really hard thing to do, but then it ends up just being absolutely necessary if you want to be able to re-architect so that you can accelerate software delivery.

I mean, there's this book, one of the black and red books in the sort of Martin Fowler enterprise series, which I don't think really has got enough attention, is like Refactoring Databases.

Michael Stiefel: I know the book. I think it was Jez Humble who wrote that book.

Chris Richardson: Anyway, one of the tricks in the book is you want to extract some columns out of a table. One of the examples I use in my workshop is imagine in a food delivery application, order management and delivery management are basically intertwined, right? So you look at an order table, it has columns that are to do with order management, but then it also has columns that are related to delivery management. And if you want to extract out the delivery service so that you can rapidly iterate on that and fine tune the courier scheduling algorithm, you need to move the delivery related columns into their own database.

But one kind of trick you can play is you could leave the columns in place as read-only replicas and then they change in the service, replicate them back so that those parts of the monolith that are reading those columns are completely unaware that the ownership is now in the service.

Michael Stiefel: But presumably then that part that is reading them as read-only has to understand that there may be a latency or they may not have the absolute truth.

Chris Richardson: Yes, that is one obvious issue with this approach, is we're now in an eventually consistent scenario.

Michael Stiefel: And you have to understand whether that eventual consistency, do you have to assume that things will generally work, therefore you don't have to worry about backing things out because then you have to get into an undo situation or a rollback situation conceivably if the truth turns out to be different than what the service made an assumption about?

Chris Richardson: Yes. I mean, there are messy transaction management data consistency issues.

Michael Stiefel: Do you find that this situation is a temporary situation, that it results from the process of developing the microservices and in the end they can go away? Or is this just a fact of life, or is “it just depends”?

Chris Richardson: Well, let's just say ultimately you extract module A into a service and then module B is reading some replica of module A's data. At some point in the future, module B is going to be moved into a service as well. So it will be accessing module service A directly instead of this hack of a data replica.

Michael Stiefel: So eventually, the hope is that this sort of kludge goes away when you have gotten that true microservices architecture.

Chris Richardson: Yes.

Michael Stiefel: Because you're going through the owner of the single source of truth.

Chris Richardson: Yes. So there's a bunch of hacks that in play that kind of grow and then hopefully shrink over time as you migrate things. And when you're figuring out what to migrate out, you end up having to do some analysis to think about this sort of transactional data consistency and you want to migrate out chunks, which are maybe larger than you want to in order to avoid inconsistent data that could create problems.

Reporting with a Microservices Architecture [17:27]

Michael Stiefel: Do you find when you do this that reporting is another source of problem because when you do reports and you want to do joins on the data and you want to merge data, do you find that you can go through the services or you have to access the data directly?

Chris Richardson: Yes. I mean, I get the reporting aspect is the other question, right? It's like, "Oh, okay, we're going to split up a monolithic system, a monolithic database into lots of little databases". And then it's like, "Ah, what about reporting?" Because that requires a global view.

And I guess there's a couple of strategies? One of which is, I mean, maybe this is in order of starting with least desirable, so that you could ETL out of each service's database.

Michael Stiefel: Right. Right.

Chris Richardson: But then that partially defeats the purpose of the microservice architecture where you want to have loose design time coupling where things can change without having to coordinate between teams and accessing databases schemas directly is definitely a no-no. So services could publish events, which hopefully, fingers crossed, encapsulates a lot of the internals, the data warehouse lake or whatever the trendy term is, just subscribes to those events and gets updated.

The other approach that maybe is more in the philosophy of microservices is the data mesh concept, where services publish data products, and then you've got all that mesh technology for reporting using said products, right?

Michael Stiefel: Certainly there is a reason for pulling the read-only data out because you avoid lock contention. If the reads and the writes are happening at the same time with some degree of frequency, you don't want to necessarily be doing reports and updating the same database at the same time.

Chris Richardson: I guess I was sort of making the assumption that reporting across a bunch of services can't be done efficiently. And so having that data in a centralized database is a requirement. So strictly speaking, that is not always true. Some kinds of reports could be done by collaboration through service APIs. I mean, that is a possibility as well.

Michael Stiefel: Is there anything else that you see as a critical issue that we haven't talked about in doing this kind of transformation or modernization?

Problems With Greenfield Development [20:25]

Chris Richardson: As far as breaking apart the monolith is concerned, I feel like we've touched on the big issues. One obvious thing to talk about would be, "Well, what about just doing a big bang rewrite", which is what some organizations like to do. And that is usually pretty much an anti-pattern. I think it's incredibly risky. You deliver no value until the rewrite is done, which is probably years in the future.

And then you also have not got any validation on your technical decisions until your application is in production. And I kind of take the point of view that until something is deployed into production, it's in the hands of the users, you do not have any true validation that it is working correctly? And the longer you defer that, and the more that you build on unvalidated decisions, the greater the risk that you are building the wrong product, the wrong thing.

Michael Stiefel: Yes, right. So essentially what you're saying is that one of the advantages of doing it this way is essentially at each point in time, you can look at what are my major risks in this gradual evolution of the system and choose to address them immediately as opposed to postponing them to some future point.

Chris Richardson: Yes. I mean, I think that is, in a way, an expression of the philosophy of fast flow, which is you are continually deploying changes into production and at the same time getting feedback on those changes, both from the production environment, which is where I made the right technical decisions, and also feedback from the users about how I implemented the right features. And arguably, the nature of the modern world and the complexity of the technologies that we're using require us to get fast feedback, otherwise we really do risk building the wrong product the wrong way.

Generative Artificial Intelligence [22:41]

Michael Stiefel: Yes. The complexity of the modern world and the technologies makes me wonder, and I've seen this in several areas, in the legal area, in the medical area, where people are saying, "There's so much information out there to master. We need artificial intelligent assistance to help the lawyers, to help the doctors".
Do you see anything like this happening in the software world?

Chris Richardson: I guess we should talk about Gen AI.

Michael Stiefel: Yes.

Chris Richardson: I swear though, I think as an aside, I feel like Gen AI, I mean, it's many things, but one of the things that it is, it is a mind virus, right? Software is eating the world. Gen AI is eating our brains in more ways than one.

Michael Stiefel: It is.

Chris Richardson: Right? It's taking up so much attention. And it is taking up a lot of my attention as well, but it does sort of manifest itself in unhealthy obsessions with the technology.

Using Gen AI To Understanding a Complicated Code Base [23:51]

But anyway, I think one of the challenges with a large code base is just understanding it. You just don't want to go, "Oh, I'm going to throw AI at a problem". Because software is built by humans, and humans need to understand the software.

One thing I've done over the years is like, "Okay, we've got this legacy system and no one understands it or no one has a global view". Each team understands their particular part. So one technique that I've used is this visible architecture workshop. I did this once with this billion dollar product and architects, like 20 architects from all over the world flew in and into a conference room and they built a model of their architecture using Lego or Duplo blocks, string and stuff. So it sounds kind of goofy, but people really enjoy architecting with string and blocks and so on and so forth. So I still think that doing that kind of human activity is really important for creating a shared understanding.

Michael Stiefel: Do you find that it's also useful to show management that this thing really is this complicated and it's really not that simple to take the next step? Because I remember one time I was doing some sort of re-engineering project like this and the management thought, "Oh, we have a very simple data model". So what I did is I sat down with the architects and had them put on the wall the entire data model, and it covered three sides of this huge conference room and the CEO walked in and said, "Wow, it really is that complicated".

Chris Richardson: Yes. I think it is useful to make that complexity visible because it's funny, right? Software just exists in a computer and it's not like a building or an airplane or some other complex piece of machinery. You literally do not see it in the same way.

And so like this architecture thing we did, it was spread over three tables in a conference room with string going everywhere, right? And it's like, "Oh, here's Postgres database, here's Sybase". It does bring it to life. It helps communicate the complexity to everyone and forms a basis for decision. So yes, that's really useful.

And then yes, AI can help. You can point gen AI at a code base and ask it questions. Earlier this month, I thought, "Okay, I'm going to use Gen AI Claude code to generate some documentation". It actually created roughly 70 pages of documentation for this architecture, diagrams with embedded Mermaid, or PlantUML and stuff, but some of it was literally made up. It just invented things. It invented the names of events and it invented functionality that had not been implemented. And I had to tell it, "Please verify that every program language element that is in the documentation exists in the code". And then it goes, "Oh yes, I invented a bunch of... "

Michael Stiefel: "You're right".

Chris Richardson: Yes. I mean, obviously the code base was kind of small. I mean, it was maybe, let's just say, of the order of 10,000 lines, but it was able to identify system operations that flowed across multiple services and create some sequence diagrams and documentation for each of those requests. And it was kind of nice.

The AI Architect [27:52]

Michael Stiefel: So let me construct a little thought experiment here. Suppose we have a perfect world, whatever that means, and we have this AI architect, this piece of software that's called the AI architect. What would it look like? How would the architecture be done? How would the AI architect get the requirements, handle changes to the architecture? Who would the design be proposed to? An AI implementer or to people? How would the incidents be fed back to the AI architect to improve the architecture? How would information about improvements needed in scalability or security be communicated to an AI architect?

It seems that it's a really very complicated problem when you think about what architecture is and how an AI would handle it.

Chris Richardson: Yes, it depends on who you're talking to, right? Based on reading LinkedIn over the past few months, that problem has already been solved.

Michael Stiefel: Oh, I didn't know that.

Chris Richardson: I just see a constant stream of posts announcing how amazing Gen AI is and how it solves all problems.

Michael Stiefel: Maybe Gen AI is generating those posts.

Chris Richardson: Probably is, right? Yes. And the prompt says, "You're writing about Gen AI, use extreme hyperbole". And then they get a massive number of engagements as well, hundreds of reactions.

Ambiguity and Requirements Analysis [29:27]

Michael Stiefel: Just take a piece of that problem about requirements definition. When I was teaching graduate school way back in the Stone Age, there was this idea that there'd be these requirements definition languages, which then could be taken and fed into these code generators that would generate this code. This was a very 1980s idea, and of course we know how that turned out.

The problem is that one of the things that goes into requirements analysis is understanding ambiguity, and computers are very bad at handling ambiguity and even LLMs aren't any better in this regard. How is the AI agent that's attempting to do architecture or attempting to understand requirements come to grips with ambiguity? How would it ask questions to resolve ambiguity? This seems to be a problem to me that is extremely difficult to solve. And I don't see anybody who's really addressed it.

Chris Richardson: Yes. I mean, to be honest, I'm struggling to wrap my brain around the scenario you described because I feel like it's so far removed from my reality of what it's like to use Gen AI for development that I cannot imagine that scenario ever becoming practical.

The Unpredictability of Gen AI For Tooling [30:54]

Michael Stiefel: So let me inject a little twist to this. One of the things that have been proposed, and it's usually proposed in terms of creating human level artificial intelligence, is that the AI needs a world model. It's just not enough to have Gen AI. You have to have a model of the world, whether it's common sense, relationships between objects.

Would it be possible to build a world model of architecture that would help an AI architect in the future try to solve some of these problems?

Chris Richardson: Yes. I mean, my sort of negative description of Gen AI is that it's a next token predictor that knows nothing and cannot reason, so it's an illusion, which a lot of the time produces plausible answers. But fundamentally, it really doesn't know anything.

LLMs to me are very strange compared to tooling that we normally use where you know what it's going to do. And if you don't, it's because you haven't learned enough of the rule. Whereas Gen AI, it's like, I don't even know if there are any rules. And you have to experiment and see what happens and come up with the right magic incantation to get this mysterious thing to achieve a reasonable result.

Michael Stiefel: I have not found LLMs capable of doing any kind of conceptual analysis or any kind of real abstraction about anything. And software is all about abstraction.

Chris Richardson: I use them to write code, and I think it makes me more productive. And it probably does, but part of it is sometimes I think studies have shown that perception of productivity is different from the reality of productivity. And so in the realm of architecture, can it do any intellectual heavy lifting? I guess I'm deeply skeptical.

Michael Stiefel: I'm skeptical too, because of this nature of my inability to get it to understand abstraction or conceptualization of anything.

Chris Richardson: As an architect, I have to understand a code base. It can help with that. And it can even help with a language that I don't even know.

How Do You Validate What Gen AI Has Produced? [33:30]

Chris Richardson: Which could be very relevant if you're modernizing a legacy code base. It can extract useful things. And also I use it to create proof of concepts like POCs or little demos. But then part of the problem is, as you mentioned earlier, how do you validate that what it's created is actually correct?

Michael Stiefel: And as you say, you sometimes don't even find out what's correct until it actually goes into production.

Chris Richardson: I mean, it's funny. I've been playing around with this simple proof of concept where I wanted a Spring Boot application to communicate with a Kafka broker using mutual TLS. It's a bunch of containers. One of them is the certificate authority and then Spring Boot application Kafka container.

A while back, I created this proof of concept and I asked it to create, I think, four, five proof of concepts. And they're all different. Each one is slightly different and works in a slightly different way, but also the Gen AI proudly said that, "Oh, this part, the interbroker communication can't use SSL". It just decided that that was not possible in all of the examples. But in reality, it is. You just have to know the right Kafka incantation to set it up.

Michael Stiefel: That's interesting.

Chris Richardson: Yes. It is interesting in the sense that, "Oh, here I am. I'm trying to use it to learn something, create a technology demo, but it's giving me misleading information".

Michael Stiefel: Yes. And fortunately, you knew enough to understand what was misleading.

Chris Richardson: Yes.

Michael Stiefel: Which did not necessarily have to be the case.

Chris Richardson: Yes. And the latest version, which I created last night, it interacted with the CA in a way that I think is insecure as well.

Michael Stiefel: I think if I could summarize the discussion that we just had, the state of the art with AI and architecture at best, it could perhaps help you understand a system, if you're smart enough to ask it the right questions to have it validate itself. But in terms of doing any kind of abstraction or any kind of conceptualization or understanding the big picture, it has a ways to go, and we're nowhere near that right now.

Chris Richardson: Yes. I mean, that is my feeling based on my experiences of using it, but there might be someone listening to this podcast who vehemently disagrees.

Michael Stiefel: Well, I can think of one person in particular.

Chris Richardson: Yes, I probably think of the same one too. But I do think that the behavior of Gen AI does depend on the particular context that you're operating in, or the particular problem that you're trying to solve. Like for instance, I don't know Golang very well. I've used it to create some little Golang applications, and it seems to excel at that. No problem. Of course, I have no way of judging the quality of the code. I'm just judging by the fact that it builds a functional application for me.

Whereas with enterprise Java stuff, it's kind of like Java domain models, "Oh, it's great at whipping up things", but then messing with some of the infrastructure stuff, I think it's all like what's in its training data, what's it been trained on? There's this unpredictability to it.

The Two Types of Software Projects [37:38]

Michael Stiefel: Yes. And especially in a software world when we're constantly trying to do something that has never been done before. Because if you think about it, that's what software development's all about. Because if I'm an engineer, let's say a civil engineer, I can wind up building the same bridge over and over again for the rest of my career or some variation of the same bridge.

Chris Richardson: I think you probably offended civil engineers by saying that.

Michael Stiefel: But there's a lot of repetitive, well-based knowledge about how to build bridges. And most bridges are not extensions of technology.

For example, when I was in graduate school, I took a course in nuclear engineering. I got a master's degree in equal engineering. And one course I took was on reactor design. And on a final exam, there was a question of you have to build this cooling system for this reactor, but the point was that you did not do it from first principles of physics. You took the ASME standard and applied it to the particular cooling problem at hand. This does not exist in the software world because we're constantly doing new things. I want a copy of Word, I just copy the bits. But if I'm building a system, it's usually because no one has done this before. And almost by definition, that's not something an LLM is good at because it's only good at what it's seen.

Chris Richardson: I would say two things. I think there are two types of systems that get developed. Some of them are just clones. I think there is a tremendous amount of reinvention of the wheel, which is kind of unfortunate. We're going to create the Nth billing system again. I think there's a large amount of that. And it's possible within the one system, there's a lot of wheel reinvention because you can't quite outsource it to SaaS or a library or something. But then there are parts of the system, the really valuable parts are the parts that are unique and special. And hopefully they are so unique that LLMs cannot help with that. There's not a lot of training data there.

And it's funny, some of the examples that come to mind where I've had LLMs just sort of crash and burn recently would be like writing good quality tests for Kafka client code. Completely lost.

Another example is I had an LLM create an OpenRewrite recipe. I find LLMs really useful to create a tool that can do code transformations more efficiently than an LLM can. LLMs are insanely expensive, but you use it to write a tool that can then do the job for you. But it missed a fundamental misconfiguration step with the Gradle project that meant I just got continual null pointer exceptions that had no clue how to figure it out.

Michael Stiefel: So I think we've sort of gotten to some understanding of the limits of LLMs. We've talked about modernizing legacy systems. Are there any other critical issues that you see facing software architecture today that we haven't talked about?

Software Development is Basically Making Decisions [41:22]

Chris Richardson: A more higher level concept is I sort of feel like architecture in particular, but software development in general, is all about making decisions. So actually it's important for individuals, organizations to have high quality decision-making processes.

I wrote about what I call deliberative design, which is like define the problem, which in itself is not always obvious. And then actually figure out what good looks like, how do you evaluate the quality of a solution. And then it's good to brainstorm some solutions, evaluate them with respect to the various trade-offs, and then pick the best one or the least worst one, and then most likely document that decision using Architecture Decision records.

And I feel like that decision-making step actually happens at all levels of software development. We're continually making decisions and we need to be a bit more explicit in how we make those decisions. Sort of a very basic point, but thinking things through is important.

Michael Stiefel: It's very often the trade-off we make that we don't realize that we're trading off is the one that will come to harm us in the end.

Chris Richardson: Yes. Was it at QCon? There was that excellent keynote in San Francisco, there was that excellent keynote about hidden decisions.

Michael Stiefel: Yes.

Chris Richardson: There are decisions that you just make without thinking about them. And I guess maybe the prime example is if you just sit down and figure out, "Well, how am I going to build this?" You should really have thought, "Should I build it or buy it”?

Michael Stiefel: Buy it. Yes.

Chris Richardson: And that's often the hidden decision.

Michael Stiefel: But there is also a flip side of that hidden decision where some programmer, when they program an "if statement", is actually going to make a trade-off that you didn't think of because essentially when they write that "if statement", they're going to make that trade-off.

Chris Richardson: Yes.

Michael Stiefel: So this has been a very interesting conversation. We've explored certain niches and corners of, I think, of architecture that people neglect, who don't think about very carefully, so I've found this very interesting. This is the time where I like to ask my architect's questionnaire to all the architects to get a more human side to architecture.

The Architect’s Questionnaire [44:14]

What is your favorite part of being an architect?

Chris Richardson: Number one is, I suppose in a sense, I like solving problems in interesting domains. I especially enjoy domains that actually have a real world physical aspect of them.

One of my clients this year was a logistics company, and so I'm in the world of software, bits and bytes, but there's containers on container ships out there. And that is so awesome. I feel like I've been on this mission for decades now of figuring out better ways to build software and then sharing those ways with people and trying to help them build better software faster, basically.

Michael Stiefel: What is your least favorite part of being an architect?

Chris Richardson: Maybe it's technology. It's funny. Yes. I'm sort of semi-serious. There in a sense that for a long time in terms of technology, I was like, you could say, working in the Spring Java space. You step out of that, you go out, you're in the world of JavaScript frameworks and it's like, "Oh my God".

Then I went down and I was in the world of infrastructure and Kubernetes and the Kubernetes ecosystem and it's like, "Oh my God, it's so complicated". Maybe the fundamental issue is it's just badly documented and navigating that world is shockingly difficult. I mean, it's kind of fun, like, "Oh, it's like great provision infrastructure", right? Like, "Oh, writing a bunch of Terraform and Kubernetes and YAML to provision a Kubernetes cluster on EKS. But oh my God, it's so complicated".

Michael Stiefel: It's a miracle that anything ever works.

Chris Richardson: Yes. And I think maybe one of the reasons LLMs are so popular is because using modern technologies in a lot of cases is so complicated because they're not well documented that it's easier to have the LLM figure it out for you.

Michael Stiefel: Assuming it tells the truth.

Chris Richardson: Yes.

Michael Stiefel: Is there anything creatively, spiritually, or emotionally about architecture or being an architect?

Chris Richardson: Emotions. I can neither confirm nor deny that I have them.

Michael Stiefel: Okay.

Chris Richardson: Well, it was sad when Spock died at the end of Star Trek II.

Michael Stiefel: What turns you off about architecture or being an architect?

Chris Richardson: I don't know if anything really does. I mean, I feel like dealing with infrastructure technologies is a frustration.

Michael Stiefel: Yes. Okay. So maybe do you have any favorite technologies?

Chris Richardson: Do I have favorites? I mean, obviously there's things that I've used for a long time like Java, Spring, Spring Boot, AWS Cloud and so on. Are they favorites? I guess they're favorites, but I guess in a way I just sort of see everything in terms of just a bunch of trade-offs, right?

What's remarkable, if you look back when I was doing software, working on LISP systems back in the mid 80s, right... So I mean, basically it's 40 years ago, which is just shocking. Hardware has advanced so much. I remember we were excited when we got a Sun Workstation with eight megabytes of memory, and now I'm using my latest MacBook with 128 gig of memory.

Michael Stiefel: Your phone has more memory than that.

Chris Richardson: Yes. It's like the computers had L1 cash amounts of memory, right?

Michael Stiefel: Yes.

Chris Richardson: But has software development evolved?

Michael Stiefel: We're still over budget and we're still behind schedule.

Chris Richardson: Yes. I feel like the practice of software development, there's some sort of disciplines around TDD and stuff like that, but I feel like fundamentally it's humans just muddling through, trying to make sense of stuff and writing some code.

And the biggest development has not been in the evolution of programming languages. I would say it's been open source libraries, so you do not have to build things from scratch. It's been Google. Or at first it was Google, then Stack Overflow. And then you could say LLMs as a way of looking things up, but it's like the essence of software hasn't changed much in 40 years.

Programming languages, I mean, the mainstream of programming languages, right? We went from C to Java, kind of a step back sideways to Golang, but it's sort of like Common LISP had rich functional object dynamic language with sophisticated IDEs back in the '80s.

Michael Stiefel: What about architecture do you love?

Chris Richardson: It's almost like figuring out better ways to solve problems or better ways to build software. So I kind of enjoy building software, solving problems, but then I also like figuring out better ways to build that software.

Michael Stiefel: What about architecture do you hate?

Chris Richardson: Kubernetes YAML files.

Michael Stiefel: Yes.

Chris Richardson: No, I mean that's sort of a slight ... Yes. I mean, it's not like I hate anything per se, but I mean, it's sort of obviously a source of frustration.

Yes. I mean, actually there's maybe a better way of expressing some of this. It's like, we write all these declarative configuration sort of based technologies and then we kind of add funk features in them to try and turn them into programming languages. So there's all these configuration languages basically are half-baked programming languages instead of a real programming language.

Michael Stiefel: Yes, yes. This argument has always gone back and forth between imperative and declarative programming. There seems to be a tension between the two of them and two different schools of thought.

Chris Richardson: Yes. Well, I think about Maven... Real basic example, Maven XML versus Gradle DSL configuration language. I would argue the Gradle approach is better.

Michael Stiefel: Why would you argue that it's better?

Chris Richardson: I mean, because it is very declarative, yet when you need it, there's a full-blown programming language in there. I don't know. I'm just sort of throwing off random things right now, actually.

Michael Stiefel: What profession, other than being an architect, would you like to attempt?

Chris Richardson: I guess there's one that I could admit to. To me, human rights are really important. And I feel like we're living in a world where they're kind of neglected somewhat. And for a long time, I've thought about in another life being a human rights lawyer would've been a nice thing to do.

Michael Stiefel: Do you ever see yourself not being an architect anymore?

Chris Richardson: Well, I think more generally, right? Could I imagine not doing software, regardless of whether you call it an architect or not?

Michael Stiefel: Right. Right.

Chris Richardson: And no, I can't. You have to pry the keyboard out of my cold dead head. I think that's how the expression goes.

Chris Richardson: Yes.

Michael Stiefel: When a project is done, what do you like to hear from the clients or your team?

Chris Richardson: Answering that as a consultant, right? The number one thing is I love to hear that I made a positive impact to their organization. I mean, that always gives me a sort of warm feeling.

And then more generally, as an author, it's funny, right? You can spend a lot of time just working away in isolation on things and then you actually meet someone and they go, "Oh, you made an impact on my career". Positive impact. And it's really heartwarming to hear that.

Michael Stiefel: Yes. I remember when... I've written two books and I remember one of them where someone came up to say, "Well, I use this technology and I learned it from you", or "We couldn't figure out how to do this, and your book explained to us how we could accomplish this". I find that very rewarding.

Chris Richardson: Hearing that is always just really nice.

Michael Stiefel: Yes. Well, thank you very much. I have found this a very fascinating conversation. We sort of touched a lot of different areas that I think are important for people. But hopefully our listeners will have some of their assumptions challenged and also some of the current hype -here are countervailing points of view.

Chris Richardson: Yes. I think what we talked about feels like a bunch of random things, but...

Michael Stiefel: No, they weren't random. There was a thread that goes through them, and I think it's an important thread.

Chris Richardson: Yes. So I hope everyone who listened found it interesting or at least entertaining.

Michael Stiefel: Well, thank you very much.

Chris Richardson: Yes, thank you.

Mentioned:

• Refactoring Databases - Evolutionary Database Design by Scott J Ambler and Pramod J. Sadalage
• Hidden Decisions You Don’t Know You’re Making - Shawna Martell and Dan Fike