In this podcast Michael Stiefel spoke with Bernd Ruecker about how important process orchestration is for solving business problems, and how it is often misunderstood by architects and developers. They also spoke about the importance of visual tools, training new developers, and the dangers of being trapped by the joy of technology.
Key Takeaways
- Process orchestration is an architecture intended to solve business problems in long-running, asynchronous, distributed environments with human interaction.
- Orchestration is best used when a business requires somebody to be responsible for the end-to-end result. An event-driven architecture is best used when you want to accomplish downstream results that do not require central control.
- Usually, the real issue is not orchestration vs. event-driven architectures, but the fact that most systems are difficult to change because they are "spaghetti architectures" with system parts interacting in an uncontrolled manner.
- Developers often avoid orchestration engines because they think their problems are simpler than they really are. When the hidden complexity emerges, the systems become more buggy and complex as they try to reinvent solved design problems.
- Visual Tools are a very important tool for developing complicated, distributed systems.
Subscribe on:
How Did You Become An Architect? [00:46]
Michael Stiefel: Welcome to the Architects Podcast, where we discuss what it means to be an architect and how architects actually do their job. Today's guest is Bernd Ruecker, who is the founder and chief technologist at Camunda, a company involved with process orchestration. It now has about 500 people, and your clients include banks, insurance companies, and telcos. Bernd has contributed to various open source workflow engines over the past 20 years and has written three books, and is writing a fourth. And I've written two books, and I know how time-consuming book writing is. It's great to have you here on the podcast, and I'd like to start out by asking you, were you trained as an architect? How'd you become an architect? Because it's not something you decided one morning, you woke up and said, "Today I'm going to be an architect".
Bernd Ruecker: Yes. Thanks for having me, Michael. No, it's probably not like that. For me, honestly, I mean, I have probably a very typical thing that I started to do things like programming with a computer super early on, when I was like 13 or 14 or whatever. And then I think one of the key moments for me was really when a friend of mine during high school basically started to start his own business. He was selling a kind of modded graphic card, it's a very specific thing back then, but over the internet. And he had a lot of trouble because he got successful with that, and he had a lot of piles of everything, piles of hardware lying there, piles of emails lying there, piles of invoices lying there, piles of stuff lying everywhere, and basically asked me if I could help him.
And I started to, naive as I was, I basically, from the perspective I have now, I would say I started to write an ERP system for that, and that was really by accident. And I dived into that, and that was kind of coexisting with, let's say, my interest about a kind of software architecture in a way. I tried to get books around that. I tried to understand how to structure a system, and I could try that out. And I got into Enterprise JavaBeans back then. I got into how to structure different components, how to build them that I can run them, and I wrote a system that is, really weird, still up-and-running today, to some extent.
And that taught me a lot of things about, and also taught me that I find it more interesting to understand that kind of bigger picture around that system instead of diving into one thing very, very deep, and I think that was kind of the roots of it. And I did study computer science. I did a master's degree for software engineering, so I really focused on that in my university, as well, but I got started, I would say, even earlier than that.
Michael Stiefel: So, in other words, it was sort of an accidental discovery of necessity that got you into architecture?
Bernd Ruecker: Probably, yes. Yes.
Michael Stiefel: In other words, the mess it is and you figured out how I'm going to organize this mess that I'm in, and that led you to sort of architecture. And then you somehow, you mentioned that you were working on essentially an ERP system. Was that sort of what got you thinking about orchestration?
Bernd Ruecker: Oh yes. 100%. So I mean, part of what the system has to do is basically looking at the data address, kind of what orders do I have, what hardware do I have lying around and stuff? But at the same time, you also have to look at the processes, and that's where we started, like the workflows, like the order fulfillment, like the returns good handling. These were basically the problems where we started, like how can we manage that process that it runs efficiently?
And also back then I started to look into that, I found it fascinating, and probably we can discuss that later on, as well, but I'm a big fan of visuals. I also was always a big fan of UML Class diagrams, for example, for structuring, and I looked into visuals for workflows. And back then that was not yet very well-developed, I would say. There were a couple of weird things happening. There was BPEL-
Michael Stiefel: I remember BPEL.
Bernd Ruecker: Okay, yes. Hopefully a lot of people forgot about that. But there was not really a good solution for these kinds of workflows, and that got me hooked into the whole question, but there should be. And even back then, there were two options I found. The one was going with super expensive enterprise software on a really complex tech stack, which was obviously not an option for that at all. And I looked at open source projects, and there were not many back then and they were super technical. But I looked into one, I got a contributor there, and I worked with that, actually, to implement the workflow within that system, basically auto-fulfillment and returns good handling. And that got me into the whole process orchestration thing, first of all, thinking about it, also in the community. Then I started giving a talk about that. That brought me into freelancing around workflow engines. And in a way that's also the story how I co-founded a company that does process orchestration as a product. I stumbled into that, as well, if you will.
What is Process Orchestration? [05:40]
Michael Stiefel: Well, look, life, sometimes some people know where they're going from day one and some people just stumble their way to success. But you said two things that were kind of interesting, and I want to sort of focus in on them. You mentioned process orchestration, and process orchestration is a type of distributed system, and distributed systems are becoming increasingly common in our world. We see them all over the place, but people still really do not understand when people are involved in the distributed system. And process orchestration is all about that. And why do you think people have so much trouble dealing with process orchestration?
Bernd Ruecker: It's a very good question. To be honest-
Michael Stiefel: And you might want to define, just for our listeners who are not familiar with this, you might want to also define what process orchestration is.
Bernd Ruecker: Probably that's a good first step. So process orchestration is basically getting order in certain steps you have to do. So you orchestrate those steps, you coordinate those steps, that's kind of the wording behind that. And normally, then you have technology that can do that. And the way it works is you define models, like nowadays, it's BPMN. I'm a big fan of BPMN, business process model and notation. It's a graphical model. You define where you say, "I have those, for example, three steps I have to do in a sequence". So I do the first step first. When that's done, the next step is happening. And then you can run such a workflow engine or process orchestrator that can interpret those models and can run instances of that, like the orders I talked about.
And then they run through basically all those steps. And then the workflow engine makes sure that, if there are decisions on the way, there are complex patterns, like if you are waiting for too long, you should do something else or push something to a human and whatever. So that's one thing, process orchestrators can interpret those models so you get a visual. And the second big thing is it's also about long-running processes. So the process orchestrator can wait for things to happen, that's very often a human, but it might also be an asynchronous answer from a system that takes longer or a response from a customer. So there are a lot of things where you might need to wait, and that's what a process orchestrator can do. And then it's normally like persisted and you have all the tooling around that, where you can see what's currently happening, where are you waiting, and so on and so forth.
What Is So Difficult About Understanding Orchestration? [08:01]
Michael Stiefel: So what do people find so difficult about this?
Bernd Ruecker: That's the question that also puzzles me for a decade now, I think. I think that there are a couple of perceptions around that. It's a lot about perception, I think, because I started with visual models, for example. If you ask developers about visual models, very often you get a negative feeling around that. That's true for UML Class diagrams, I never understood that. That's also true for process models. And I make the case with UML first, because that's probably what most people out there know, and a lot of people probably learned that in university and it's a drawing, it's like dead paper. Or you even had model-driven architecture quite a while back, where we tried to generate software out of these models, which never worked very well.
But I also recall when I started there was a tool, which was actually super great that where you wrote Java code and then you could just look at the code you wrote as an UML diagram, because it knows if I have a collection typed of that type, I can just make two boxes in the right arrow. So it was kind of a automatically documenting things, it was live documentation, and it worked the other way around, as well. You could add attributes there and it was adding that to the code. I found that so helpful to get an overview of what I was doing, and the same is true for these process models. They are so helpful at their executable code, so they are not a wishful thinking document, they are executable code, but they are readable by every human. And that does not actually have to be a computer scientist, it can be a business person, as well, it can be whatever. And that's so valuable, but somehow developers are scared of these process models. That's the first perception I see everywhere.
Michael Stiefel: I mean, some of that is sort of historical. After all, the original idea behind COBOL was that even a manager could program. I don't know if you remember COBOL, but that was one of the first computer languages developed, and of course, that was an utter failure, from that point of view. I mean, COBOL is used all over the place from the idea that that would be obvious. I think some of the fear programmers have, and I remember from when I used some of those visual tools, is sort of the paradox of programming.
Normal engineers, or a civil engineer, can essentially spend their entire career conceptually building the same bridge over and over again. But in software, if we want to build something new, we reprogram it all from scratch, because if you want another copy of Microsoft Word, you just copy the bits and you're done. You only undertake a programming project if there's something new to do. But the technology that we've automated is based on what we knew in the past. So there's a disconnect between what we could automate in the past, but the fact that we're doing something new that we never did before and that the tools are not capable of doing that. And that's the frustration I had when I used some of those visual modeling tools.
Bernd Ruecker: Yes. But I would look at it from a different lens. For me, it's really a little bit of a different view on the same thing. I mean, you can write code in whatever, Java, Node, or C#, it's also an abstraction. You don't write it on whatever assembler level anymore, right? And the same is for BPMN, for example. It's a language that's defined exactly how it should behave, and then you express problems with that language on a different abstraction level, but it's still general purpose. You can model whatever you want to model there. And the other thing people forget then very often is you're not, especially if you could use good tooling, that's probably, again, something a lot of vendors did pretty bad in the past, but if that's flexible enough or developer-friendly enough, you can say, "There is an SPI, there is a hook where it can hook in code, if I don't get any further with BPMN, then I simply just code it".
But there are certain elements in BPMN which are great for expressing long-running problems, and I simply should use them because that's what I see on a weekly basis, at least. Otherwise, people start implementing their own homegrown, bespoke workflow engines without knowing it. It's the, "Oh, we just need that order status column in that table over there", but we have to monitor that. We write a small schedule looking at that every day and then we do this and that, and they start to build workflow engines. I see that so often, and it always starts with, "Oh, we don't want to have the complexity of an additional component", but you have the complexity of that mess you wrote.
Michael Stiefel: Plus, you're rethinking something from scratch and not getting the benefit of other people's experience. So that raised something also interesting that makes me think about the low-code movement. So in other words, when you use a system such as this, you're trying to minimize the amount of code that a developer has to write, which is generally a good thing.
However, when you're trying to figure out why the system is not working, do these orchestration engines, or whatever you want to call them, get in the way of understanding what's happening? Are they too much of a black box?
Bernd Ruecker: Let's say the core engine itself should be a black box. I mean, even if you have the source available, you could look into that, nobody wants to do that. I mean, I recall times when I debugged into Hibernate, that's nothing you ever want to do just because you can.
Michael Stiefel: Yes, I've done that. Yes.
Bernd Ruecker: So I would say overall, that it should a black box, and that's good as it is. But it's, again, very general purpose, so if you have a good product there, the core should work and it's not... I mean, it's still complex to get that right, especially at scale, for example. But what it does, from looking at it from the outside, is not that complicated. So it basically triggers certain activities, and that's either connector, which is code, or it's your code, so it's normally not that big of a problem. So you still have the possibility to debug into things, for example, that you just don't debug into the core engine, you debug into what you made out of that, basically the activities you put into the process. You have the possibility to write automated unit tests, for example. So that's not gone because you used such an orchestrator.
Michael Stiefel: I guess what you're sort of saying is that we write stuff on top of the operating system, and we don't step into the operating system.
Bernd Ruecker: Yes. Yes. Yes, you're right.
Michael Stiefel: And there are bugs in the operating system, but I guess what you're saying is we look at them sort of as a, this system is supposed to X, it doesn't do X. We have to call up the vendor and understand why that is the case.
Bernd Ruecker: And I mean, overall, it's a bit like also the cloud discussion, "Oh, we can't put that in the cloud. Is that secure?" I'm 100% sure it's more secure than what most companies do at home.
Michael Stiefel: Yes, yes, yes. The only interesting thing about the cloud, just as a side issue, as you mentioned it, it just builds a more tempting target for someone to hack.
Bernd Ruecker: Probably, yes. Yes.
Michael Stiefel: Yes. Willie Sutton, who was a very famous bank robber, and he was asked why he robbed banks and he said, "Well, that's where the money is".
Bernd Ruecker: True, it makes sense. But at the same time, if you compare using a product that's built for that-
Michael Stiefel: Obviously. Yes, yes, yes.
Bernd Ruecker: And used in a lot of organizations, it's probably better than just code it yourself. Yes.
Michael Stiefel: Yes, yes. But again, when you're dealing with human fear-
Orchestration vs. Event-Driven Architectures [15:44]
Bernd Ruecker: Yes. But honestly, the typical objections are not about, "Ooh, is my software still stable?" It's very often about, "I don't need that component". It's still that belief, because it's so simple, my problem is so simple. It's not. And probably because people are not really looking into, let's say, the full scope of process orchestration, like on an enterprise level, that's a different topic we should probably look into. And the one thing, and that kept me actually busy for a couple of years, discussing orchestration versus choreography, basically. The whole thing, do I want to control what's happening, or do I want to let it evolve by pushing events around, for example?
And there was a belief for quite a while, I think it changed at that moment, but there was a belief that event-driven is so much easier to do. It's so much more decoupled. It's better, or probably more beautiful or modern or whatever you want to call it, architecture. And that led a lot of projects into we don't need orchestration. Orchestration is kind of bad. They brought that together with, oh, centralized bottlenecks and whatever, and that's 100% not true. And that's another perception we saw there where people deliberately didn't want to go into the orchestration route, especially out of IT, which is kind of weird.
Michael Stiefel: Yes, it is. But you're not arguing against event-driven architectures altogether?
Bernd Ruecker: No. No. So my point is there is a place for event-driven and there is a place for orchestration. Basically, both are typically about different components or services communicating their way back to the distributed system. Because in my whatever, order fulfillment process, I need to talk to the CRM system, to the ERP system, to the stock, to the whatever, logistic system, so I need to get all those pieces together. So the question is how do those components interoperate? And with orchestration, basically I have one component that has the duty, the responsibility, of orchestrating the end-to-end process and saying, "Okay, the first thing I have to do is, for example, getting the stuff from stock or retrieving the payment", and so on and so forth. It can control that, to some extent. With event-driven, with the choreography, you basically distribute that responsibility, saying, "There's an event, the order just came in", and then some component knows, okay, then I have to collect money.
And that's an architecture question, actually, one of the core architecture questions, how do you want to distribute responsibilities amongst the components? And that's a design choice you can make. And there are sometimes good arguments where you want to be event-driven, and there are sometimes good arguments where you want to be orchestrated. And especially if I look at these typical business processes, like order fulfillment or whatever, I want to get a new bank account, I want to get a claim settled or all these kind of business processes, then you very often, from a business perspective, you want to have somebody responsible for the end-to-end, because the customer is only happy if the parcel arrives at my door. It doesn't look at how beautiful my events are flowing within, just if I get my stuff here or if I get my money paid out or whatever, then the customer is happy. So there is a big emphasis on the end-to-end and understanding certain KPIs (Key Performance Indicators) and bottlenecks and everything on the end-to-end level.
And because of that you want to have one owner, and because of that you should have also on the IT side probably one personal, one component being responsible for that. And if you want to help that component responsible for something, it needs to control certain aspects, like the sequence of things for example, or the SLA. What happens if the logistic provider doesn't talk back to me within two hours? Is that okay or not? And deciding all these kind of factors, that's an architectural decision, and that's... To make a very long answer short, it depends on how you want to lay out the system, and in most bigger systems I see a coexistence of both event-driven and orchestration.
Michael Stiefel: Where do you see, ideally speaking, event-driven should predominate in your architecture choice?
Bernd Ruecker: Every time when the component that triggers something is not really responsible for doing it. I'll give you one example, I like examples. I find that much easier to understand, and it's actually one thing I discussed with Sam Newman regularly. He wrote these microservices books, and he has an example also of an account opening in his book and I have an account opening in my book, so that's a good collision of examples. And basically, what I look at from an orchestration perspective is the whole the customer applies to the bank account, for example, is opened because that's a responsibility of one component, one microservice, if you will, or however you want to implement that, and that should be orchestrated.
He looks at, okay, the customer needs to be put into the loyalty points bank, he needs an account there, he probably needs a whatever letter, and certain things that are not in the responsibility of that core opening process. And then events are great because then you could just push an event on the bus saying, "Hey, the customer is created". And then it could build different components reacting to that saying, "Okay, then we do this, but if we don't do that, nothing happens". The bank account is still open. Your COO will not come to you basically asking you, "Why is it not on the loyalty points bank?" It's a different responsibility, and this kind of trade-off, I think, is very important to think about.
The Real Problem Is Most Systems Are Architectural Spaghetti [21:19]
Michael Stiefel: So, if I could summarize what you said in sort of a sentence or two, and tell me what I've got it right about, it's basically, from a business perspective, where the responsibility lies?
Bernd Ruecker: Yes. Interesting side note here, while this is one of the things, I think, on a conference level I discussed the most over the last years, it's still, when I look into our customer base, it's not the predominant problem. It might be discussed, but the predominant problem is that they have an integration spaghetti there, where systems are talking to each other in a totally uncontrolled manner and you have no way of really extracting the end-to-end business process. It's already hard to understand it, which makes it almost impossible to adjust it, and that's a big, big problem at the moment because there is so much change. You might want to drive customer experience, you might want to add new services, you might want to infuse AI everywhere, and they have mostly no idea how the process is currently running and they can't touch it. And I think that's a much bigger problem at the moment, to be honest.
The Public Does Not Understand How Fragile These Systems Are [22:26]
Michael Stiefel: Yes. Well, it's also a big problem from the public's point of view because they don't understand how fragile these software systems actually are. I don't know if you're familiar, for example, in the United States there was this, the security vendor for some Microsoft systems had a bug, and this caused tremendous outages in the airlines and, especially Delta, took days to recover, and there's all kinds of finger pointing going back and forth. But the public really doesn't understand how hodgepodge or fragile these software systems are, and you seem to, when you talk about this from the IT perspective, I don't think upper management sometimes realizes how fragile these systems are. Do you have any feeling about this one way or the other, how to explain this to people, whether upper management or the public?
Bernd Ruecker: But I think that's, honestly, one of the, let's say, core areas where we also look at when we talk about business processes and process orchestration, because it's exactly that connection between business and IT. The IT side is implementing the process, okay, but the business side needs to understand the end-to-end process. And if I talk to the business, they have different language. They talk about whatever business model canvas, customer journeys, value streams, and these kind of things. We can get also the business to level where we talk about an end-to-end business process and probably business capabilities. And then we want to link that to implementation, and the process, for me, is one of the key elements to do that because that's also what a business person understands. And then as soon as you have that running, you can even link data to that, yes.
The Architect is the Translator Between Business and IT [24:12]
Michael Stiefel: You hit on one of what I think is a key role of an architect, because the architect is the person who can talk to the business people about business and the technical people about technology. They're sort of the interpreter of the two different worlds.
Bernd Ruecker: Yes, 100%. When you ask about architecture, what always comes to my mind first is I think there are still different levels to an architect. So I started, I would say, as a solution architect, looking at a single solution only, which is a bit different to looking on an enterprise architect or in the business architecture level. But I totally agree, especially the latter two should be able to, I like the Gregor Hohpe metaphor, actually, of the architect elevator, that you have to ride the elevator up and down, from the engine to the penthouse and you have to be able to talk to all of them.
Michael Stiefel: Which means that one of the skills of an architect is, even if because of their generalization they lose some of their technical chops, they still have to keep their intuition about what is technically right and wrong.
Bernd Ruecker: I agree, and I' am still a big fan, I mean, I also do a lot of discussions nowadays about business value, for example, express business value, and I understand why it's so important to do that, right? At the same time, I try to program, probably not on a daily basis anymore, but at least a weekly basis. I need to touch code very regularly to, exactly, keep that feeling what you're talking about, so yes, 100%
Michael Stiefel: Yes, yes. Good. So you mentioned about processes in the enterprise and you wanted to get back to that. Is there anything that comes to mind when you think about that?
Bernd Ruecker: I think I touched on that briefly. So what we are currently seeing is a lot of change in organizations. That's probably not news. AI or agentic AI, or whatever the buzzword of the day is, puts a lot of pressure, and we see organizations struggle with that. And what we also see them doing, if you look at it from an enterprise perspective, very often they try to optimize things in local areas, like in that one application or I use a, whatever, robotic process automation bot over here, I want to use that AI tool over here. But very often that's local because that's how the teams are siloed in that organization, very often the business teams. And then they start to do projects which, on a local level, might make sense because they have an ROI, oh, we automated that, we saved manual labor, there's a ROI.
But zooming out, and that's what a lot of companies miss, zooming out, looking at these end-to-end processes, it doesn't have to be necessarily good what they are doing. Either it doesn't have any influence on the overall end-to-end process, it can even cement certain bad behaviors, bad patterns, like making it harder to change that overall process, and it definitely not looks at improving it end-to-end. There are tons of examples. So we have seen, for example, AI to do support requests you could save if you make the overall application process much smoother, for example. And then, of course, you should invest there and not making the support better, even if that looks good on an isolated basis.
And all these kind of local optimizations, I think, are a huge problem. So we need to elevate that discussion to understand that value stream and the business process, and for me, process orchestration is kind of the key element to make that happen because it can actually look at the process and make the connection to really executable software and then orchestrate different pieces, different tasks, together. And that's super powerful if you look at that on an enterprise level. So not so much in my local project, how can I orchestrate the process better? We have that, as well, but really look at it more holistically. And that's what we are currently doing with quite a couple of customers, which is super interesting.
Michael Stiefel: I don't know if you're aware of this or not, but this is actually a broad economic phenomenon. I don't know if you're familiar with something called Pareto optimization, but the argument there is very much the one that you mentioned, is that if you want to optimize something and you can't do local optimizations, you generally have to reoptimize the whole system to get an incremental optimization. And what you're describing actually fits into a larger economic system, and you seem to say that process orchestration is a way of forcing that issue to the front.
Bernd Ruecker: Yes. And it's a way of probably solving the riddle, at least to some extent. I mean, that's not the only piece, but I think it's a centerpiece. And it's something, again, which you can make a CIO or a CEO or a COO understand because business process is not-
Michael Stiefel: A foreign language.
Bernd Ruecker: I think that's one problem of the microservice community, for example, it's discussed in IT, IT only. If I go to a business process, business capability level, I can talk to business, as well. And that's a different thing because you need them. I mean, that's actually why the business exists, so you need them, and that's saying me as kind of an engineer by heart, you need them, of course, on board to make transformation happen.
The Technology Is Fun Trap [29:24]
Michael Stiefel: I think, in my career, I've had the same experience, that it's amazing how many of the technology people think in terms only of technology and don't understand the business reason for doing something. After all, it's the business that's paying, it's the business that's driven. I mean, yes, I'm fascinated with technology, too, but the idea is in service of a goal.
Bernd Ruecker: Yes. I totally agree. I thought about that quite a bit, as well, because for me it was kind of a transformation, a journey, I would say, personally because I was super nerdy and enthusiastic about technology for a long time.
Michael Stiefel: It's fun. Technology is fun.
Bernd Ruecker: It is, and I loved it. I mean, we did a cross-orchestration engine that can scale on geographically-distributed data centers without adding latency, but we still have the same throughput, for example, so that's obviously awesome and exciting.
Michael Stiefel: But you tell that to a business person and they go, "Duh".
Bernd Ruecker: Yes, it's like, "Duh".
Michael Stiefel: Right.
Bernd Ruecker: On georedundancy, they might understand, but not how we did it.
Michael Stiefel: Right.
Bernd Ruecker: But I still had that progression, I would say, to understand why business value is important, and then also starting to understand how you can explain that better. Because very often it's an indirect story you have to tell why this technology enables this, what it can do in a software application, and that will basically increase whatever customer experience, and then you can talk to them. And that's so important, and probably, I would say, it's a typical journey a lot of people run through, and I think it's probably a good thing. I want to keep all those enthusiastic 20-years-old people that love technology. They are awesome, and we need them.
Architecture and Development are Two Equal Roles [31:05]
Michael Stiefel: And maybe some of them will stay that way for 40 years.
Bernd Ruecker: Yes, it's also fine. Yes, yes, yes, true.
Michael Stiefel: I think it's a mistake to think that everybody has to become an architect. To be a super developer is great, too.
Bernd Ruecker: It is. It is. And for me, for example, I did a very conscious decision not to go into people management. And for me, that architect, for example, is also a path you could take where you say, "I don't want a people manage, but I want to get to a broader view", and have kind of whatever, probably making bigger decisions, if you will.
Michael Stiefel: That must be tough running a company.
Bernd Ruecker: You have to find the right co-founder. Again, a piece of luck I had in my life.
Michael Stiefel: But also, it's important to explain that story to the technology people, why they can't fall in love with this piece of technology, and they have to do it maybe the old-fashioned way because that's better for the business.
Bernd Ruecker: Yes. It's not always easy. You probably know that there's a blog post called Choose Boring Technology, or something like that, and that's what it's very often about.
How Do You Train Developers? [32:06]
Michael Stiefel: Yes. So two questions that sort of come to mind, and they're separate questions, but I think they're things that you've touched on that are important, how do you train developers? If developers have trouble understanding when to use events and choreograph and when to orchestrate and distributed systems and asynchronous systems, how do you train new developers to understand this?
Bernd Ruecker: There is, as always, nuance to that. So there are things where I just think people have to be made aware just by trying things out, and that's, I would say, for the smaller problems, for example, mastering asynchronous communication, having probably consistency problem, like transactional problems, long-running things, those you can occasionally train by doing examples, probably let them writing things, probably review stuff, so that-
Michael Stiefel: And wonder why their system's hanging.
Bernd Ruecker: Yes. I even personally learned the trouble of two-phase commit early on in my life in a real-life scenario. I mean, that's something that sticks with you afterwards. So that's okay. I would say the bit more architectural questions, the bigger, hairy ones like event-driven or orchestration, which might influence not only one communication link, but probably how you lay out the whole architecture, that's probably not for a newbie. That's probably something where it's good to have some experienced people on board. We very often do coachings, but when I talk to people, it's very often architects that already have seen something, otherwise it might be hairy.
Michael Stiefel: So sort of what you're saying is that gradual exposure to increasingly more complex situations is basically the way people learn?
Bernd Ruecker: Yes, probably. I would say so.
The Importance of Visual Tools [33:51]
Michael Stiefel: And the other thing you talked about is sort of your love, if that's too strong a word, but your appreciation of visual communication and visual tools.
Bernd Ruecker: Oh, I would say love is a good word.
Michael Stiefel: Okay. I think, given that people very often don't appreciate this, and I know from the work that I've done with workflow and the little work that I've done with process orchestration, visuals are very, very important, especially with long-running processes that you can't necessarily get in your head completely.
Bernd Ruecker: Yes. It's even a problem that you can express very well with visuals. It's a flow of activities, probably some different path, then there are events happening where you go somewhere else, so it's a problem that fits well on a visual. And to be honest, I don't get it why people are so against it. And every time, when I discuss a process with people that don't like visuals, what they do first is take a whiteboard and draw it.
Michael Stiefel: Yes.
Bernd Ruecker: It's like, because otherwise you can't follow. So I personally don't get it. And then sometimes get arguments like, "But how can I do diffing?" Yes, you can do visual diffing. I'm not sure if that's the most important question, but you could do it. So no, I just don't get it.
Michael Stiefel: There's a limit to the amount of complexity the human mind can conceive of, and they're not linear processes, necessarily, and there's feedback. It seems almost perfect for a diagram.
Bernd Ruecker: Yes, I agree. And same thing, and probably you need different levels of abstractions, for example, you can do subprocesses or certain things that might happen you might want to handle in a different process or some things you might still want to handle in code, so you can play with that. It's one tool in your tool belt, and if it fits, it simply should be used.
The Architect’s Questionnaire [35:39]
Michael Stiefel: Yes. So one of the things I like to do to ask all the architects that appear on the podcast is the architect's questionnaire, which gets into the more human side of being an architect. So what's your favorite part of being an architect?
Bernd Ruecker: I think I said that relatively in the beginning, seeing the big picture. Not going super deep in one element, but understanding the different pieces and how they are connected. Probably also how they are connected to business strategy, how they are connected to the technical implementation, so I think that's my favorite part, I would say.
Michael Stiefel: So what is your least favorite part of being an architect?
Bernd Ruecker: To some extent, that's probably also connected to my current role, which means I'm very often talking to a lot of companies, but always as an external. And I think that's also true for architects in organizations, so you're not part of the team typically. You're very often an outsider, which I can live with, but it's probably the one my least favorite thing of it.
Michael Stiefel: Is there anything creatively, spiritually, or emotionally about architecture or being an architect?
Bernd Ruecker: I love that question. I mean, the creativity part, I think it's clear. I think it's super creative. You have to very often find elegant solutions for very complex problems and make or communicate them in a way that they're easy to understand. That's very creative, and I love doing that. It always makes you a little bit happy if you see that this seems to be understood and works. And the spiritual or emotional element, it's probably harder to answer, but for me it's, again, I would say happy is kind of the thing. If you see that it's working out, if you see that people are understanding it, if you see that whatever you draw helps people understand things or implement that, it's just a good feeling.
Michael Stiefel: The world is a little better off incrementally. What turns you off about architecture or being an architect?
Bernd Ruecker: I would say it's the, there is a typical perception around architects also being kind of the naysayers, the no-sayers, the governance entity, the police, if you will, which I think is not true, at least not for good architects at all, but it's kind of the one thing very often I hear.
Michael Stiefel: Do you have any favorite technologies?
Bernd Ruecker: That's easy. Of course, BPMN. Of course, Camunda. I contributed to a couple of other workflow engines in the past, as well. So there it's great technology. Otherwise, I would say simply also programming. So I am a Java person, I started with Java and I like Java. I like the type languages, to be honest. I like the dependency management, I like the tooling, the ecosystems, the open source around it.
Michael Stiefel: What about architecture do you love?
Bernd Ruecker: Probably repetition. I would say breaking things down into manageable pieces, solving a complex problem. I would say that's more or less it.
Michael Stiefel: And what about architecture do you hate?
Bernd Ruecker: Discussions. Too long discussions, too many discussions, not fruitful discussions. I know it's part of the thing, and to some extent I like it and it makes things better, but at some point in time it's too much.
Michael Stiefel: What profession, other than being an architect, would you like to attempt?
Bernd Ruecker: I haven't found any so far. I mean, co-founding the company, I had a couple of roles in the past, but it always was architecture related, to be honest. I think my absolute favorite was kind of what we now call a solution engineer doing proof of concepts, like flying into the customer three days, get something to work in their environment, and leave. You don't have to finish it, but you can draw the boxes to sketch the idea and prove that it really works, so that was always my favorite.
Michael Stiefel: Do you ever see yourself as not being an architect anymore?
Bernd Ruecker: Probably, let's see. I mean, what I said early on, what I experienced at the moment is that I had that phase of my life that, I would say 10 years, where I was so into enthusiastic about technology. When I was at a conference I picked up something, some keyword I had to Google that, I could try it out. I was sitting in the evening in the hotel room hacking something to understand it. To some extent, that changed a little bit, probably also with family and kids and whatever, not so much time, and seeing that generation is still there doing that, probably now with AI and stuff. So I kind of foresee to probably do more mentoring than architecture actively myself, but not yet.
Michael Stiefel: And when a project is done, what do you like to hear from the clients or your team?
Bernd Ruecker: When I read the question, my first thought was, it's not really an answer, but my first thought was, don't make any more projects. Go into the product thinking world, that's where you should be, actually. So a product is never done. If you do software, it should be kind of a product, it should be maintained, it should be a life cycle, it should keep doing things. So that's one thing. But I totally get the question.
So the other thing, and that's probably, again, because of my time in my life I'm currently, I would like to hear from them what's the value of what we just did? What's the story that connects it to the business value? What could we tell everybody in the company why we did what we just did, and why is this great? I think if they achieve that, for me, that would be even better than, quote unquote, just having a great technical solution.
Michael Stiefel: Well, thank you very much. I found this conversation fascinating, and I think you've illuminated many dark corners for people about both orchestration and how to think about architecture, so thank you very much.
Bernd Ruecker: Thanks for having me, Michael.
Mentioned: