Evolutionary Architecture from an Organizational Perspective

Introduction to Evolutionary Architecture and its relevancy

Based on the book by Neal Ford, Rebecca Parsons, and Patrick Kua, Evolutionary Architecture (EA) can be described as an architecture designed to accept incremental and guided changes. Along the evolution of the business requirements, we take care that our architectural characteristics don’t erode. Therefore, in EA, great emphasis is put on Fitness Functions -- i.e., architectural metrics that help us assess the state of our architecture.

EA is very often perceived as an end goal of software architecture. We can’t expect all requirements to be known beforehand. Our architecture must embrace that fact. The development of new functionalities should be considered as a constant movement forward with no end state. Hence the common saying "the software product is never done."

Problems with technical-only perspective for EA

Most materials around EA are devoted to technical topics. Besides Fitness Functions, popular topics include Microservices, Continuous Integration and Delivery, evolving database schemas, automated testing, and many others. There is no doubt that these are crucial elements to achieve EA.

At the same time, based on Conway’s Law, we know that some organizational structures are counterproductive and, as a result, the desired architecture may not be delivered. There are always people factors in software. Teams won’t produce long-lasting products if they are forced to work against each other.

EA relies strongly on the organization -- without proper cooperation of the whole company, we can’t assume that technical patterns will be sufficient to handle the evolution of the software.

Organizational characteristics for EA:

As far as EA is concerned, there are some interesting organizational characteristics that should be considered:

IT and Business as one team

When it comes to organizations, it is very often the case that IT capabilities are treated as a cost center. Business teams listen to the client’s needs and propose features. Software teams work as a feature factory -- they are ordered to deliver particular features as fast as possible. The software team isn’t aware of the client’s needs and thus, they don’t get feedback on these features.

This situation creates additional layers of communication and hinders understanding the business context. We adjust our architecture without knowing where it should lead us. Then we get information too late to change the direction. So, we are forced to create software across the architecture, only to deliver assumed value.

In such an environment, the architecture very easily drifts away from the business needs. There is too much delay between user needs and created architecture.

Business and IT must work together to understand the business environment and adapt the architecture accordingly. Only then is the feedback loop between the new customer’s needs and a created solution short enough to evolve architecture in the right direction.

The delivery team directly listens to the client’s needs and proposes a solution. Therefore, our architecture evolves naturally with the overall business. There isn’t an additional layer of communication that slows down accommodating the change. When the architecture doesn’t correspond to the business environment, we can remodel architecture much more quickly.

Additionally, the delivery team works more closely with the clients. They understand their needs. Based on that, the evolution of the system becomes more business-oriented. We don’t create architecture for the sake of the architecture -- we create a spine for the overall business goal.

This idea of empowered teams is shown in detail in the book Empowered by Marty Cagan and Chris Jones. A team is responsible for gathering clients’ needs, discovering the right solution, implementing it, and gathering feedback. The feedback loop is short and we shape the architecture accordingly.

Evolutionary goals

When we have organizations focused on some short-term goals, it is easy to lose sight of the big picture. Features are so important that we cut every corner to deliver them. A particular deadline is treated like there would be no day after it -- but usually, there is.

In the end, we discover that despite having the working system, we can’t adjust it to new needs. The system’s architecture was designed to fulfill short-term goals. All the constraints are set in a place where we can’t move them without breaking the whole system apart.

Gerald Weinberg said, "The better adapted you are, the less adaptable you tend to be." Organizations that focus only on the current situation aren’t able to adapt themselves to future needs. The architecture is coupled so strongly that no change can be introduced without a huge sacrifice.

Short-term goals must evolve to pursue the long-term goals. Only then will the architecture be able to evolve.

When we set our focus on a detailed set of features, we can’t forget that, in the end, we want to run a long-living business and a long-living system. Therefore, the current work shouldn’t restrict the capabilities of the system architecture. Broader business context and further goals allow us to pick the right architecture -- not only for present needs but also for future ones.

An interesting approach to this problem is to use a 2-dimensional matrix to gauge the decision. We put:

• short-term gain on one axis
• long-term gain on the other axis

Then we can measure how our decision affects the overall architecture. On multiple occasions, we can realize that short-term beneficial decisions are very damaging to the evolution of the system in the long term.

Understanding and aligning to the business domain

Usually, there are multiple groups of users that use a product for different reasons and goals. Even a single functionality can be defined differently, based on who we asked. Some requirements seem to be contrary to each other. All the needs are mixed, and we deliver the software according to our understanding.

This confusion affects the overall architecture. Different clients’ needs overlap each other, and so do our solutions. We try to develop our system in one part, only to find that it is coupled to other parts. Complexity and dependencies impede the implementation of new features.
We end up with a monolithic big-ball-of-mud -- a clear contradiction of Evolutionary Architecture.

EA strongly relies on the modularity of the software. But it is a result, not a predecessor. When you don’t understand the domain, it is impossible to come up with a proper division of the software. If you want to achieve EA with modularity included, you must focus on the domain beforehand.

Usually, Domain-Driven Design with its patterns is recommended to tackle this problem. It is crucial to focus on separating subdomains with clearly defined responsibilities. Then we handle cooperation between subdomains and check if all business processes are handled properly. The architecture reflects the understanding of the domain. Each module can evolve separately.

Don’t forget that, from a sociotechnical perspective, business and IT should align their structure according to the subdomains. It is often called an Inverse Conway Maneuver. Such alignment creates stronger cooperation between the delivery team and its clients. We have fewer communicational overheads and shorter distances to the requirements which leads to a better evolution of the module’s architecture.

Evolutionary patterns for subdomains

We tend to handle problems similarly. We work in a given business context, and after moving to another, we apply the same approaches. Unfortunately, it doesn’t always work correctly.

It can lead to a scenario where both sides are negatively affected. Parts of the business, which require a more cutting-edge approach, are pulled back by business areas focused on standards and clear expectations. In such scenarios, it is hard to experiment and test new ideas. At the same time, mature areas lower their efficiency due to the overall platform’s instability. A conflict is a natural result of such opposite forces.

Wardley Maps teaches us that different subdomains have different levels of maturity and evolve differently. We can use this fact to handle these changes in our evolutionary architecture.

More stable subdomains (e.g., Orders, shown above) are recognized as understood by your clients. We focus on stability -- failures aren’t tolerated. Your implementation must reflect it -- more complex patterns are crucial to handle complex business scenarios. Rigorous testing is perceived as a necessity. Architecture can be more stable and constrained.

On the contrary, more dynamic subdomains (e.g., Marketing, shown above) require a more innovative and flexible way of handling problems. We test our assumptions to receive feedback as soon as possible. Changes are introduced daily, and they break the settled patterns. For these subdomains, more sophisticated patterns could be a burden. The architecture should be as flexible as possible to adapt to the daily changes.

Wardley Maps is only one technique to think about evolutions of the subdomains differently. Other interesting techniques are Core Domain Patterns, Cynefin Framework, or Diffusion of innovations.

Culture of cooperation and contracts

Evolution is a constant change. We naturally alter our ways of working to adapt to new business requirements. Then, there are common cases in which we require new information from external teams so that we can handle our job. Likewise, people from different teams use our functionalities to run their code.

It starts slowly, but over time, it turns out that we don’t understand our inward and outward dependencies. We rely on unspoken contracts. Change in our functionality breaks someone else’s code. Such a situation generates synchronization problems and delays due to the external team schedule. It stops us from making changes and leads to misunderstandings and anxiety.

The architecture reflects this issue. Instead of being a modular and separate structure, it becomes a spider’s web of connections across the system and starts to rely on deep knowledge of every team’s functionality. Ultimately, the software can’t evolve in the desired way.

In such a dynamic environment, cooperation between teams must reach a much higher level to allow the smooth operation of the whole solution. Only then can the architecture follow the dynamic environment and accept changes without violating the established order.

All expectations from the other teams should be clear and accepted. We should be very deliberate in the information that we share as our contract. We can’t promise a contract that we will break soon. It must be a conscious decision what to make available for other teams. At the same time, teams must rigorously prove their ability to deliver the established agreement. Breaking changes should be authorized and handled across the teams.

The architecture based on these contracts is more stable. At the same time, it’s more open to the evolution of the software. The contract becomes an architectural constraint; however, it works as an enabling constraint. We can evolve our software as much as we want, keeping that contract in place. Change is accepted, being a part of the architecture.

On this topic, I can recommend the Context Mapping technique. It helps to analyze relationships between different modules and establish a mutual understanding of these connections between teams.

Hypothesis-driven and experimentation mindset

When approaching a new problem, we tend to stick with our beliefs. Confirmation bias reassures us that our solution will solve the problem once and for all. We don’t want to seem uncertain.

After a few weeks or months, we eventually deliver a solution. And it turns out not to meet the clients’ needs. So, we try to fix the situation on the fly. We apply drastic changes just to have a positive response from the client -- not without a cost.

Such alterations have an enormous impact on the overall architecture. As a result, the solution is barely running, with every constraint crossed and module dissected. Nobody remembers to focus on Fitness Functions when we don’t deliver value to customers.

Hypothesis-Driven Development teaches us to focus on learning from new challenges, instead of believing that we know the best answer. Only customer feedback is a source of truth. We try to come up with an experiment to check whether our assumptions are right, get enough data to prove or disprove them, and adapt our next actions accordingly.

With such an approach, the architecture of our solution would never go in the opposite direction to the clients’ needs. We regularly check if the technical structure fits the problem. If not, we change the technical structure. If this attitude is treated as a matter of everyday practice our evolutionary architecture will evolve alongside the business domain.

System thinking applied to the architecture

Big companies consist of multiple different divisions. As a result, business processes are spread across different teams. Each of them focuses on their backlog to deliver value. We are responsible for a part of the whole system. The question is -- will it work as a whole?

This local focus results in solutions being applied without looking outside the small context. Some problems aren’t fixed in the right way. Some of them aren’t fixed at all because they aren’t connected to a particular module but based on the interactions between modules. Moreover, reinventing the wheel is quite common.

In such cases, the implementation of single parts can overbalance the architecture of the entire system. We end up with a problem of local optimization. The system can evolve separately, but it doesn’t evolve as a whole.

Russel Ackoff, a system thinker, once said "The performance of the system depends on how the parts fit, not how they act taken separately." (video)

Each team should be aware of working within a specific environment, with higher goals to be achieved. We succeed provided that the whole system works properly. Sometimes, the evolution of a single module needs to be stopped to allow evolving of different components.

The architecture of the system must embrace that approach. Making an architectural decision is a multi-level issue. Therefore, we should consider many perspectives to avoid harming one module for the sake of another. The architecture succeeds only when the whole system evolves together.

In this scenario, an interesting practice would be to set up an Architecture Guild. The guild can coordinate multiple different needs to one straight goal. Teams still work independently but they are aligned to the whole system by a cross-team group that shares technical knowledge and finds recommended solutions.

Summary

Evolutionary Architecture is a set of technical principles which are crucial to evolve architecture to fulfill business needs. There is no doubt that technical excellence will be required.

At the same time, the sociotechnical part of developing software must be taken care of. Good practices are required not only from the technical side but also from the organizational side. Lack of focus on proper company cooperation and understanding of the business will lead the system’s architecture  nowhere.

About the Author

Radoslaw Maziarka is a Solution Architect and Development Manager in the .NET area, who gives presentations, trainings and conducts workshops. After work, he supports companies in IT projects and provides technical help to non-governmental organizations. Believer in agile work, automation, and continuous self-improvement. You can find him on his blog and Twitter @RadekMaziarka.