Towards an Agile Software Architecture

Software architecture is the skeleton of a system. It defines how the system has to behave in terms of different functional and non-functional requirements. On one hand the traditional waterfall approach puts hard constraints on all the phases of the project development and hence makes it rigid. On the other hand the agile movement allows us to welcome changes, even late in the development phase. Although we are moving from a rigid development to a more flexible one, software architecture is a part that is sensitive to changes due to its nature as a skeleton. A key factor therefore, when following the agile movement, is to embrace the notion of a sustainable software architecture – one that enables the expansion of the system to happen in a gradual, easy, and maintainable way along with growth of the complexity of the project.

In this article I cover my experience when working with both traditional waterfall software architectures and agile ones. I depict the similarities and differences between these with focus on three areas:

• the specifics of the software architect role;
• the timespan of the software architecture;
• the output of the software architecture.

What Is Software Architecture?

There are hundreds of definitions of what a software architecture is (you can actually add your own as well). The reason is that everyone defines it based on their context. I am particularly fond of the definition of the IEEE, as it describes the fundamental concepts together with being easy to visualize in your head. Moreover, it applies to both the waterfall and agile process as it depicts the essence of a software architecture, rather than how one produces it. In the rest of this article I am going to refer to this definition:

The fundamental organization of a system embodied in its components, their relationships to each other, and to the environment, and the principles guiding its design and evolution.

Waterfall Software Architecture

Traditional waterfall development is characterized by a set of phases with definite start and end dates, where each phase contains a certain set of activities. All phases are chained together and each of them relies heavily on the delivery produced by the preceding one. Figure 1 illustrates a common set of phases involved in waterfall development.

                                                  Figure 1: Traditional Waterfall Model

The work on the software architecture usually begins after the software requirements have been defined, as at this moment it is assumed that the system is well-defined in terms of what it should do. The next step is to examine who is in charge of defining the software architecture and what the actual output of this phase is.

The Traditional Software Architect

Software architecture is in practice usually done by a software architect. This is a person with strong technical knowledge and experience – typically a developer who has been promoted after reaching a certain level in the company. This person is responsible for analyzing the software requirements and making certain technical decisions about the future system based on these requirements. While in many companies there is usually one software architect per project, in some bigger companies there could be a team of software architects working together. This is typically the case when the project domain is complex or the project has a bigger timespan, for example, 2-3 years or more. Not all companies have a designated software architect role – many companies delegate these responsibilities to their senior developers. In the rest of the article I refer to the specific activities a software architect performs, rather than who does them, unless explicitly mentioned.

A traditional software architect has four main features:

• Focus on the big picture – a software architect should consider how the system would look like months in the future (sometimes even years). Moreover, he should consider all other systems (for example any 3^rd party systems and data stores) involved and how the communication between them is going to happen.
• Compliance-oriented – a software architect should consider possible compliance issues. This could be legislative norms, licenses, standards, or others, but he should ensure the future system is going to meet these important criteria.
• Produces blueprints – an important delivery is a collection of documents and diagrams describing the architecture from different perspectives. The development team uses these deliverables to start building the system.
• Not much hands-on experience – the goal of the software architect is to produce the final documents for the development team. Although the software architect usually has experience as a developer, he is rarely involved in the development process – he rather mentors developers to build the designed system. At some point he may even move to another project and leave developers on their own.

The Agile Movement

While the traditional waterfall architecture is a one-time activity with definite start and end dates, the agile software architecture is an ongoing process, which may never end. This allows us to apply changes in the architecture, if necessary, on a regular basis. One of the mechanisms to welcome changes in a project is to apply iterative and incremental development. In Scrum these iterations are called sprints and one sprint typically lasts between 2-4 weeks as shown on Figure 2. By having such small windows any change that arises will be discussed soon. Moreover, there is also strong focus on the team collaboration and any issues between the team members are solved right away in order to prevent misunderstandings and poor communication.

                                                                  Figure 2: A Typical Sprint in Scrum

The agile movement allows you to welcome changes in the project, but it does not tell you how fast you should react to them. Software architecture is very sensible to changes, as it is the backbone of your system. For example, do you think you can change the platform or language you use in the middle of the project? Such a change, even if rare, would require many iterations. It could even bring you back to the beginning of the project. When the software architecture is concerned, some type of changes are painful and require time to implement.

The Agile Software Architect

Scrum defines three types of roles:

• a product owner –responsible for providing information about the specific business domain
• a scrum master – responsible for facilitating the communication and collaboration in the team
• a development team – responsible for implementing the user stories and producing working software

In order to convert the traditional software architect role into one that fits the agile world, we have to take a look at some possible variants. One way to structure the Scrum team is to have teams of developers and a separate team of software architects working closely together. Figure 3 illustrates this scenario applied to multiple Scrum teams.

               Figure 3: A Separate Team of Software Architects Works with Multiple Development Teams

While this is a valid way to structure your teams, it has two problems:

• The software architects’ team might become a demander, while the developers’ team – a producer, which is typically the case with the waterfall approach. What happens is that a software architect defines the vision, while developers are left to implement it. This could decrease the motivation among the developers, if they are not involved and treated respectfully.
• As a consequence of the above, some developers might want to become part of the software architects’ team as to be able to choose what other developers should do. Trustpilot presented a case where they had a core team and development team, but this decreased the moral on the working place and the collaboration in general, so they moved each member of the core team to one development team, which happened to have a positive impact.

Another approach is to place the software architect directly in the development team as shown on Figure 4.

                                     Figure 4: Each Development Team Has One Software Architect

In this case the agile software architect gets a bit different responsibilities:

• Balances between the big picture and now – an agile software architect has to think about what is happening during the development together with aligning it with the big picture of the entire system.
• Hands-on experience – an agile software architect is also a developer and works on the implementation of the system. This gives first-hand feedback on the taken architectural decisions.
• Produces prototypes to make informed decisions – when an important technical decision has to be made, a quick prototype could reveal whether this decision is feasible and how it would affect the existing system. Furthermore, communication with the entire development team is essential as it is a collaborative effort, rather than a single-man activity.
• Focus on sustainability – it is extremely important that architectural decisions lead to a sustainable software architecture – one that will support the project in the long term. An essential part of this is the personal responsibility and empathy. The agile software architect is part of the development team, so he gets first-hand feedback by his decisions as described above. This increases the level of personal responsibility in contrast to the waterfall approach, where decisions are transferred to the development team to implement.

You may choose the structure with a separate team of software architects, if you want to share them among different development teams (and maybe projects). Moreover, you could also apply this structure, if you are working on a complex domain where many viewpoints have to be considered. In these cases, however, you have to make sure the software architects work closely with the developers and are present to support them. The agile methodology puts focus on collaboration and by separating software architects from developers, you make the collaboration difficult. As a result the development process becomes closer to the waterfall model. I personally prefer the second variant, in which one software architect is present in each development team, because the communication between the team members improves. On a higher level architects can still (and should) coordinate together.

Timespan of an Agile Software Architecture

An important aspect of agile software architecture is when it has to start. Opposed to the waterfall model where we have well-defined phases, in the agile world there is no certain point that people agree to be the starting one. One typical approach is to introduce sprint #0 – a special sprint where the development environment is configured and some fundamental decisions are made (for example programming language, platform, database, etc.).

A common pitfall with this approach though, is people tend to prolong it as they always find things that are “almost ready”. “One more week and we can start the regular sprints” is often heard. In many cases you find yourself already working on the system, without even having the user stories, because “it will be really cool to have that helper feature implemented in advance”. In such situations you should beforehand agree on an end date for sprint #0 – this could be the duration of a regular sprint or something close to it.

One may wonder then, what if you are not ready with the architecture when the regular sprints are supposed to start. Well, that is totally fine. It might actually never be ready. And that is also fine. Software architecture is a constantly ongoing process. This is something you should always come back to and revise as it is the skeleton of your system. You cannot afford to develop a system without making sure your architecture supports it. Simon Brown says:

Agile teams don't necessarily create agile software architectures.
But a good architecture enables agility.

Governing Principles

The world we live in is complex and so is every business domain. When building a software architecture it is really easy to overcomplicate things right from the beginning and hence make the consequent development more error-prone. Two principles has become a de-facto standard when making decisions:

• Keep It Simple, Stupid (KISS)
• You Aren’t Gonna Need It (YAGNI)

What these two principles try to enforce is to make us think if we really need a specific feature or decision at that very moment. If we can postpone making a decision to a later moment, we will keep our architecture simple and hence easy to manage for a longer time. One common way software architectures become complex is by introducing abstractions – this could be a new fancy layer that copies data in one format and transforms it into another, or this could be creating many classes, interfaces, factories, etc., in order to make your code 100% testable.

However, one pitfall when applying these two principles is that we tend to delay everything until the last possible moment. By then it may already have become too difficult to implement the required change. Instead our task is much more difficult, because

we should take decisions not in the last possible moment,
but rather in the most responsible one.

What I usually do if I am about to make an architectural decision, is to do some small preparations that do not take much time, if the certainty of this decision is high. I consult also my colleagues and we discuss it together.

How to Organize the Documentation

The waterfall approach requires extensive documentation to be written, because it is used to transfer information between the phases (and hence the people involved in each phase). This is not only a time-consuming process, but it often leads to misunderstandings due to poorly described features. Furthermore, documentation is hard to keep up-to-date, because the development tends to go fast and in many cases doesn’t reflect the documentation anymore. As we use agile with iterations to write code, we could do the same with our documents. We start with describing only the important aspects of our system and then continuously add more information when necessary.

What to Document

Don’t document the same aspect multiple times in multiple formats. For example, there are tools to help you generate diagrams from your code – this could be very convenient instead of creating separate diagrams for it, as they tend to become obsolete. Furthermore, if you have created a visual artifact to describe a certain aspect of the system, there is no need to create an extensive text document describing the same thing with words (unless you want to add details you cannot express visually). An important note here is that you and your team should have common understanding about the used drawing notations.

How to Document

When documenting software architectures, one might think UML is the way to go. UML is a standard, everyone knows it, universities teach it, and so it must be the tool to unite everyone in the organization. My experience shows though, that few people use UML in practice. One of the reasons could be it provides many ways for describing a system from different perspectives, so if one doesn’t use it regularly, one can become frustrated.

There are no specific tools for documenting software architecture in the agile world. One could use whiteboard drawings, Post-It notes, text documents, wikis, etc. (see Figure 5). In practice it is safer to use only 2-3 different artifact formats, otherwise it may become difficult to store them and search for information. For example, you may need to take a picture of the whiteboard after drawing, so that you have a digital version of your diagram. If you need to edit it later, however, you have to choose whether to do it digitally or draw it again on the whiteboard and take a new picture.

I use a diagramming tool to produce simple diagrams of the components of the system. This is usually Microsoft Visio or draw.io (which has integration with Google Drive), but there are plenty of other tools available both on-line and off-line. When drawing on a whiteboard at a meeting, I redraw the same diagram using the diagramming tool afterwards in order to keep my artifacts in the same format and close to each other. If I need additional explanations to my diagrams, I tend to create text documents aside.

                         Figure 5: Documentation in Form of Whiteboard Drawings and Post-It Notes

Conclusion

Software architecture defines the skeleton of the future system. It is not just a diagram with lines and dots, but rather a complete set of decisions that govern the development of the system, including the code itself. Every decision made is a trade-off and should be carefully considered. The agile mindset requires to be open to changes, even late in the project, in contrast to the traditional waterfall model, where the requirements are expected to be rather stable. However, changes in the skeleton of the system are not always easy to implement and may even take you back in the development process. Therefore, it is important not to delay the decisions until the last possible moment, but rather at the most responsible one – even with a small risk of implementing something that is not required at that moment. Moreover, agile software architecture requires a complex mixture of looking at the big picture and the ”now” in order to create a sustainable platform everyone can build on.

About the Author

Boyan Mihaylov is a developer, architect, and enthusiast. His passion is to solve everyday problems using IT. He's agile and uses improvisation techniques to create agility.