Agile is adaptive. When and how to apply architecture depends on the context. This article first explains why this is the case and then how you can still give proper attention to architecture in an agile setting. Adaptability and conversation are the essentials.
The Agile Manifesto was written over 10 years ago in response to the plan driven approaches that were the standard at the time. In a plan driven approach planning the work and doing the work are separate steps. The preparation of a good plan requires a stable foundation: well specified requirements. Hence the explicit need to stabilize these specifications. Based on those stable specifications an equally stable architecture can be created and subsequently implemented.
In reality this turned out to be rather difficult; as human-beings we're just not wired that way. Have you ever been part of a project where you got a good, complete book of specifications at the start, with which you withdrew to your team, possibly at an offshore location, resurfaced again right on the 18-month plan, and launched without further ado to the delight of your client? We haven’t. The client gradually discovers what he wants, the development team discovers how to create it, and meanwhile the surrounding environment tends to change as well.
Therefore agile has a different approach to planning, one that embraces all this change as long as it adds value for the client. This approach provides us no basis to build a big design on and no desire to create such a design either as that would be too fragile. This doesn't mean architecture isn't important anymore, however it needs to be robust to future change. What that comes down to? It depends. How much work do you still have to do upfront? Just enough.
You probably already knew these answers? You'd hope we can be a bit more concrete by now, 10 years after the manifesto kicked Agile into gear? Yes, so do we! And right now would be nice, as our customers, the business side of our organizations, continue to raise the bar. Success is no longer delivering according to plan, but delivering software that makes an actual business impact. Techniques such as Impact Mapping, Feature Injection, and User Story Mapping are rapidly gaining popularity; each of them ways to help us build the right product. Ways to build the product right, with an effective architecture, just can't stay behind. To clarify, as architecture is a broad topic, the architecture we are about to zoom in on is the software architecture: the architecture of a single application.
Architecture? We do TDD and we write clean code.
Applying Test Driven Development can greatly benefit the design of your code. We find it especially powerful if you work outside in, from acceptance tests inwards, like the London School of TDD propagates. Working from the outside in reduces the tendency to add all kinds of what-if code. Furthermore you test the way classes interact, tests not created when applying unit testing in its strictest form. In our experience these are the tests that actually support your refactoring, rather than get in the way.
Now, if you also write nice, clean code that clearly communicates to the reader, you probably have some good stuff going at the detailed level. But what about the bigger picture? How do you document overarching design aspects that are not that easy to spot by looking at the code? How do you discuss them and come to agreement? Wouldn't some easy to understand architecture sketches be beneficial?
You’ve probably made some sketches one time or another not unlike these depicted from one of our projects. May we ask, who was present? Architects, leads, and management, or your whole team? Where the sketches revisited and updated later? If so, was that valuable for your project? If not, what held you back?
In our opinion creating and maintaining such material as a team is an essential part of architecture. By sketching the architecture and discussing with all involved in coding, you connect both the application-wide concerns and the nitty gritty details. TDD gives you a framework for making detailed design decisions, but it’s the architectural activities that help you make healthy ones. And while almost anything looks good on a whiteboard, it’s the warts subsequently forming in your code that tell you your architecture model isn’t ok yet, or not ok anymore.
Emergence
There’s a common misconception that no specific attention to architecture is needed in agile; that it is tossed out together with the document that defines the architecture in advance. The concept emergence that the principles behind the manifesto speak of is interpreted as will happen spontaneously. A better interpretation is: we do that together, gradually. Architecture is a team activity and shared understanding, not a document that is passed down to the team from an ivory tower.
The best architectures, requirements, and designs emerge from self-organizing teams.
-- Agile Manifesto Principle #11
Characteristic for an emergent property is that it does not exist without the parts of the system from which it derives. It is not traceable to a specific part but it affects every part of a system. Water for example has no fluidity when you remove the molecules. A single molecule is not liquid, but water flows as a whole. Similarly, there is no (emergent) architecture without the application and the team. You can’t point out a single architect who decides what the architecture will be. But the shared conceptual framework affects the behavior of the team and what the application looks like. The traditional process has no place for this line of thinking. Again we see here that agile really does come with a change of mindset to a focus on people and interaction.
Is there still a dedicated architect then? Perhaps, though not with the same role as before. When you’re working on an application with multiple teams (or on sister applications), it helps the teams to keep their focus when someone else looks after overarching concerns. Such an architect has architecture as a primary focus, but he works to serve the teams instead of prescribing to the team or working well ahead of them. He understands that he is not valuable for teams when he makes them wait for the perfect plan and that they can help to prove the architecture by implementing it. He’ll experiment and validate, both through spikes and by regularly coding together with team members. During architecture sessions he continuously asks the team if what’s discussed aligns with how they’ll actually write the code.
Options have value
An architecture that is robust to change, one that actually makes change easy wherever possible. It is an architecture that does not needlessly restrict your options. Every time you make an architectural decision, especially one that is hard to change later on, you limit your ability to deal with change. Commit to hard to change decisions only when you know why you're committing at that specific moment and always keep looking for opportunities to postpone decisions in a responsible manner. It’s just as in your private life: you’ll buy tickets right away for that concert that you expect to sell out quickly. But an evening in the pub with a few friends will probably be a last minute arrangement. Among other things this means: keep it light. Don’t weigh down your application to handle things that are not necessary (yet), but make it easy to add them later. In other words, reduce the need for changes to you architecture by making it extendable. Apply the Open - Closed Principle to your architecture. Let‘s make this more concrete in the next two sections.
Simplicity--the art of maximizing the amount of work not done--is essential.
-- Agile Manifesto Principle #10
Simplicity
Keeping things light also means pursuing simplicity, reduction to the point where further reduction would remove significant value. That's about actively removing functionality of little value. And it is just as much about the structure of the application itself. The simpler it is at any level, the easier it is to understand, from a clear architectural drawing to a cleanly coded implementation. Devote a considerable amount of attention to the layout of components and simplify the components themselves. Simple software is adaptable software!
How do you divide your application into components? What does your application tell you, right when you open your development environment? Hopefully what functionality it provides , with the techniques and frameworks pushed more to the edges. Alistair Cockburn suggested years ago that a breakdown in concentric circles is more natural than in a stack of layers. Architectural models using this breakdown have recently found the spotlight, to such an extent that they will hopefully become the new norm.
The core of such a layout begins with the functionality the application offers, the features, or use cases if you will, like Buy travelcard, Change travelcard, Cancel travelcard. In many applications these are hard to find unfortunately, though providing these features is what the customer is after and the application is about. We like to distinguish them from the business domain entities they work on. Therefore we create them as a separate layer around the business domain. Around the core you will find the interfaces to the outside world. Not only to the UI , but also to the database and interfaces to and from other systems. You can immediately see that things most likely to change are easy to adjust; another database, a mobile UI in addition to the existing one, making a services interface available, etc.
Without exception all dependencies point inwards . Thanks to dependency injection, each layer is completely independent of other ones. Nor do the various interface components in the outer ring know of each other's existence.
Bounded contexts
We've worked with architectures like we just discussed for some time now. We've experienced the many benefits and yet at times we're still not happy. As the application grows and ages, the weight of the core too often starts to drag us down. The model’s underlying assumption is that the Entities are least likely to change, as they represent the business rules, preferably valid enterprise wide. It's that stability that makes it ok that everything depends on them. But simply by implementing the business rules gradually, as we iteratively add features, we already break that stability. Nor do the various departments of an enterprise and their systems generally have unified, stable definitions of their business entities; they depend on context and also evolve over time.
At some point we should let go of the notion of one unified core and work with bounded contexts instead, a concept from domain driven design. A well known example that separates two bounded contexts into two separate models is Command Query Responsibility Segregation (CQRS). Your application can have similar separate models to serve different business contexts. These contexts are meant to allow change at different speeds for different, usually organizational reasons. To make this convenient, we'll have to break the binary dependencies between them.
Russ Miles suggests we do just that and switch to simple events to communicate between components. The freedom one can gain is very powerful, but it is also where the trade offs start. For example, do you want just mild decoupling, or are you prepared to do without the advantages of exchanging strongly typed objects entirely, in exchange for the flexibility provided?
Say you interface with a mainframe that is updated twice a year and never quite at the date predicted. At the same time your business wants changes to the business rules of the insurance products they offer just about every week to tune them to market opportunities. You probably want to handle these two concerns in different components, no binary dependencies between them in either direction. But is it ok for them to both depend on a strongly typed interface between them, or will only interfacing via a very loose data structure without explicit schema provide the flexibility you need? You can have that flexibility, but it will make both components more complex and demand more of your process to make sure you only take compatible components to production.
You find yourself balancing flexibility versus complexity in a trade off that likely changes over time. This makes it important to group functionality into components wisely and revisit this grouping as new needs emerge, a continuous attention to architecture. To Russ this is therefore as much a process tool as it is a model. He calls it the life preserver.
Do and learn
What will your application look like when applying the techniques we discussed? We are not familiar with your project and don’t believe in an ivory tower architecture, so we will not presume to tell you. It will probably reflect the organization and the constraints you are working with. Conway’s law states as much and perhaps that’s actually a good thing.
organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations
—M. Conway
So this is where we send you on your way. Proceed from the known principles: separation of concerns and the pursuit of a limited number of components with strong internal cohesion and few dependencies. Add the agile principles we described: an open attitude towards change, teamwork, cooperation, emergence, simplicity, and above all working software.
Make it your mindset and simply start, not just as the appointed architect, but also as a developer. Doing and learning along the way are essential. Ask yourself what parts of the work now done upfront your team can already do without and how the team can show that it has found more effective ways to see to those needs. Good luck!
About the Authors
Wim Heemskerk helps teams to be agile in their practices. He is an Agilist, a software craftsman, and a Stoosian. As a hands-on change agent, he stimulates the alignment of process, technology and organisation. Wim connects the dots to translate complementary models and their principles to day to day actions. He works to create lasting change, the kind people say they created and wanted themselves. He supports others on their quests for agility, leadership, great software and test automation that actually pays off. He tweets as @WimHeemskerk.
Minze Tolsma is a software craftsman with a deep interest in the way human beings work and think. He embraces everything which can add value to the way his customers. He strongly believes in leadership from one step behind: be flexible, try to thoroughly understand the business you as a craftsman are serving and then act with all the tools you’re equipped with. Minze works for iPROFS (Haarlem, The Netherlands) as an software architect and knowledge manager. You can find him on Twitter and LinkedIn.
