Beyond Copy-Pasting Methods: Navigating Complexity

Agile and complexity - the phantom menace

Why is agility a good idea? When you take Jeff Sutherland’s book, it says Scrum: Doing twice the work in half the time. At first sight that looks like a pure efficiency issue. But if we ask experienced agilists why they are doing agile, they usually come up with a list of challenges for which agile works better than other approaches: users only half understand their own requirements. Requirements change because the context changes. You have to build technological substitution into your design. Your solution is connected to many other things, and they all interact and change. But also inside the project you know there will be unpredictable problems and surprises. If we look behind the obvious: what is the common force that makes these challenges behave the way they do? The answer is complexity. Exploring complexity has a big advantage. Once we understand more about the complexity behind the problems which we are trying to solve with agile, we in fact clarify the purpose of our agile practice. This is the starting point from which we can build a common focus and sense of priority within our agile culture.

Agile methods - the temptation of the system

From my perspective as a leadership coach working with agile IT organisations, the structural part of agile fosters many capabilities which help with complexity: iterative progress means adaptability at each iteration. It means we can experiment: trial and error instead of analysis and logical conclusion. But also, we gain momentum by preserving the individual iteration; for example the sprint, from too much disturbance. The opportunity to involve customers with as little filters as possible is another complexity-competence (and we may question whether the typical requirements capture gives us enough unfiltered context, and the possibility of dialogue). Another one would be the use of diversity of perspectives in a cross-functional team. Or AB-testing.

But methods and processes are a bit like recipe-books: they help adopters become good fast, without necessarily understanding why they are good. For an organisational developer, the temptation is to drive the whole transformation from this point of view: we let people learn agile processes and methods, and we develop the organisation further by simply moving towards more sophisticated processes and methods.

The downside of this temptation is revealed at the points where it is possible to use these tools well - or poorly. What precise topic do we choose in the retrospective, how do we apply our judgment when we prioritise our backlog, how do we keep the connection to the customer’s living requirements open? This is where understanding the nature of complexity helps us make decisions.

For example, by switching between the rigidity of the sprint and the openness at a sprint change, we in fact play with the balance of stability vs. adaptability. If we do that badly, we end up either in what complexity theory calls premature conversion (make things rigid too soon) or in eternal boiling (keep things open for too long). But no structural rule can tell us precisely where the right balance is - it depends on our context.

The same applies to the coordination between what happens in the development teams and what happens on the level of product strategy. From a complexity perspective, a good strategy acts as a vector: even if it is not possible to indicate the precise end result, it gives us a direction - it says “north”, and leaves it to the teams to define the next steps. But at the same time it must include a clear idea of “south”, of what is to be avoided. There can be many reasons for lack of clarity in the purpose. For example, sometimes the corporate vision is just too vague: I’ve once worked in a company in the chemical industry that had the vision “Shape the Future” - this could mean just about anything, and therefore nothing. A better example is the “Leitbild” that BMW put on their webpage as early as 2007: “We deliver premium goods and services in the area of individual mobility.” This means: even a Mini will always be premium in its sector, we’ll never try to compete with Hyundai. We won’t do buses or trucks. We see services, such as leasing or ad-hoc renting (Drive Now as joint venture between BMW and Sixt) as important as producing cars. And we don’t think combustion engine only. This is valid until today. On the other hand, unresolved questions in the strategic orientation can have a devastating impact on efficiency and the agile spirit at team level. A software company was developing a next generation product from scratch. The composition of the company’s investors changed frequently, and their existing customers wanted to leverage the outlook onto the next generation product for their negotiations of existing contracts, which were very big. As a consequence, the direction oscillated between “substitute the existing product for big corporate customers first” and “go into the mass market first, and migrate the big customers later”. These insecurities and shifting priorities were so fundamental that the “agile”, software development resulted in a lack of focus and very poor efficiency. The strategy did not do its job as a vector.

The return of the mindset

So, in order to avoid this trap of a blind adaption of processes and methods, to consciously fill the gaps that it leaves for individual judgment, it is important to develop our appropriate mindset. The agile manifesto is a text that addresses the mindset, and very consciously avoids functioning like a recipe-book. I’ve seen teams profit from going back to the 12 principles in the manifesto and finding out what it means to them. At the same time, it is a normative model: a number of statements saying “this is better” or “do this”. If we want to apply it to our own context, we have to find out, for each statement, why it is a good idea.

This is where the sense of purpose  and the creation of a common focus comes into play again, and where understanding the nature of complexity comes handy.  Like a participant of one of my trainings told me once half a year later: “I look at situations in a different way now.” It is not a precise tool or method, but a competence which leads to a specific impact in a specific context. Let me illustrate how the right mindset fills the gap between the structural framework and the individual context.

I once had to return a fiber optic cable in a shop because they unknowingly sold me a version that didn’t fit my router’s socket. Because it was during my moving house, I was more than the legal 30 days late. The shop assistant had the structural authority delegated to him to take it back anyway, and in the end he did. The reason for this high degree of autonomy was the premium position of the brand. But the way he did it: first he didn’t trust my explanation, then he asked his boss for an o.k., then he almost told me off because they often get customers trying to trick them into handing out money… The brand message was destroyed. If you take a manager’s, or coach’s perspective on this situation, you see that this kind of behaviour cannot be regulated by structural tools. You need to have a clear purpose, and then regularly, as a team, connect the purpose to these kinds of examples and discuss your way of action. You will never get a definite conclusion. But if people have exercised their mindset on a dozen of such examples, in the next, slightly different example, they will act in a better way.

Enterprise agility - the need for a common language

The relation between structure and mindset gains in importance when we think about scaling agile across the enterprise. The core of agile is in software development. If we want to expand that into enterprise agility, it would be completely insufficient to just teach everybody the process and methods. I’ve heard of a pharma research department where they introduced the Spotify model in three hours of workshop, and thought the job was done. To me, this looks like a guarantee for zombie agile. We need to address the mindset, and more specifically, the “what for” of agile, so people can link the purpose of the transformation to their need. People want to understand enough so they can decide which methods they can translate into other contexts, where they have to build their own agile framework, and even where agile is not such a good idea.

At one of my customer’s in the telco business, they have done so at a very macro-level of organisational development. They have allowed (not ordered) local experiments with various alternative organisational structures. They have kept the organisational constraints wide open, to the point where the lack of rigidity, and sometimes compatibility between various parts of the organisation became a problem. They decided to live with this problem for so long until several experiments had lost the taste of novelty and reached a level of maturity sufficient to draw valid conclusions along the lines of “if we could start over again, we would...”. On that basis, they are now encouraging a more standardised, and regulated variety of organisational forms to emerge. In the next phase, it is to be expected that those organisations which are unhappy with their experiment will merge into the new standard. Over the whole course of action, this is asking for a lot of patience, and tolerance to variety, from many key players. But at the same time, both blind bureaucracy and resistance to change are kept to a minimum.

If we want these kinds of open, contextual transformations to happen, we need a common language which is sufficiently abstract so that many different contexts in the company can be mapped into it, but at the same time clear enough so that with little effort people can come back from the common concepts and translate them into something specific they can do tomorrow in their own environment.

That’s where the Amber Compass comes into play: a set of principles, activities and tools to help people at all levels of the organisation master complexity.

A handrail to competence - the Amber Compass

The Amber Compass is like a handrail that guides you in the right direction, without being too prescriptive - see it as an invitation to think in a certain way. It draws much on the work of Dave Snowden - in fact some people from Cognitive Edge have been involved in its early development. In the core, there is our understanding of complexity, represented by the Cynefin Framework. This is a good place to start, because it is a contingency model; instead of saying “Everything is complex, therefore do this”, it says “Depending on the situation, choose appropriate lines of action.” If we take this as a kick-off to understanding more about what is fundamentally different between ordered and complex, we are on a useful path to developing our complexity-savvy mental models.

The next circle includes the principles which help us focus our actions. The agile examples above connect to several of them: remove filters, and adjust granularity, but also aligned autonomy: how can we delegate decisions so that the organisation can react to the unpredictable and the exceptional, while at the same time keeping a common line of action? I work with a company in the semiconductor trade at the moment where we are slowly delegating more decisional authority down to the frontline. At the same time, we have to work hard to increase the common consciousness of how the business works, and therefore what the common focus should be. We do this not via general directives, but by increasing the common discussion of shopfloor examples.

And finally, the outer ring represents activities in which we can engage, some of which we also mentioned: experiment, or manage constraints. Of course, the image of the compass only serves as a reminder. But even if each keyword is explained, it works in such a way that it allows all kinds of approaches, methods and tools, while at the same time giving us an idea of what direction to follow. So in itself, the compass is a vector, and is adaptable to many contexts. Here are a few examples:

In a company that, for whatever reasons, refused to put a strategy to paper, a head of personnel development realised he had to “manage constraints” tighter around his systems such as the competency model, leadership development programs etc., because these were slow and expensive to change. So he devised a strategy of personnel development which became the informal anchor for many other units and processes which were desperate to connect to something stable - in complexity terms he created an attractor.

A hardware test manager, in order to reduce the complaints about internal invoices, replaced a rigid, and therefore inapplicable, boundary rule “If our internal offer is not confirmed in written prior to the test, the test is called off” by a more flexible rule of thumb, i.e. a vector: “Always make sure that the internal customer has realistic expectations.”

I’ve even had a discussion from an elementary school teacher’s point of view, about the activity “explore-exploit”; it was about the balance between routines, which help the class set to a certain kind of work quickly, and novel, individual, and sometimes spontaneous course designs, which are better adapted to a specific situation.

That’s where I see the connection to expanding agile across the enterprise. We need to get away from the methods and structures, understand how our environment works and what to focus on, and then we can come back and decide which methods to copy, and what to do in a different, but compatible way.

Taking it on from here: developing your own complexity-savviness

The Amber Compass, and the Navigating Complexity training around it, are attempts to cover a bit of a white spot on the map. Many things you can find on complexity are still very academic and theoretical. Much remains to be done in order to bridge this gap to practice - without falling in the trap of recipe-book applications. One way we are trying to do so is with the use of playing cards. They let you discuss practical everyday experiences, such as traffic or neighbourhoods, and connect them to complexity, so that when in the end you see a theoretical framework, you find generic labels for conclusions that you’ve already made yourself. For example, we’ve put a video on youtube which lets you explore the Cynefin Framework with the help of different ways of shopping.

If you are the type for books, some of the more readable ones are Harnessing complexity by Robert Axelrod and Michael Cohen, and Embracing Complexity by Jean Boulton, Peter Allen and Cliff Bowman. Or the many YouTube videos of Dave Snowden. I also like books about practical examples, like Stanley McChrystal’s Team of teams, Yves Morieux’s Six simple rules, or even Niels Pflaeging’s Complexitools.

A drawback I see in this is that we end up with something like a bunch of flowers, but what we lack is order. Let’s be clear: your most pressing question is “Ok, but what can I do now, here?”, and in a complex context there is no generic answer to that question for you. But we can try to make it easier. I hope that the Amber Compass fills that gap, by functioning as a type case; it gives us places where to put our examples and ideas into order. Thus we can operate our own learning cycle: we try out a context-specific approach, which leads to a context-specific experience. We connect this to a generic category or concept, so that we find it again when we talk with others, or when we link another context to this category. We then don’t just copy-paste our approach, but think about how the new context would justify certain ways of using and adapting our experience. This is why, for example, in sports like surfing, snowboarding and skateboarding, developments in one discipline quickly lead to new, but different developments in another. Call it conscious curiosity if you like. And over time you become better and better. 

About the Author

Bernhard Sterchi, from Palladio Trusted Advisers, is a leadership trainer, coach, and organisational developer. He is the author of www.teleadersfairytales.com