Key takeaways
|
How do senior managers respond to being told that their culture is getting in the way of achieving their Agile goals, that they have the wrong value core values and principles and that the main obstacle is the design of the organisation itself? As someone who has delivered this news more than once, I can assure you they know all too well what the problems are in their own organisation, they have tried many solutions, and may yet try some of yours, but they are not going to replace themselves and their peers with agile-mindset approved, lean-thinking, emotionally-intelligent ‘wunderkinds’. They are not acting from selfish or greedy motives, but actually carrying-out their purpose as managers, which is to protect the assets of the organisation on behalf of its owners (Appelo, Management 3.0). And in particular, protecting those assets from unpredictable things such as staff.
The ‘systems-thinking’ lens
Although they see something different, both Agilists and managers view their world through a biased lens. Agile and Lean experts have a ‘systems-thinking’ bias, which reveals the way the organisation needs to be re-engineered to deliver what is valuable to the customer. Through this lens, improvement is a matter of understanding the customer’s needs and then continually optimising the process (including tools and people) to satisfy those needs. Anything that gets in the way of delivering value to the customer is a ‘blocker’ and if we have learned anything at all about Agile, it’s that it’s very good at revealing blockers.
Imagine you provide business intelligence (BI) and you have a customer that runs a chain of coffee shops. If you are able to provide a process that analyses point of sale data, the value to your customer is that it allows their managers to understand, empirically and at scale, what is happening in their shops. For instance, if your BI can tell them how many coffees are sold in each shop each week, how many kilos of coffee are used and how many staff worked there, they can optimise the activities that are controllable and improve their profits. They may be able to predict the number of staff needed well-enough in advance to have people available, reduce the number of deliveries they make each month, plan the next seasonal product launch, and so on.
Real examples of crimes against systems-thinking
Once you see the customer domain through this systems-thinking lens, it becomes obvious that the all-stakeholder ‘planning meeting’ actually blocks the team’s progress for half an hour three days a week, that asking the database team to restore a snapshot delays testing by up to five hours, and that the five-step approval process for releasing to the customer means someone has to plan a release before they’ve even started to work on it. These are all real examples I’ve come across recently, and if you care to send me your best stories I might even publish a black museum of crimes against systems-thinking.
It’s easier to devise improvements when you first encounter the blockers and before you get institutionalised to accept ‘that’s just the way things are done around here’. This is why new people can bring a fresh and useful perspective to an organisation. Re-framing the situation can help the team focus on activities that increase value to the customer. Then, for a while at least, enthusiasm for the improved activities is high, but then the day-to-day practicalities of working and interacting with other people builds-up, more time is spent pursuing non-valuable activities, and the organisation re-forms elastically, to a state of ineffective activity. The secret of course, is to repeat the improvement process regularly, which is exactly why regular improvement activities are vital to sustain change.
The prediction lens
Notice that the senior management of the coffee shop chain (our customer in this example) values information that gives them the sense of control – they want to manage, predict, reduce, improve, prevent and promote. Notice too, that it doesn’t matter to them how the analysis process takes place – it could be a spreadsheet, a web service, or even the Delphic oracle – it’s simply a process that transforms point of sales input into business intelligence output.
Viewing processes externally, and merely as inputs and outputs, has been essential to the growth of the large organisations we have today. It allows them to function in a component-like manner, with many processes choreographed and sequenced to form a highly complicated machine capable of producing interesting and fabulous products and services. In Reinventing Organisations, Laloux describes this level of organisational evolution as achievement or results-driven, citing Coca-Cola as a well-known example. Results-driven organisations are superb at scaling-up a brand, selling it and moving-on to do the same again with the next. Some are uncomfortable that building-up and selling brands prevents investment that would benefit the organisation in the long term, but no-one can argue with its financial success in the medium term.
The sausage machine
I call this the ‘sausage machine’ model mostly because I enjoy the reaction I get from IT managers when I remind them they are apparently running a sausage machine. Yet because they are managers, they do understand and appreciate the model. I got the same positive signs of recognition when I presented it at the Agile Alliance conference earlier this year in Atlanta, so it seems to be true internationally.
What I remind team members though, is rather more cautionary. Teams working inside a sausage-machine process need to be able to perform their work properly, or they must expect to be replaced. It is their individual and collective responsibility to ensure their competency in craftsmanship, domain knowledge, communication skills and understanding of their process is maintained to a reasonable standard, or better, lest their sponsors decide to look for a better sausage-making component elsewhere.
Since sausage machine organisations are driven by results (quantitative) and not by relationships (qualitative), managers and team members sense that they are less important than the metrics. One senior manager informed me he would lose his team in Europe simply because they were four times more expensive than his team off-shore, without any regard to the quality of their output (qualitative not quantitative). This is a clear message sent directly from the top that says we like the idea of the Agile Manifesto, but when it comes to decision-making time at the top ‘We value cost reduction over improving our quality’. The message coming-up towards those managers astute enough to pay attention is equally clear; good technical people leave for jobs where they feel valued. And if they aren’t leaving, perhaps you should be wondering why not?
For the coffee shop chain of this example, the BI analysis process is just another sausage-machine component (with apologies for combining charcuterie and coffee). BI analysis may be upgraded or outsourced, or the coffee chain management may select a different provider altogether to supply the service. This provides them with the flexibility to change, improve and scale. This is exactly the type of agility that large organisations want – it is sausage-machine agility.
What do Executives really want from Agile, and Why?
Earlier this year I had the opportunity to ask several senior managers what they actually wanted from Agile and I received the answer “Agile delivery”. When asked what they meant they described it as being able to deliver value in weeks not years, flexibility to change priorities, and the ability to change direction if necessary.
Now that sounds much more like a proper requirement than the usual assumption which is that ‘We want to adopt agile’. Sausage-machine agility is certainly a potential solution to this requirement, but it doesn’t explain why they wanted it. (See my InfoQ article on User stories are placeholders for requirements on the importance of agreeing the ‘what’ and the ‘why’ of user stories.)
So I investigated the ‘why’ and discovered that senior managers want agility so they can make and adapt plans which allow them to protect the organisation and increase its value. These plans are necessarily short to medium term because that’s how long they expect to hold office, and coincidentally, how long many owners expect to hold an interest.
Reflecting on what we’ve learned since the start of the Agile movement, we might be forced to admit that organisations don’t want us to replace their values and principles at all, even if it gave them a better way to develop better software. They want Predictable Agile Delivery. The problem is that software development is just not predictable.
Organisations don’t want us to replace their values and principles, even it gives them a better way to develop better software.
You can’t control what you can’t measure
The last time I worked with a client whose employees produced something we could actually see, was at Toyota’s UK car factory, where we wrote a system for senior management to see how the maintenance department’s time and resources were being spent. The information, recorded live on A3-sized paper and then typed into the system, was then analysed to see how the engineers had actually spent their time whilst at work.
This is an example of managers seeking to eliminate waste and improve the process in order to protect the sausage-machine from the people who operate the process from within. It’s not because they don’t trust their own people (even though that is implied), it’s because engineer’s time is expensive and is worth optimising. If there are 24 engineering ‘resources’ working 35.7 hours per week, then they should be able to provide 856.8 hours of maintenance, which can be broken-down into scheduled and unscheduled tasks, training, emergency response, problem-solving and Kaizen. This is systems-thinking; the list is finite, the time absolute, and the figures don’t lie.
It is the sausage-machine model operating according to Lean principles, and it applies to any situation that could be automated, including situations where people are involved in the process simply because it would be more expensive to use a robot.
In the past ten years, I have worked with teams building smart billing systems, card and cardless payment systems for mass transit, financial systems for banks, and booking systems for airlines. Despite these being the largest organisations in their industry, we never once saw the physical things that earned the revenue, because there were none. So the value of the organisations was comprised solely of the knowledge of the people within it and in the minds of the customers who paid for its services. These things are neither measurable nor controllable, but they can become predictable.
The Linearity Misunderstanding
The Butterfly Effect (Lorenz, 1972) illustrates how a small change can have an unpredictable effect on a complex system. Lorenz suggested the possibility that a butterfly flapping its wings in Brazil could set a tornado in Texas because weather is a dynamic and non-linear system. This effect, and the title of his report was about the fallacy of prediction in such systems. The popular idea of a butterfly causing a tornado completely misses the point, something I too missed until an ‘ah-ha’ moment reading Nasim Nicholas Taleb’s explanation in The Black Swan. After stating that linear systems are predictable because there’s a direct relationship between input and output, Taleb explains that we are unable to predict non-linear events due to the scale of what is unknowable. There is simply too much that we cannot know that is going-on inside the system. Isn’t that true of the human mind, the mind which we need to analyse and solve problems and write the instructions for our machines to follow?
The uniqueness of each organisation, its culture, behaviours, traditions, processes, people, tools, environment, vitality and appetite for change, intersected by the needs of a customer for a certain product guarantees a large amount that is unknowable. That means that a set of changes that worked for one organisation will not likely produce the same result for another. I will go further, and suggest that it is highly unlikely that teams within the same organisation will experience the same results, even though they are given the same training, follow the same process and have the same goals and directives. Therefore, any system that relies heavily on human reasoning will be non-scalable, even if it becomes predictable over time. Requirements analysis, designing user interfaces and writing code are obvious examples, as are designing the next generation of hybrid car engine, writing a piece of music, learning to solve crossword puzzles, or finding the antidote to a disease. In each case, the team can become predictable over time, but its experience, skills and learnings cannot be conferred onto another team. They must learn by their own experience.
Human teams are unique, non-linear and unpredictable. The output of a team can be linear, scaled and predictable.
This is because each of us has a different context; our experiences, preferences and emotional filters determine what happens to the information we receive and how we act upon it. The butterfly wings flap within us, shaping the effect that information has upon our environment.
Following a recent Agile in a Day class, one person developed a sophisticated spreadsheet to help him calculate and predict precisely the days and hours available to the team. As an experienced and effective project manager of waterfall projects, his background prevented him even from trying the usual simplistic agile approach to estimating capacity.
In another organisation, scrum masters had been issued with a spreadsheet tool to use for planning. Dutifully, they subtracted each team member’s holidays and training days from the number of man-days available in the sprint, deducted half the time of a couple of people who had been allocated 50% to another project, pre-allocated some man-hours for bug-fixing, some more for doing on-call support and even adjusted for the scrum master’s time spent on administration. Then went into the planning meeting with a definitive number of day’s available ‘for scrum work’. Once they had estimated their stories in ideal days, the product owner selected the stories, the sprint began and two weeks later they delivered anything from 0 to 50% of their goal. They were unpredictable, and although the reason was clear to me, I was unable to convince the teams to ditch their spreadsheet. They had been told to use it and their managers approved it, and they weren’t going to risk changing that.
When I spoke to their managers about it, I discovered it was they who had been gaming the planning process by carving-out time from the scrum to achieve their own deliverable targets. At least that explained the mysterious time pre-allocated for ‘interruptions’.
So I told the senior managers the way they were predicting capacity was disabling a feedback mechanism, and without that feedback loop, the teams would never become predictable. I suggested if we could help them to become stable and predictable, we could help them to improve. We declared ‘predictability’ to be the focus for all teams and I went to work with the scrum masters, helping them apply the outcome of the previous sprint to the capacity of the next one. It was really difficult for the teams because they had always estimated in man-days, even before the introduction of Scrum and they couldn’t easily distinguish between ideal man-days for estimating and actual man-days for planning capacity. One team, bravely tried estimating in story-points, but this attempt was doomed once members realised they could fix the exchange rate between story-points and man-days and simply estimate in man-days then multiply by the exchange rate.
Within a couple of months, most of the teams were estimating capacity based on the output of the previous sprint and they were comfortably hitting 75% or better of each sprint’s targets. Another team finally gave-up ideal day planning after they forgot to account for a bank holiday, which meant that even their ideal planning days were embarrassingly wrong.
What’s the purpose of Managers?
Once you realise that predictability and agility are the real goals of so-called agile transformations, and that both senior managers and team members are strongly motivated to reach for them, you realise that it is now the manager role that holds the keys to making progress with agile. In most organisational hierarchies, managers form the layer between these two stakeholders and the managers who are most effective have learned to exploit the wormholes between the layers, so they can connect strategic ambition with practical execution.
If a large part of the value of an organisation is embedded in the collective minds of the people in IT who design and build its operating systems, and if the performance of those people is determined largely by non-quantifiable things such as motivation, understanding and problem-solving, then it becomes clear that the fundamental purpose of managers has to change. Managers are no longer required to protect the assets from unpredictable employees, but to help employees solve challenges, and to protect the organisation by making employees happy-enough to stay and keep their knowledge within the organisation.
This raises the question of what is valuable to the organisation, but more importantly makes us re-consider the role of the manager in medium and large-sized organisations that are in transition. I intend to look at the manager role in a future article, but in closing this one I want to pose the question, how long should managers expect the period of transition to be take? Given that the agile program at Microsoft took place over a period of ten years (as documented in HBR and confirmed to me by Stephen Denning), how long might it take an organisation whose primary business is banking, insurance or manufacturing and whose technology practices are based on mainframe computing to become agile?
Since no two teams can be identical, each must identify and improve its unique impediments. They can do this only at their own pace of change, thus improvement is inevitably incremental. On the other hand, we shall see how automation plays a part in improvement, precisely because automated systems can be linear. An automation developed by one team can be applied to many teams.
Predictable Agile Delivery in summary
Human teams are unique, non-linear and unpredictable, but given the right conditions, their output can become linear, scaled and predictable. Learning to recognise this difference is important because it informs both the management responses required and the means of improvement.
The sausage-machine model can help managers and their teams understand the evolving domain of knowledge-work and manage the transition from plan-driven, to predictable agile delivery. Systems-thinking and related engineering approaches, including Agile with a capital ‘A’, are founded on a set of values that does not exist in a large organisation. Team members and senior managers generally support those intentions, but it is difficult for them to overcome the cultural and systemic blockers to positive behaviour change.
Managers have an enabling role to play: encouraging the development of predictability; understanding the needs of their teams; and rolling-up their sleeves to clear the blockages themselves or by escalating the problem promptly and responsibly.
About the Author
Russ Lewis (@ConsultStorm) coaches organisations towards Predictable Agile Delivery. He’s a 'big picture' techie-turned management cage-rattler who now gets his kicks helping team members and managers feel fulfilled in their work. He does this by engaging at all levels and facilitating the communications that lead to improvement. Russ architected and led Transport for London’s Contactless Fares payments system and the expansion of Oyster card to National Rail; supported the Agile transformation at Amadeus SAS; speaks at conferences and writes one of the top 100 Agile blogs.
