Predictable Agile Delivery: The Executive Challenge

How should senior leaders and executives approach agile? For most people in senior positions of responsibility and authority, agile development is outside their personal experience. Like RUP, ITIL, OO, Java, Six Sigma and Prince, agile seems like another set of improvements, another methodology, complete with another bunch of consultants coming-in with their carefully-honed presentations and promises.

In my previous article in this series on predictable agile delivery, I established that large organisations have a need for delivery cycles measured in weeks not years; delivery that is predictable so that short and medium term plans can be made; yet have the agility to be able to change short term goals without penalty. As agile grows-out of its years of self-obsession and teenage petulance into a post-agile state, ‘Predictable Agile Delivery’ feels like a realistic goal that advantages both the business sponsor and their development stakeholders.

In this article, I will share some ‘good, bad and ugly’ examples that I have encountered over the past 10 years of helping large organisations improve; of practices that often work and some that always fail; and recount what happens when you rattle a few well-chosen cages too.

Celebrate your culture, nurture and develop it

Anyone who has attempted to introduce Scrum into a large organisation or looked at the results of any of the agile surveys will tell you that the development team ‘get it’ but the problem is the managers, or the culture, or the organisation’s governance procedures. Sooner or later, they will hold-up the Agile Manifesto and state the organisation’s culture “has to change and must adopt these values and principles”. As a father of teenagers it reminds me of a teenager rolling her eyes skywards and stamping her foot saying; "you don’t understand me. I know things you don’t know, and you have to change because you’re old and wrong".

Ironically, teenagers would be right in the respect that organisations do need to change, and reviewing values is a good starting point. But they fall into the same trap as the people they are criticising – telling others to change, because you can see what they are doing wrong with no understanding of the difficulties of bringing about behavioural change in adults.

Yet this is exactly what happens in enterprise-scale agile transformations. Sponsors and senior managers want the benefits of agile as they have been described, so they are naturally buyers of it. Nobody has the nerve to tell them it’s going to be a personally difficult journey and they should expect rewards to be gathered daily over a long period. They believe they are buying a single-shot transformational medicine and confidently believe “we’ll all be agile by Christmas”.

Executive sponsor: “How much does agile cost?”

Salesman: “3 million over 2 years and you will make savings of at least 5 mill”.

Executive sponsor: “Great. Speak to my managers there, there and there to arrange delivery and I’ll announce it next month. Now, what’s the next thing I have to deal with?”

Obviously I’m exaggerating to make the point, but it is true that the values of an organisation permeate from the top downwards.

I say, call-out the culture of the organisation and embrace it. Culture is what makes each organisation unique and ultimately, that is where its differentiating proposition lies. So have fun with the fact that the CEO moved the headquarters to be closer to his favourite golf course and start rewarding staff with golfing incentives, finding tournaments to sponsor and developing golf-course friendly user interfaces for your products. Hire people who are keen golfers and start celebrate your organisation’s ‘golfness’ and stop expecting transformation – that only happens in fairy-tales.

Budget matters

Sometimes, as in the case of software development, top-down values and beliefs collide with empirical bottom-up fact. To quote some examples that I have come across recently, it is not possible to improve a team’s effectiveness if they are not allowed to hire the people who have the skills they need, or if they are made to hire two extra people in India or China just to get one person in USA, or if they are split according to budget-defined projects instead of product-centric teams. In each case, effectiveness was reduced and can never be raised to the levels that agile is capable of delivering. Inducting new team members reduces throughput for a time; lowering the average skill-level on the team reduces competency and slows-down the more skilled people; and limiting the work that individuals can do creates single points of knowledge that are bottlenecks at best and risks at their worst.

Ironically, the executives at the top of IT want to improve their teams’ effectiveness, but the people around them who control the budget may inadvertently prevent it from happening because their world-view is informed by costs in a spreadsheet rather than teams, individuals and opportunities developing at their unique pace. Calling this out is unlikely to change the immediate situation, but it is important that executives and senior managers recognise the dysfunctions that exist in their own team and budgeting process.

I found myself in this situation when I was recently asked to find a half million-dollars in cost savings across six teams. I looked at the rows in the spreadsheet, some contained the names of people I knew, some I was coaching, some were yet to be hired and yet others were unknown names. Instinctively, and according to some cognitive bias, I wanted to keep the teams I was coaching and the people I knew. Then I looked at the costs and noticed I could get three ‘resources’ (to use the lingo of cost-saving) in a low-wage country for the same cost as one in Europe. By the time I had allocated 50% of a person’s time to the project for six months in order to reduce his cost, I realised I had gone too far. Second-time around, I considered each team and what they needed to do and came-up with a figure that was actually 400k more than the starting-point! I decided this version was, at least, truthful, even if nearly $2m over budget and went with this to the project manager and we worked together to scale the costs back to where they needed to be to keep the teams going. Collaboration triumphed as usual and we were able to keep our funding by making the savings requested. 

Some managers prefer to exercise control using spreadsheets and figures, and teams must be prepared to support all improvement arguments with hard data that can be expressed financially. The basic metric should be ‘lead time’; the time it takes to transform a change request into released software, perhaps expressed in man-days. Thus it may take three man-days to change an input field, at a cost $1250 (3 x the average day rate) with an average defect rate of 10%. All process changes can now be measured against this bench-mark and the cost savings (or not) will be known: increasing the work in progress limit; automating regression tests; doing stakeholder analysis; reviewing the releases a week in advance. This hypothesis-driven approach is similar to that described by Ash Maurya in Running Lean and at once provides risk-management and experimentation. It is better suited for existing products than for completely new developments.

Be open about your values

Something I have seen in a workshop setting that was useful for senior managers is to agree their own set of values or principles, and then to encourage their departments and teams to conduct the same exercise. Writing a value statement in manifesto style provides each professional group with a set of shared values and makes it much easier for them to collaborate with their peer groups.

I once asked a senior manager which was more important to him, quality or delivery of software in his area. At the time he chose quality, and explained that his priority was for customers to be able to trust the software, so it was more important that it worked without error even if that meant slower delivery. We shared that with the teams as:

We value software quality over accelerated delivery

That is not to say a customer who needs their software delivered on a certain date should be told to wait, or that priority 1 incidents don’t take precedence over everything else. It means the team has to be realistic about committing to what it can deliver by that date, and when pressed to include more is able to hold-up their values and say ‘We may be able to do more, but our priority is quality and we already have the work we are prepared to commit to completing’.

At one workshop, a group that had been frustrated by too much talk followed by too little action came up with:

We value doing something over talking about what we should do

Again, the intention was not to abandon all planning or setting of shared goals, but to be able to better balance the time spent between planning and doing.

You can see how these simple but clear statements of value allow the people who are responsible for producing the output to make decisions both within their own teams and beyond. Distributed decision-making like this is the key to scaling, that most elusive and desired of agile ambitions. Without it, all decisions are made ‘at the top’ whilst the people below wait for those decisions to be made. Worse than that, decisions made by those who aren’t actually doing the work (the chickens of pig and chicken fame) are often based on bad information, incorrect data, estimates and downright lies.

Decide who you can trust then let them get on with it

Which brings me to another difficult subject – not everyone at the top table will drink the Kool-Aid – some of them are not believers and there’s no way to change that. 

When a development team member pushes-back repeatedly against Scrum, against team-working or simply against improving their process, the result is inevitable; either they will leave the team because they aren’t happy, or they will be moved-away from the team because the team isn’t happy. The same may apply to managers, but only up to a certain level of seniority. Once a manager has reached the top table, that person is responsible for such a large group of people, projects and money that they cannot easily be moved by their managers. (For such people the end only comes from retirement on their own terms, removal as directed by the board, or sensationally for those around, getting caught with their fingers in the till and being escorted off the premises by law enforcement officers.)

Some senior people have learned how to say all the right things when speaking to peers and even their own line managers, who may also lack actual experience of working within an agile team and are unable to spot their non-belief. Their teams will reveal a very different story to anyone who spends time with them, such as coaches do. This is exactly what managers at all levels need to do – be present with their teams, and sometimes be an observer. Attend their review meetings, drop-in on some of their planning meetings and retrospectives. Listen-in to the daily Scrum meeting occasionally. Being present and listening, even asking a question or two and offering some insight will be rewarded by discovering either that you can trust this team, or that they haven’t the skill and you need to stop and reconsider how to continue.

A good example of this is a team that recently started its Scrum journey, together with their manager who was also the product owner. Although not co-located, he attended the daily Scrum meeting each day and was able to see the team becoming blocked by taking-on too many diverse tasks (improperly prioritised work) and had underestimated the complexity of the scripts they were optimising (leading to too much WIP). In response, he started to work with the team-lead to re-prioritise the work for the next sprint and encouraged team members to elaborate the existing stories in more detail based on what they had just discovered. He also witnessed the team becoming self-organising as one team member corrected another at the Scrum for saying he would wait for his next task to be assigned to him and was able to get a sense for which individuals were under-performing and either needed extra support or were potential candidates for cost-saving.

It is important to realise that moving to agile is only a small part of the domain of accountability for a senior manager and it may not even be that important in the greater scheme of things. In my experience, some are comfortable to work with a coach, getting feedback and discussing the progress of their management team, whilst others prefer to rely on the traditional methods they have learned, commanding their managers and teams to be agile and steadfastly upholding the rightness of their approach. In both situations it is useful to make the data visible.

The CIO of a major investment house tracks improvement using two simple KPIs, the number of incidents raised and the number of releases made. These metrics are a useful indicator of the relative health over a group of applications as they show tends and differences. They show which areas of the business (each under an executive team member) are improving and which aren’t even trying. Within teams, a simple Kanban board showing the end to end process and the rate of progress of items through that process will naturally help focus attention towards improvement as it reveals bottlenecks as well as cycle times.

“Speak truth to power” – Quaker saying

Recently I got into trouble with a senior business manager by saying that we plan all the time in Scrum. He’d joined the end of a meeting whose purpose was to start healing the rift between IT and the business and he didn’t know we were discussing Scrum in general. Equally, I didn’t know that his background was spending two months a year planning the IT changes he thought his business colleagues wanted, followed by ten months of frustration trying to ‘get back on track’ whilst everyone else worked to keep the day to day business running. To reinforce his position he stood-up and said “I don’t care how you do it, I just want it done”. I should have taken that as a clear statement of value and advised my IT teams to deliver accordingly, but we really were trying to improve the engagement with the business, so I replied “but you’ve got to care, your survival depends on it”. Of course he stormed-off in a rage to demand I was fired for insolence, and I spent the next 72 hours trying to work-out what had just happened.

Of course, I knew that he was unused to people of the lower ranks expressing a view that wasn’t what he wanted to hear, and he was furious to hear it from someone in IT. Most people are sensitive to this and would not have challenged him, and they would have kept their job. Too often, managers are told what they want to hear and because it’s what they want to hear, too many managers fail to challenge the information. They don’t dig-down and see if there’s any valid data to back it up, and they don’t go and see for themselves what teams are doing. With hindsight, the point I wanted to make was that the business should care greatly about what we build. Next time I think I will stick to talking about the role of the business in prioritising for the good of the product itself. I may even agree with him that the business need not be concerned with the how.

Summary

Agility is a significant challenge for organisations comprising thousands of employees. They have evolved defence mechanisms that make change difficult and slow as a risk-management mechanism. Executives and senior managers are the instigators of change and must be prepared to lead some amount of personal change. Rather than trying to change culture, they should understand their own values and recognise the culture of their own organisation. We are in a transition period between the dominance of the machine model business and the purpose-driven business and senior managers should recognise that software development work brings a new order to the workplace, where managers are enablers and teams are producers of original and valuable mental output. Trust has a part to play and that is built on openness and communication. 

About the Author

Russ Lewis (@ConsultStorm) coaches some of the world’s largest organisations towards Predictable Agile Delivery. He’s a 'big picture' techie-turned management guru who now gets his kicks helping team members and managers succeed. He does this by engaging at all levels and facilitating the communications that lead to improvement. Russ architected and led Transport for London’s Contactless Fares payments system and the expansion of Oyster card to National Rail; supported the agile transformation at Amadeus SAS and currently spends his time helping to run a global DevOps program at an investment bank and speaking at conferences.