Jez Humble talked about organizational obstacles to moving fast at scale and how to address them at the GOTO Berlin 2015 conference. InfoQ interviewed him about how we can focus on value instead of costs, why having a shared understanding of an artifact can be very valuable, the waste which is there when you don’t know which features your customers will actually need and discovering the needs of customers quickly with low costs, and how to use the concept of improvement kata to improve continuously.
InfoQ: You mentioned that in most organizations people put more effort into estimating the costs than predicting potential business value. How did we get into this situation?
Humble: In my talk I quote Douglas Hubbard, author of the excellent book How to Measure Anything, who writes in an article for CIO magazine "Even in projects with very uncertain development costs, we haven’t found that those costs have a significant information value for the investment decision… The single most important unknown is whether the project will be canceled. The next most important variable is utilization of the system, including how quickly the system rolls out and whether some people will use it at all." We got into this situation because costs are relatively easy to measure, and business value (measured in terms of ROI, lifecycle profits, or whatever the metric is that you care about for your product) is much harder to measure.
What’s not sufficiently well understood though is how utterly unimportant initial development cost is. For a successful product, the ongoing development and maintenance cost will dwarf the initial development cost (companies typically spend 70-80% of their IT budgets on keeping the lights on and adding capacity to existing systems.) For an unsuccessful product (which is most of them), it’s a sunk cost. Unfortunately companies don’t tend to estimate development costs as a way of limiting their downside in the event the project is cancelled or the product isn’t used. If they did, they would be much better at developing prototypes to validate their product hypotheses with real users, which is exactly what the Lean Startup (following decades of good practice, beginning, ironically, with Winston Royce’s original "Waterfall" paper) advocates.
InfoQ: Even more important is how we can get out of this and focus on value in stead of costs. Do you have suggestions how to do this?
Humble: I think fundamentally it comes back to making sure you’re solving a real problem, in a way that users will pay for - either with money or with their time. Many products - including internal IT systems - do a pretty poor job here. With internal IT the problem is often made worse by standardization - people are forced to use systems that have not been designed with their needs in mind. As an industry, we have to take UX much more seriously by discovering the real problem we need to solve, and then determining the minimum possible amount of work we can do to solve this problem effectively. Jeff Patton talks about minimizing output and maximizing outcomes.
A lot of the time we jump straight to solutions and become obsessed with delivering them as expansively as possible. Tools like impact mapping can help by focusing on the outcomes we want to achieve, coming up with a number of possible approaches to achieve those outcomes, and then designing experiments to test which approach will achieve the outcome at lowest cost and effort, rather than jumping straight to designing and building a solution.
InfoQ: You mentioned that having a shared understanding of an artifact can be very valuable. Can you give an example?
Humble: In agile methods, user story cards are the classic one. As Ron Jeffries pointed out some time ago, a story card is designed to be a token, a reminder of a conversation. The token is not the canonical document - the shared understanding created by the conversation is. No amount of documentation can replace a shared understanding - requirements documents are fundamentally leaky abstractions (which is not to say they’re useless, just that they are only useful in the context of a shared understanding). Impact maps are similar - the point of an impact mapping exercise is the shared understanding generated by the activity, not the impact map generated, which is a token reminding us of that conversation.
The idea that the conversation and the shared understanding are primary is one that creates anxiety in a lot of people, particularly when we have to cope with building things at scale. The communication bandwidth required to create a shared understanding increases exponentially with team size. The answer to this problem is not more process - documents can’t replace shared understanding - rather, it’s careful architecture (both systems and organizational) to minimize dependencies between smaller, cross-functional teams. The rise of distributed systems has made this problem particularly urgent, and distributed systems architecture has important lessons for organizational architecture, but many organizations still haven’t taken these lessons to heart.
InfoQ: Can you elaborate on the waste which is there when you don’t know which features your customers will actually need?
Humble: I like to quote Ronny Kohavi, who has years of data from online experiments run at Amazon and Microsoft showing that 2/3 of features that were thought to be good ideas actually deliver zero or negative value. If you’re not testing your features, less than half of them will be delivering value to your organization. The others are an enormous source of waste - there’s the opportunity cost from not building features that would have actually delivered value, the cost of maintaining those features, and the complexity which they add to your systems that reduce the rate at which you can evolve them.
InfoQ: Can you give examples how you can discover the needs of your customers quickly with low costs?
Humble: Over the years, the UX community has come up with a number of ways to perform user research, from very low fidelity - paper prototyping - to very high fidelity (A/B testing). For me, the most important achievement of the Lean Startup movement has been sharing clever ways to test ideas. One of the most famous stories from Ries’ book describes how Zappos began without any supply chain - Nick Swinmurn would go to the store, buy the shoes that had been ordered through the website, and put them in the post.
With product development, as with science, designing good experiments is the hardest part, but it provides the best opportunity for rapidly discovering great products. Yet it’s still not commonly practiced or taught in business schools, and people see it as too difficult, perhaps because it requires creativity rather than implementing cookie-cutter processes.
InfoQ: You mentioned the concept of improvement kata in your talk. Can you briefly describe it, and show how it can be used to improve continuously?
Humble: The improvement kata is a scientific, experimental approach to process improvement that mirrors the experimental approach to product development I’ve been describing in this interview. It was written up by Mike Rother in his book Toyota Kata, and there’s lots of creative-commons material and workbooks freely available on his site. Rother has spent many years studying how Toyota is able to consistently stay ahead of the competition by making cars faster and more cheaply, at higher quality, than the competition.
What he found is that managers at Toyota aren’t in the job of telling employees how to do their work. Instead, people in the organization work together to set a long-term challenge (defined in measurable terms). Then, at the program or team level, they start by documenting the processes as they currently work, and then set shorter-term measurable targets for the processes that they think will get them some way towards the challenge. The key job of management is then to support employees as they experiment with ways to achieve those targets.
The point of the Improvement Kata is to make process improvement work a habitual part of everybody’s daily work. Rother points out that if we aren’t constantly working to improve our processes, they will gradually ossify - particularly in the face of changing conditions in our organization. Effective risk management is a living activity that requires constant work to evaluate and improve processes, and this can’t be done effectively in a command-and-control way. Instead, the people who operate the processes must have the authority to experiment with improvements to them.
What struck me most when writing my new book Lean Enterprise was noticing the parallels between this experimental approach to process improvement and the experimental approach to product development pioneered by companies like Amazon. Together, they provide a way to achieve highly resilient and adaptable organizations at scale - an alternative to both waterfall, and the current crop of scaled agile frameworks, which in practice do little to disrupt the highly siloed, command-and-control flow of information and decision-making that prevails in many enterprises today.
