Jan Bosch is professor of software engineering and director of the Software Center at Chalmers University of Technology in Gothenburg, Sweden. At the Bits&Chips Software Engineering conference he will give a keynote titled "From opinions to facts: building products customers actually use" and a regular talk on "Climbing the stairway to heaven: continuous integration and deployment for business success". The conference will be covered by InfoQ with news and articles.
InfoQ did an interview with Bosch about the benefits that companies can get from increasing delivery speed, the next steps that organisations can take after adopting Agile and DevOps, using experiments to innovate, practices for experimentation and how organisations can become more innovative.
InfoQ: Can you elaborate how companies can benefit from increasing their delivery speed? Why does it matter?
Bosch: Although many companies spend a lot of time focusing on strategy and have done several attempts to “predict the future” for their industry, the fact is that many developments are missed by the incumbents in industries. Of course, especially in Europe many refer to Nokia as the prime example of a company that missed the boat and was disrupted by new entrants. Nokia is often viewed as the exception and as an outlier. However, research into the Fortune 500 list shows that 52% of all the companies that were on the list in 2000 are now gone from that list. All the companies had strategies and long-term predictions, but still failed to miss the changing tides in the market. Having studied this in several companies and having actively lived it during my time in industry I know that the key differentiator for companies is to minimise the time between the identification of a customer need and having a solution addressing that need available in the market. Agility and responsiveness to rapidly changing customer desires requires a “Need for speed".
InfoQ: Many organizations have adopted agile, and now DevOps is hot. What's next?
Bosch: Based on our research, we have developed a model that we’ve called “the stairway to heaven” (see the book Continuous Software Engineering). This model is based on working with dozens of companies over extended periods of time, keeping track of how development practices have evolved in these companies over time. In the model, an organisation starts with a traditional, typically waterfall or iterative development approach. As a next step, the company adopts agile development practices for its R&D teams, but leaves the rest of the organisation unchanged. The agile teams often express frustration about their lack of ability to verify the software that they write, leading to the third stage: continuous integration. Once continuous integration is successful and the company always has a production-quality version of its software, the company often moves to the next, continuous deployment step. In this stage, DevOps becomes an important organisational principle as the frequent deployment of new software does not allow the traditional, slow handovers between the R&D and operations organisation. This then requires the development and operations responsibility to be in the same team.
However, few people seem to realise that continuous deployment is a means to an end and not a goal in itself. The actual goal is that it allows the company to move towards a different development model: Continuous deployment enables the use of A/B or split testing as well as other ways of quantitatively testing the relevance of new functionality to customers. This allows companies to adopt a development model that is driven by hypothesis-driven experimentation with customers rather than a traditional requirements driven development approach. The reason this is so important is because research by us and others shows that more than half of the features in a typical software system are never or hardly ever used. These features can be considered waste and should never have been developed in the first place. We need a development model that weeds out these features before the full development efforts to build these features has been committed. We have developed some of these models and refer to this stage in development as “R&D as an innovation experiment system”.
InfoQ: How you can organize R&D as an Innovation Experiment System?
Bosch: The main focus area associated with moving towards the stage where we treat R&D as an innovation experiment system is the relationship between product management and product development. Traditionally, product management focused on what to build and product development focused on how to build it. This division can work for organisations in predictable and slow moving markets. Unfortunately, fewer and fewer markets satisfy this requirement. In addition, many companies working in this traditional model operated in a model where a new feature or product would only get sufficient priority if customers explicitly asked for the feature or product. There is a, sometimes significant, time lag between a customer need appearing and the customer being able to verbally express that need. The time between a customer need being present and the customer being able to express it is when new market entrants have the best chance of disrupting the incumbents.
So, we can’t wait for customers to ask for features or product and we can’t just build stuff assuming that “if we build it, they will come”. So, how do we find out what features or products provide the most value to customers? The answer is constant experimentation with our customer base, either in the context of products already deployed in the field or through other experiments with customers. The way to organise for this is to bring product management and product development together in a single team. Although this is norm in basic agile methods, the product owner in many companies is a technical person, e.g. an architect or senior engineer, rather than someone actually presenting the customer. Each agile development team (satisfying the Amazon 2-pizza rule) also incorporates someone shouldering the product manager role. Together, the team comes up with hypotheses to test with customers and then develops and performs the experiments. This may result in features being withdrawn from the product because the customer feedback is tepid. Alternatively, it may result in features being only partially implemented when compared to the requirement specification from product management, but that, based on the data from product use by customers, are sufficiently developed. Also, it may result in features being implemented very differently from the original assumptions of the team as feedback from customers shows that this is required. Finally, of course some features may be implemented as specified as the team correctly predicted the most pressing needs of customers.
The key to this way of working is (1) for product management and development to engage in teams, (2) for both product management and development to embrace the uncertainty that follows from the way of working and (3) to build a culture where data trumps opinion.
InfoQ: Can you give some examples of using data to decide what to build or what not to build?
Bosch: In the Web 2.0/SaaS world, this is the norm and several companies in the SaaS domain spend more effort re-implementing already developed features to see if alternative realisations result in better business outcomes. For instance, for e-commerce sites, the conversion rate of customers coming to the website that complete a transaction. Some of the companies that I work with have now started to more critically evaluate features during and after implementation in order to determine if the feature should be kept in the product or if it should be removed. Also in the embedded systems domain, there are now examples of companies removing features after these features have developed as the added complexity was worse than the value provided by the feature. Unfortunately, I can’t share the specifics on these features due to confidentiality constraints.
InfoQ: What are the benefits that you can get from using this approach in product development?
Bosch: As we mentioned earlier, somewhere between half and two-thirds of the features in a typical software system are never or rarely used. Imagine that we could free up 50% of our R&D resources because we only build functionality that customers really, really want and that adds value to the product for them. Then imagine that this 50% of R&D resources is allocated to innovation and experimentation with new features in existing products and entirely new products. The impact on the competitive position of any organisation would be phenomenal. This presents the key benefit for companies employing this approach: the accuracy of R&D investments, in terms of return on investment, is dramatically improved. In the Software Center (www.software-center.se), we actively work with the partner companies, including Ericsson, Volvo, Saab, Jeppesen (Boeing) and a host of other companies, on studying this opportunity. For example, in some research that we did with a company in Gothenburg, we measured the frequency of “service starts”, i.e. times a feature is used. As the figure below shows, only few features are used very frequently. Many features are never or hardly ever used and could be considered waste.
InfoQ: Which kinds of practices can organizations deploy to experiment. Can you give some examples?
Bosch: There are many practices from which organisations would benefit when adopting, but I will focus on three practices here. First, the organisational interface to the customer has be broadened dramatically. Traditionally, R&D staff never meets a customer unless there is an unexpected problem that appeared at the customer site. If R&D teams are supposed to come up with insightful hypotheses about valuable, new customer functionality, they need to understand the reality of the customer and the deployment of their products. This requires R&D staff to spend time with customers and at customer sites. Not just some people, but everyone has to spend some time at a customer at least every year. This may seem like a costly exercise, but the value of increased understanding and customer empathy far outweighs the cost. For example, at one of my earlier employers, all staff at the company was required to spend at least one day per year at a customer, shadowing the customer through his or her work day.
Second, organisations need to adapt a data-driven way of working. This requires instrumentation of existing and new products in order to determine feature usage. Our research shows that all companies we work with collect enormous amounts of data, but this data is predominantly used for troubleshooting. R&D is not provided access and if it has access, the teams do not use the available data. In the words of Edwards Deming, we need a culture characterised by “In God we trust; everyone else bring data”.
Third, most organisations work in functional silos and collaboration across these silos is slow, frustrating and fraught with failure. For companies aspiring to be agile and data-driven, it is not sufficient that product management and development work effectively together. The entire organisation, ranging from sales and marketing to the release and delivery teams need to effectively work together. This requires moving away from traditional, bureaucratic organisational structures to empowered, self-managed teams with unprecedented amounts of autonomy. In the management literature, there is a novel stream of publications exploring organisations where individuals are self-managed and self-directed and where the organisation operates without hierarchies, bosses, mandatory internal decision processes. Instead, these organisations have found new mechanisms to coordinate the work in organisations and this is broadly viewed as the next phase in structuring organisations. A well known example of a company that has gone all the way to building a self-management is Valve, the game company responsible for Dota, Portal and Half-Life.
InfoQ: If organization want become more innovative, which advice can you give them?
Bosch: There are three things that I would like to get across.
First, innovation requires that individuals have time to spend on innovation. So, allocating your staff 100% or more and then telling them to innovate as well is just an illusion. Several Silicon Valley companies have institutionalised 10-20% of employee time to be freely allocatable by employees. If you want to use the creative energy of everyone in the organisation, you need to give them the space to do so.
Second, many innovative ideas get killed based on internal feedback and selection processes driven, typically, by senior leaders. The principle needs to be that innovative ideas should be dropped ONLY if a sufficiently large sample of customer feedback shows lack of support for the idea. The measurement stick should be the customer, not the opinions of the HIPPOs (highest-paid person’s opinions) in the organisation.
Finally, build a culture where experimenting with innovative ideas that do not live up to their initially assumed potential is accepted. I don’t like to the focus on failure that some thought leaders have, but punishing people in the organisation for trying out things that do not pan out is the fastest way to kill an innovative culture. Of course, we prefer innovations that lead to significant benefit for the company, but the only way to get there is to try out many more ideas and innovation and select from those based on customer feedback.
About the Interviewee
Jan Bosch is professor of software engineering and director of the Software Center at Chalmers University Technology in Gothenburg, Sweden. Earlier, he worked as Vice President at Intuit in Silicon Valley and Nokia in Finland. He is also active as a consultant for leading companies and in the startup space as a board member. More information about his background can be found at his website: www.janbosch.com.