In the blog post how to create the agile enterprise, principal analyst Michael Azoff summarizes Ovum’s view on creating an agile enterprise. Michael explains what the concepts and goals of an agile entreprise are:
The concept of the agile enterprise arises from practices that embrace common principles and values, such as putting uppermost the delivery of value to the customer and high-quality products.
The ultimate goal is business agility, where the mainstream business processes have adopted agile ways of working and where IT use is optimized, from “run the business” to innovation, and from tactical requirements to long-term strategic requirements affecting the future of the business.
According to Ovum, large enterprises face three primary challenges:
(...) first, the most successful enterprise eventually hits the innovation wall and questions how to break through to continue to innovate and act as nimbly and creatively as a start-up. Second, the challenge of responding to changing market conditions and keeping an organization’s strategy in touch with these changes is hampered by the annual budgeting process. Third, software development has been transformed by agile methodologies and the next step is to transform the whole IT department.
The Ovum report “Agile and Lean Business Transformation” provides insights on how to transform a business using agile-lean principles. InfoQ did an interview with Michael Azoff of Ovum:
InfoQ: Thank you Michael that you want to do an interview with InfoQ. Could you shortly introduce yourself to the readers of InfoQ?
Michael: I started out in academia working in solid state electronics then moved into industrial R&D and neural networks. After an entrepreneurial spell and IT consulting I started working as an IT industry analyst in 2003. My immediate colleagues have stayed more or less the same over the last ten years although through a series of acquisitions the company name has changed: we are now Ovum, part of the Informa group. At Ovum I’m a Principal Analyst and lead the software development and lifecycle management research, this includes methodologies and processes, especially agile and lean related topics.
InfoQ: The Ovum report explores three areas for creating the agile enterprise: innovation management, organization management, and IT department agility. Why are these areas so important?
Michael: They are important because they represent three frontiers that organizations today must address to survive in today’s economic climate; not just the playing out of the recent financial crisis but increasing competition from economies globally and the changes to commerce that the Internet is demonstrating. Richard N Foster has examined the longevity of S&P500 companies and they are reducing from an average over 70 years in the 1950s to 18 years and decreasing today. Therefore if successful companies are to remain successful innovation must be a high priority. What caught my attention recently is innovation in the process of innovation, such as Lean Startup. The second area, organization management, is another big topic in its own right with countless management gurus and MBA courses peddling their versions of the truth – what for me is interesting is that there is a movement called Beyond Budgeting which has a lot in common with agile and lean thinking, and it strikes me that for the right organization culture it could make the difference in surviving beyond those current 18 years lifespan. Finally, IT department agility is critical for organizations given the pivotal role IT has in businesses today; being agile can improve the gap between strategy and vision and the reality of what is delivered. Agile practices originated with developers and is being grown through the rest of the IT department by DevOps – the end result is ideally agility and leanness throughout the IT function.
InfoQ: The lean startup approach, used by small start-ups, is mentioned as a solution to innovation management. How can you apply lean startup within an enterprise?
Michael: You need to start with a champion at a senior enough level to gain a budget and be allowed to run the incubator without interference. There needs to be clear demarcation lines drawn up to protect the reputation and lower risk to the parent company brand, so what the incubator is allowed to do and how to do it must be addressed, but once the relationship is defined between the parent and incubators (there could be multiples), then the incubators have the freedom to act as startups. In my case study of Adobe, what made the difference between innovation within the existing product groups and the lean startup incubator was that the product groups rarely innovated beyond the bounds of their existing products, whereas the incubator used its freedom to create fresh new products.
InfoQ: Your blog mentions “customer development before product development”, what does this mean and how can this be done?
Michael: This is a key insight that Steve Blank had and which influenced Eric Reis and his Lean Startup ideas. Essentially, when you are dreaming up a new product you have to discover who the customer is: there are plenty of examples of failed products where the innovation into the product failed to understand whether anyone would want to buy it. As a side note, many businesses do not record their own failures and with personnel changes and passage of time, may end up repeating those mistakes, hence the success of the museum of failed products (mainly for supermarket goods): they simply purchase every new product that appears on supermarket shelves. The yogurt shampoo is one example. So customer development is about satisfying yourself that there is a genuine market for your new product and with the size of growth that you desire. The Lean Startup process may well change the original product that kicked off the innovation cycle; that is the point of customer before product development, to prevent wasting resources on an untested product idea. In case you think this is just market research – the Lean Startup model of customer development is designed as a scientific experiment to test ideas and be driven by data and evidence; if the idea falls flat, then you need new inspiration for the next iteration.
How it is done depends on your market, so for web based businesses a good approach is cohort analysis, which follows groups of Internet users of your new web site and examines their behaviour. As you make changes to the web site you can see what effect this has on users and refine the service/product to achieve the desired level of usage and revenue.
InfoQ: Once you get a better understanding of your customers, are there ways to get more insight in the potential business value (business case) of the products or services that you want to develop using lean startup?
Michael: When organizations research new products they assess carefully what level of return they are likely to earn: the big IT players may turn down perfectly viable products or services that other companies would die for simply because the level of earning is not high enough given the resources and overheads involved in a large enterprise. A key metric in Lean Startup is the rate of growth and that should match business expectations. Measuring this metric objectively is what Ries calls Innovation Accounting and cohort analysis helps in interpreting results.
InfoQ: Annual budgeting in enterprises can hamper their response to changing market conditions, as you mention. Are there alternatives solutions to how budgeting is done?
Michael: I like the analogy that describes the annual budget process as being like a bank that is open for only one week in the year. The faults with this approach can be summarised as: 1. A lot of the resources going into the process are wasted because as time progresses through the year, disconnect grows between the assumptions that go into the budget and reality. 2. Performance targets that are based on the budget forecasts similarly become outdated. 3. Limiting budget allocation to a short window in the year limits the ability to respond to changing market conditions. Companies have been using a variety of alternate approaches, such as using rolling forecasts with quarterly budgets. The team behind Beyond Budgeting (BB) investigated companies with some of the more successful alternatives, and pulled these ideas together to go beyond just budgeting but also into transforming the management of organizations.
InfoQ: Can you explain what Beyond Budgetting is, and how it can help enterprises?
Michael: Beyond Budgeting (BB) covers a lot of territory, but to summarise: the starting point is the abandonment of an annual budget cycle. Of course there are still the periodic reports that companies need to publish for compliance reasons, but internally budgeting can be made continuous, based on needs and opportunities, linked to rolling, or even better, dynamic forecasts. Secondly, BB advocates a customer led management process, in contrast with command and control – so this is about changing organization culture, giving front-line managers greater freedom to make decisions without everything going back to the center and waiting approval. Thirdly, BB improves the work environment for most people within an organization – it makes for a better motivated work force. It helps enterprises by addressing the three main faults with traditional budgeting described above.
InfoQ: Changing budgeting processes can be hard in large organization. Do we really need to do this, which benefits can it deliver?
Michael: It is fair to say that the traditional annual budget is for many companies a time consuming and painful exercise, so there is already a readiness to hear out alternatives. Going further and dropping a command and control culture depends on the CEO and his/her vision for the future of the business, as well as how far the existing culture needs to change to being a BB culture. This question of organization culture will be treated in a separate Ovum report – but for now let’s say that there is no simple recipe and it is not always right to abandon command and control if doing so will break the company – that makes no sense, you need to carry the work force with you. Businesses typically start with BB by picking a subset of ideas and trialling them first before exploring further.
InfoQ: You talk about “taking agile and lean to the whole IT department”, but what about organizations that haven’t started yet with agile development teams. Should they first change the way they develop their software, or is it better to start by adopting agile and lean for the whole IT organization, end to end?
Michael: The first thing to say is that if it isn’t broken don’t fix it. If waterfall works for you, that’s fine. The point here is that waterfall is a problem for many software development teams using it, so agile and lean ideas have a receptive audience. (The same can be said of the annual budget process). An agile IT department means agility in development, DevOps in operations, and the drawing in to agile/lean of all the remaining silos: network engineers, database administrators, IT services, web designers and so on, plus all their managers. Developers practising agile can act as a catalyst, and the usual sequence is for DevOps to follow once the agile developers are increasing the frequency of drops into production and operations is under increasing pressure to improve its efficiency. To be an effective agile IT department requires a closer relationship with the mainstream business than typically exists today. This is easily understood in tech companies where the mainstream business is IT, but for other industries the synergies that arise from agility and leanness across the whole business is rarely co-ordinated – more common is for agile developers and BB advocates within an organization to discover each other by happy coincidence. Traditional annual budgeting is a big bang/waterfall-like process and it hampers the way in which the business funds its IT projects. So there are many opportunities for improved engagement processes to be created that reduce the business-IT gap, better align IT and business strategy, and so on. This has been a goal for years but we now have an opportunity to figure out how to achieve it.
InfoQ: The roots of agile methodologies are in software development. When the whole IT department should become agile and lean, does that mean that agile is scaled up from teams to departments and to Management Teams (bottom up)? Are agile methodologies suitable for that?
Michael: Agility in the IT department beyond the development activity does not necessarily follow a Scrum or other established agile development methodology to the letter. Foremost it is about sharing common values and principles, from that can stem new process ideas. In the case of DevOps where operators have to write, say Perl scripts running to thousands of lines of code, it is possible to directly apply an agile development methodology. Also, some organizations have adapted Scrum to business processes. The key is to think about how to apply agile and lean ideas in novel situations.
InfoQ: What are the biggest barriers if you want to adopt agile and lean? And how can you deal with these barriers?
Michael: There need to be reasons for change from whatever existing process is used. The existing pain points need to be identified and a new process defined that addresses these pain points. So having a vision and shared understanding as to why a change is being implemented will overcome the inertia to change that often kills organization change initiatives. If there is strong motivation at the root level but management is not involved and/or indifferent then the change will be limited. At a minimum you need motivated individuals at both the senior management level and at the practitioner level, without that risk of failure is high. With such support there are still challenges but there are ways of dealing with them. The single most critical barrier to any organization change is its culture and how compatible the new process is with that culture. Changing organization culture is a separate challenge from transitioning to agile/lean, and a high risk activity.
InfoQ: Recently there is a lot of talk about DevOps. How can DevOps help to adopt agile in enterprises?
Michael: The DevOps movement is inspired by the agile activity in development and helps to break down the silo wall that traditionally exists between development and operations. It has also led to new thinking about improving operations, so for example new tools in release management and automation have emerged. So DevOps helps spread the agile message (as encapsulated in the Agile manifesto) and goes a long way to help transform the whole IT department to be agile.
InfoQ: If an organization is interested to take the next step to become more agile and lean, what would be your advice to do this?
Michael: Ideally the initiative is best originated from the C-level executives, but if not a case should be put before them and win their support. So with support from the highest level in the organization then the next step would be to devise a change management programme. This can be done in an agile way, see for example Mike Cohn on agile change management. The organization should bring in agile and lean experts and coaches to train and mentor the work force. I use a metaphor in my report, representing an agile enterprise as a mountain, with many different paths to the top. Selecting the combination of Lean Startup for innovation management, Beyond Budgeting for organization management, and agile and lean adoption throughout the IT department, is one path up this mountain to achieve an agile enterprise.
InfoQ: You mentioned that you can do the change in an agile way, but starting with the highest level in the organization to take a next step appears to me as top-down and somewhat “waterfall”. Are there alternative (and quicker?) ways to get an enterprise into a mode of change?
Michael: Depends if we are talking about an organization-wide change to agility or just an IT department transformation. In the former case you must have buy-in from the highest level, so yes, it’s top-down, at least initially. The change process can be made quite agile, for example by engaging deeply with the people affected, by doing the changes iteratively, by being transparent etc. In the latter case you can go a certain distance without the board of directors being involved, but it helps to have champions at the top.
On the question of how fast you can introduce the change to agility, whether you have a big-bang swap over day or introduce changes incrementally depends on how compatible the existing culture is with the proposed changes and also practical considerations. For example Statoil switched to continuous budgeting in a big bang fashion across its multiple global divisions because running two different budget processes within the same company would have been twice the work and twice the confusion.
Final thought: for a successful change programme you need motivated individuals throughout the organization at all levels, who have the vision, no change process will work without people that share the vision.
About the Interviewee
Michael Azoff (PhD, MIEEE) has been working as an IT industry analyst since 2003, bringing over 20 years of experience in pure and applied research and consulting in the IT industry. At Ovum he leads the software development and lifecycle management (SDLM) research and his current focus is on agile and lean practices in software development, including enterprise agile transformation initiatives, DevOps, cloud related SDLM, application performance management, and enterprise IT mobile development.