In preparation for the upcoming DevOps Enterprise Summit in London, InfoQ sat down with Dr Nicole Forsgren, CEO and chief scientist at DORA, and discussed the fundamentals of DevOps, the challenges of specifying appropraite business goals, and how to measure organisational high performance.
Forsgren began by stating that what many are referring to as "DevOps" is the most relevant and most important set of practices and tools to drive technology transformations in organizations of any kind and any size. DevOps is successful because it is a wholesale change encompassing technology, process, and culture - the output of which is a technology transformation that drives value to organizations through an ability to deliver code with both speed and stability.
There is no "one metric that matters", suggested Forsgren, and instead metrics to capture depend on what an organization does, and what is most important and relevant to the organization at the current time. Organizations should collect metrics with outcomes in mind, and this often means collecting the goal/outcome metric, as well as a few other metrics that are believed to directly impact that goal. Key metrics for IT performance capture speed and stability of software delivery, and include: lead time for changes (from code commit to code deploy), deployment frequency, mean time to restore (MTTR), and change fail rate.
The transcript of the full interview can be found below, and more details on the talk from Nicole Forsgren, Nigel Kersten, Jez Humble, and Gene Kim, "The Key to High Performance: What the Data Says", can be found on the DevOps Enterprise Summit (DOES) London website.
InfoQ: Welcome to InfoQ, Nicole! Could you share the executive summary for your DOES EU 17 talk please, and what attendees can expect to take away from the session?
Nicole Forsgren: Thanks for having me! I'm very excited to be speaking at DOES EU 17. This year, I'll be joined by Nigel Kersten, Jez Humble, and Gene Kim -- my co-authors on the 2017 State of DevOps Report, which is presented by Puppet and DORA. We will be presenting some key takeaways from this year's research, as well as some behind-the-scenes notes about surprises from this year's study and what goes into the science of the report. It should be a lot of fun!
InfoQ: The word DevOps itself can often be overloaded and means many things to many people; how do you define DevOps?
Forsgren: I define DevOps as a technology transformation that drives value to organizations through an ability to deliver code with both speed and stability. Its essential components are technology, process, and culture.
InfoQ: How have the results from the State of DevOps changed over the years, and has this surprised you?
Forsgren: I think my favorite thing about doing research is designing research to test our hypotheses, and then getting the results in to see what new things we discover. Each year, we've continued to confirm key findings -- like the importance of software development and delivery to organizational performance measures of productivity, profitability, and market share -- while also growing our understanding of what drives high performing technology teams. This year, we expanded our research into areas like architecture (digging deeper into preliminary findings from earlier research) and leadership.
InfoQ: In your experience, are academia and industry collaborating effectively to investigate the role DevOps can play? Do you believe academic institutions themselves are embracing DevOps?
Forsgren: As an academic and a real believer in DevOps as a value driver, I'm always eager to see more collaboration! Some fields in academic research are still catching up to the DevOps work we're seeing in industry, while other areas are embracing it.
In regards to the teaching aspect of academia, we're also seeing some really innovative things happening right now, and how learning content is developed and delivered. I'm not sure academics would call it DevOps, but I see key aspects of it playing a role (technology, process, and culture!), and I think adopting more of these principles would be a great benefit.
That said... academia is a different institution than software and has its own challenges, so adopting a DevOps-like model will take some creativity and flexibility. I try to stay engaged in both the research and teaching sides of academia and I can't wait to see where the roads lead, and if paths converge.
InfoQ: What type of metrics/KPIs should an organisation be tracking before and during any migration towards becoming a high performing organisation? Is there 'one metric that matters', which indicates an organisation has been successful?
Forsgren: As a general guideline, I always suggest organizations collect metrics with outcomes in mind. This often means collecting your goal, or outcome, metric, as well as a few metrics that you believe will impact that goal.
For organizations in technology, I really push to the IT performance metrics we've identified in our research, because we've found they drive value in so many areas. These IT performance metrics capture speed and stability of software delivery: lead time for changes (from code commit to code deploy), deployment frequency, mean time to restore (MTTR), and change fail rate. It's important to capture all of these because they are in tension with each other (speaking to both speed and stability, reflecting priorities of both the dev and ops sides of the team), and they reflect overall goals of the team. These metrics as a whole have also been shown to drive organizational performance.
Is there "one metric that matters"? -- Nope. There's no such thing. At least not that I've ever found. The answer is always, "it depends." It depends on what your organization does, what's most important and relevant to your organization, etc. For example, one good candidate might be Net Promoter Score (NPS) -- but that assumes you work in an industry where referrals are important. This metric might not be relevant (and maybe not applicable at all) for public goods and government services.
The other challenge with single metrics is that they can be very easily gamed. I prefer two or three metrics that are held in tension, making it difficult to be myopic in your focus.
InfoQ: In summary, how relevant do you believe DevOps is to modern organisations looking to move faster? In your experience, what is more important in a typical enterprise space: the need for organisational change, or technological change?
Forsgren: At least right now, I think DevOps (or whatever someone wants to call the methods and practices many of us are calling "DevOps") is the most relevant and most important set of practices and tools to drive technology transformations in organizations of any kind and any size.
We haven't found anything that is better: waterfall methods are too slow to keep up with the pace of business and security/compliance/regulatory changes in today's environment. Traditional agile methods are a great next step out of waterfall, but they don't scale. DevOps is delivering real value for organizations that want to move fast and deliver quality software in reliable ways. We may find a better method in the future... but right now, DevOps is best of breed.
For the second half of your question, again, I'm going to have to give you a bit of a non-answer! DevOps is successful because it is a wholesale change encompassing technology, process, and culture. All are important. The first step that is most important will be unique for each company, because it will depend on their current environment and context.
InfoQ: Many thanks for your time today. Is there anything else you would like to share with the InfoQ readers?
Forsgren: Thank you! I'd encourage the readers to check out the upcoming State of DevOps Report, where we include lots of additional insights and tips around some key capabilities to really drive DevOps transformation success.
The DevOps Enterprise Summit London conference runs June 5-6th at the QE II Conference Centre. Additional details can be found at the IT Revolution Events website.