DevOps and Cloud as Catalysts for Business Performance

The new Accelerate State of DevOps Report, released in August 2019, led by Dr. Nicole Forsgren and Dr. Dustin Smith, presents the latest research results, presenting the capabilities and practices that contribute to software development and organizational performance. InfoQ interviewed Dr. Nicole Forsgren about the 2019 Report key findings and about the most successful DevOps and Cloud strategies and techniques that drive business success.

InfoQ: Congratulations Dr. Forsgren on releasing the latest Accelerate State of DevOps Report! This is a colossal initiative and this year’s report is absolutely amazing. Thank you for doing this interview for InfoQ. What are the key takeaways you would like to share with our readers?

Dr. Forsgren: We have so many great findings this year:

• The industry continues to improve, and excellence is possible for anyone. We found that the highest performers, the group of elite performers, almost tripled. That is really exciting and it confirms what many other analysts are reporting.
• Another thing that we've done this year, which we haven't done in the past years, is dug into productivity in addition to performance.
• Just as we’ve done in previous years, we investigate a mix of cultural, process and technical practices that contribute to performance. We're able to revalidate several findings; for example, Cloud is a critical differentiator.
• We see a really strong set of findings around how change approval should be happening. Heavyweight change approvals have a negative impact on performance, and a clear change approval process positively impacts performance.
• This year we also added a really nice piece about scaling transformations.

InfoQ: One of the key findings you mentioned is that DevOps has "crossed the chasm". Could this milestone trigger other companies to have a wake-up call and follow the trend faster? This might be an opportunity for those really large corporation who have challenges understanding the importance of DevOps, Cloud and automation to start exploring these areas.

Dr. Forsgren: The nice thing about any type of technology, it hits just what you're saying right there: it becomes a safe move. It's no longer a gamble, you're not at the bleeding edge. We now understand a lot of things about what makes technology transformation successful. I've been running the State of DevOps Report for six years now, so it isn't this crazy newfangled thing. It makes it safe for so many organizations to say: "absolutely we're going to invest in the cloud". Highly regulated organizations are now in the cloud, and we can put our data in the public cloud safely today.

In many ways, it’s not only, we know it's safe and secure - we know it's safer and it's more secure. This is a great place for us to go and we know the capabilities and practices in terms of automation, investment, CI/CD, what CI should look like, what CD should look like, and what our processes should look like in order to have very high likelihood of success. So now, from an executive standpoint: if I'm making large investment decisions, I know that I have a high likelihood of this being successful.

As an example of this: I've spoken to several banks and they say "you know we got burned years ago, and I don't want to be right at the edge of the curve, I want to be a fast follower, I want to be the first person in that second wave". We’re far enough into DevOps transformations that this is a safe and smart move, and even these risk-averse and cautious companies are adopting these methods and hustling to catch up – and taking advantage of the decade of learnings we have from the rest of the industry.

InfoQ: That's great to hear! We know that scaling DevOps is possible when we make the right investments in technology, process, and culture, and leverage the cloud. Why are we still not investing enough in automation and what can we do to help organizations understand the investments they need to make, and how they can get better at it?

Dr. Forsgren: Developing and delivering technology requires investment and execution, and the purpose is to deliver value to our customers. The allegory that I make sometimes is: if we want a better performance, we need to make the investment, and we also need to execute on the investment. If I have a gym membership, I also need to go to the gym, and I also need to work-out.

I see the following two common disconnect:

• First, from the perspective of some executive or leadership levels, we sometimes believe that we can write a check, walk away, and still make it work. In reality, we also have to change the way we are developing software so that when we're developing and delivering software in the cloud it actually substantially changes the way we're delivering value.
• The second disconnect is how we talk about adopting and implementing cloud computing. Sometimes executives will say: "We made the investment, but we haven’t seen a return." So, I ask: "Well what do you mean by cloud?" We may talk about cloud with 15 different companies, and we will receive 15 different answers about what cloud is to them. So, we used the NIST definition of what it means to do cloud computing. Clouds are not interchangeable, and the major cloud providers have distinct differences. But even with those differences among the cloud providers, you have to architect and automate to use and access your cloud the right way:
  • On-demand self-service: It needs to be on-demand and support full self-service when someone needs to provision compute resources. You have to be able to provision and access it immediately and quickly, otherwise you lose the benefit of being on the cloud.
  • Broad network access: You need to be able to access your network across different devices, phone, laptop, tablet.
  • Resource pooling: And then we need to be able to pay only for what we use.
  • Rapid elasticity: I joke that this is "bursting like magic" – up and down.
  • Measured service: You only pay for what you use; most public clouds handle this.
• There are a few pieces that organizations tend to really struggle with – it’s often the self-service piece and broad network access. If we don't change the way we're accessing and using our infrastructure, we won’t get the full benefits of moving to the cloud. I’m going back to my allegory: I can pay for my gym membership but if I don't go to the gym and execute, I won’t get the benefits.

InfoQ: You mentioned in a 2017 interview with InfoQ that you have been expanding your research in other organizational areas such as leadership. At DOES 2017, you presented that year’s research and talked about transformational leadership versus servant leadership. Can you tell us a bit about that?

Dr. Forsgren: Organizations have performance goals, and they are trying to drive business outcomes. So, we wanted to study a model of leadership that is better aligned with outcomes. When I was preparing for that year’s research, I found that much of the academic literature and research showed that the transformational leadership model is more appropriate for driving performance when compared to the servant leadership model. The hypothesis (of transformational leadership driving outcomes) hadn’t been studied in a context like technology transformation before so we tested it, and we were able to find evidences that transformational leadership is impactful in technology transformation. We haven’t continued that research stream because there are so many things to research every year!  

We do note that transformational leadership does not require people to have direct reports, which is important because DevOps has a background of grassroots culture. Anyone can be a champion and a member of a team and help inspire and lead their group or organization. We highlighted this in detail in Accelerate.

InfoQ: About the culture of grassroot efforts, I really like the analysis you did looking at organizational strategies for scaling DevOps transformations. Can you talk a bit about that?

Dr. Forsgren: Thanks! This was a great section that was inspired by one of our advisors, Sam Guckenheimer. This analysis was led by Dustin, and I love the patterns he uncovered. He started by looking at broad overall patterns, and we noted some common patterns, like Big Bang isn’t used much (and is used least often among elite performers). Then Dustin and I chatted about wanting to understand what scaling strategies the high and elite performers are using – that is, if someone wanted to ask what the best are doing, could we tell them the patterns? And four clear patterns showed up among the high and elite performers, which echo what I’ve seen in the enterprises I’ve worked with:

• Community Builders: 46% - focuses on Communities of Practice, grassroots, and Proof of Concepts (PoCs)
• University: only 9%– focuses on education and training, with Centers of Excellence, DOJOs (Training Centers), Communities of Practice
• Emergent: 23% - focuses on grassroots and Communities of Practice
• Experimenters: 22% - has high levels of activity in strategies except Big Bang and DOJOs

Overall, we see that most of the strategies they use leverage structural solutions that build community. By doing this, they have built organizations that are very resilient to any type of change, like departmental reorganizations, product strategy changes, etc. The company can continue to pivot when the market conditions require them to. (I’ll point people to the report for more details on what the definitions of each strategy was and the patterns.)

I would note that these efforts still require resources, whether that's money for lunches or brown bags, or resources for time such as 20% time.

InfoQ: A question about maturity models. I really like how you described maturity models. I completely agree with you, I think they come from a very Waterfall perspective and approach, We are very used to having these maturity models and roadmaps that tell us how and when we are going to become more mature, which definitely suppress DevOps teams’ ability to innovate as quickly as possible.
Why do we still develop those DevOps maturity models spending countless hours defining criteria? We end up not following them in most cases. Like Waterfall project plans, they never get updated and prevent teams from exploring and accelerating.

Dr. Forsgren: There are a few reasons why they exist in some organizations. They are easy to sell, because they are very comfortable and very reassuring. Then leaders want to know what to expect over the next two, three, five years. If you give me a maturity model that says where you are today and where you will be next, and next and next … That is incredibly powerful because it gives me a sense of security, it gives me a sense of reassurance, and it allows me to plan for the long term. It also kind of like imbues someone (the person giving you the model) with a sense of expertise. The reality is that they are all made up. All of them are made up, just as CMMI is made up. We put varying degrees of experience into it, some organizations will put a lot of planning and trying into them, but they are all made up.

Instead, what we should be doing are constraints-based models which say:

• This is where I am now.
• These are the things that I know will make me better.
  • This list can get a little overwhelming, when we identify 25 to 30 things that will get us better. Narrow down and prioritize that list to targeting the biggest constraints. You can identify your constraints a couple of different ways:
  • You can use a process-based constraints model with value-stream mapping. In this method, you map out the value stream, describing time in time out where we waste time, and you identify the process that is the biggest constraint.
  • Another option is to use a capabilities-based constraints model. In this method, you list the capabilities that contribute to important outcomes (usually speed and stability). DORA’s research does this, and you can find these in our free State of DevOps Reports, or the book Accelerate. Then you can either use a tool to algorithmically identify your constraint (DORA’s Assessment does this), or you can ask your teams which capabilities the biggest headache or burden are to delivering software.
  • Some teams use both methods.

The challenge here is that you basically told an executive you only know your roadmap out for the next year-ish. It’s not comfortable. And not only that, we are now suggesting we only know the answer for a year, which can imply we aren’t the experts. But here’s the thing: we ARE the experts. Because now, we are the ones who know how to diagnose the real problems and move ahead. It’s powerful.

I work with many executives, and instead of selling them a maturity model, I teach them how to create their constraint-based model themselves customized to the needs of their teams, and I follow up on progress.

I will say that maturity models are good when you're talking about a single tool. You can measure awareness of a tool or its basic install, or we can talk about people progressing along one technology adoption through to expertise. But if you're talking about something like a technology transformation, it's too complex and there are so many things at play, and you're tying organizational constraints on top of it.

InfoQ: Going back to the result of the 2019 State of DevOps Report, the research established that both throughput and stability are possible without trade-offs. Do you think that both are possible without trade-off only if we do automation?

Dr. Forsgren: For the last six years, our analysis has established that speed and stability go together. In this research, I collect two speed metrics and two stability metrics. We have observed every year that speed and stability move together.

I will point out, though, that speed and stability move together without trade-offs at the high-end, which are the High and Elite performers, and they also move together at the low-end of the Low performers. So, the Low performers struggle on speed and they struggle on stability.

To answer the next part of the question, is automation required? No! What you were implying, and it's important to call this out, is if you want to get better, yes automation is required but if you want to keep being awful at speed and stability, because there are no trade-offs, automation is not required. If we don’t invest in automation, we’re holding ourselves back all over the place. It's a really interesting and smart observation on your part there. So, we do need to be increasing automation and I do think increased automation throughout the software development and delivery process will help us increase performance and business outcomes. It's not the only thing, there are several things that contribute to that performance, it is also processes, things like working in small batches which help us get smaller pieces of code through, it is an improved culture, it is the use of cloud computing.
Automation shows up in a few places: having good automated test suites, and deployments automation for example.

The other nice thing about automation is that it gives you repeatability, auditability and it ends up having huge implications in terms of security posture. So, you have great downstream effects in terms of quality. It also gives you faster feedback loops in the process and so your code gets better because if it’s fresh in my head, and I get the feedback sooner, I can improve my code immediately instead of getting feedback three months later and having to figure out what I was thinking. Automation has a lot of really nice benefits.

InfoQ: A follow-up related question: why is it difficult for organizations to invest in automation? I interviewed Jeff Sutherland, the founder of Scrum, and he was sharing the same feedback. He works with large corporations and he was sharing the same statement about automation. Do we need to hire people with automation skills?

Dr. Forsgren: We don't need to hire people with automation skills, I think we need to hire people with a growth mindset who are smart and who want to learn to do automation. People who are brilliant and who can brilliantly identify and understand the process, will figure out how to code. I'm not worried about people, I'm worried about organizations who understand the value of automating different things and instead they'll just stick with the way things currently are, whether it's a change approval process or something else, because that's the way it always was.

We're never going to run out of opportunities for automation particularly when we have very bright creative people. We find opportunities to automate and then new things bubble up, there's always something out there.

InfoQ: Does automation contribute to boosting developers’ engagement, because they get to use the latest technology?

Dr. Forsgren: I think increasing and having the ability to do more automation boosts morale for many people.

There is a subset of the population who are sometimes worried that it might threaten their work, but I think that is often an opportunity for management to clarify that they are not going to lose their jobs.

If anything, what this means is that they are going to get to do more exciting work because there are exciting challenges to solve. If there's a piece of your work that you can automate that just means that it's repetition and there's something new and creative that hasn't been figured out yet that we need your expertise to figure out.
I worked with a handful of large organizations that have adopted that approach to help shift that narrative and shift that mindset.

InfoQ: How can I leverage the Report in my own organization to help us fill the gaps or develop a roadmap?

Dr. Forsgren: We designed and structured the 2019 Report so that anyone can take and print one or two pages at a time and use that as a direction or a guide. I saw several companies online that printed one or two (or several!) pages and put them up on the wall at work, such as the cloud page. People also take one or two pages and turn them into a slide deck. That's the reason why the report is totally open for citation and we don't have any restrictions.

If you want to see what predicts SDO performance as an example, you can. I've actually seen pictures that people have been posting on Twitter. They print that out and circle their current focus for the next six months.

InfoQ: The 2019 Report presents a lot of valuable data. One of the challenges for leaders and executive is how to apply that research and data into our roadmaps and day to day work. Are you looking at developing open source blueprints or guides to help organizations get started and apply your research in their groups and organizations?

Dr. Forsgren: In terms of resources, the Accelerate State of DevOps Report is our key synthesis for the industry community. We are also preparing academic peer reviewed papers, but these are less appropriate for mass consumption. We also have an executive summary that we are in the process of releasing.

We are currently working on some blueprints to help organizations scale their transformation and adopt the cloud based on our research.

InfoQ: You mentioned earlier that you like research, and you like to research a lot of things. How did you get into research?

Dr. Forsgren: My background was actually technical. I started out as a programmer and worked as a developer for years. I was also a system administrator for few years, and I did a little bit of consulting. I realized that some of my problems categorically looked very similar, and research aims at answering categorical problems, so that you can answer them in a generalizable way. In a related vein, I would often go to my managers or to the managers of the projects I was working on, and I would propose solutions. Many times, they would push back and say things like: "oh well that won't work here because we're not like that other organizations". I realized that if I had very good research and data, that might help solve the pushback I was getting. That's how I started thinking about doing research.

InfoQ: Can you share a little about your research methodology, how do you elaborate your hypothesis and where do find inspiration for those hypotheses? Do you leverage any agile techniques such as sprinting, demos, quick feedback loops, etc.?

Dr. Forsgren: A lot of this is covered in Part Two of the book Accelerate. Once I get into research design, you can't really do a sprint. Early in the research process, I'm heads down in the literature, sketching out hypotheses and then figuring out what's going to fit in this year's research Report and what will be out of scope.

Throughout the year, I'm gathering ideas pretty broadly. To do this, I keep in the back of my mind what we've studied in the last 6 years and what I would like to revalidate, what's core to the model that must be included because it's part of the model.

I do a broad search on what to include for research design from several sources throughout the year:

• I'm looking at trends around the industry. I'm talking to analysts to see what's emerging on their radar. But I have to balance that with what is big enough in the industry. There are some trends that are hitting, but if it's not pervasive enough, I have to balance it with research design considerations. As an example, chaos testing and chaos engineering are big trends right now, but they are not widespread enough for me to include in this state of DevOps report type of research design.
• I have a handful of advisers that I work with every year. Advisors on this year's Report were Kelsey Hightower from Google, Adrian Cockroft, Sam Guckenheimer and Gene Kim.
• I also go to several conferences, I speak with several companies, executives and developers at enterprise level but also at small to medium businesses and startups because I want to see what their scaling challenges are. I also want to see what's working really well for them.
• I also keep up on recent research (both published in journals and in conferences) across related fields to make sure I’m aware of what is happening.

It’s … a lot but I love it and it’s important to ensure we’re capturing the industry.

InfoQ: I know you will be attending the DevOps Enterprise Summit this year. Could you give our readers a sneak peek into your talk?

Dr. Forsgren: Dr. Dustin Smith, second author on this year's report, and I will be presenting some of our favorite findings from the 2019 Report. Then I will be speaking at the Workplace Engagement Panel along with Dr. Christina Maslach (University of California, Berkeley) and Dr. Andre Martin (Google).

About the Interviewee

Dr. Nicole Forsgren is co-founder, CEO and chief scientist at DevOps Research and Assessment. She is best known for her work measuring the technology process and as the lead investigator on the largest DevOps studies to date. She has been a professor, sysadmin, and performance engineer. Forsgren’s work has been published in several peer-reviewed journals. Forsgren earned her PhD in Management Information Systems from the University of Arizona, and is a research affiliate at Clemson University and Florida International University.