Q&A on the Book Engineering the Digital Transformation

The book Engineering the Digital Transformation by Gary Gruver provides a systematic approach for doing continuous improvement in organizations. He explores how we can leverage and modify engineering and manufacturing practices to address the unique characteristics and capabilities of software development.

InfoQ readers can download an excerpt of Engineering the Digital Transformation with a 10% discount voucher included.

InfoQ interviewed Gary Gruver about the history of software development and what’s lacking in the way we develop software, making products and the way that they are developed visible, doing quality improvement for software, creating a culture of continuous improvement, and how to take a more systematic approach to managing software.

InfoQ: Why did you write this book?

Gary Gruver: I wrote this book because I was watching too many clients struggling to make as much progress transforming their software development and delivery processes as fast as I felt they should. They were implementing all the latests Agile and DevOps practices, but they just weren’t seeing the business results I knew were possible. I came to the conclusion that if the industry was going to transform at the rate required by the business, we needed a much more systematic approach to continuous improvement.     

InfoQ: For whom is the book intended?

Gruver: This book is intended for anyone working in an organization that requires software to provide a competitive advantage, and is frustrated that their organization can’t make progress fast enough.  

InfoQ: What are the main approaches that have shaped the history of software development, and what has the impact been?

Gruver: People started managing software development in the same way they managed all other kinds of product developments with the stage gate R&D process, but for software they called it the software development life-cycle.  Overtime, people figured out that this was not working very well because the software technology was changing so fast, that it was much harder to plan as accurately as with traditional product development. Additionally, software is much easier to change than traditional product development, so people started realizing they needed a different approach.  

The first major step to changing how software development was managed was the move from waterfall (software development life-cycle) to Agile. This approach was based on solid principles, but some organizations got so lost in the Agile rituals that they lost track of the fundamentals. The Poppendiecks tried to correct that with a focus on Lean principles that supported Agile instead of just the rituals. These steps all helped and were a step in the right direction, but people were still struggling with the Agile principle of releasing code the customers on a more frequent basis for feedback, which was lost as Agile scaled to the Enterprise. This brought about a focus on DevOps to address those challenges.  

These were all good and significant improvements, but as I worked with clients to understand the business results I achieved at HP, I felt the approaches could be further improved. I felt that if my clients were going to achieve the business results they deserved, they needed a more systematic approach that could engage everyone in the continuous improvement journey.  

InfoQ: In your opinion, what's the main aspect that's lacking in the way that we develop software?

Gruver: Probably the biggest transformation that enables a lot of changes, from my perspective, is shifting to building in quality. Manufacturing made this transition a long time ago, and it fundamentally changed the effectiveness and efficiencies of factories. Most software organizations have not made this shift in approach, so they can’t change the rest of their development and delivery processes. Because traditional manual software testing has been slow and expensive, they batch up large amounts of inventory to inspect quality very late in the process when they reach a milestone, like development complete. This is very inefficient and leads to very complex triage and defect-fixing processes, and limits the ability to receive feedback from customers on a regular basis.

Automated testing is starting to change that, but most people just use automated testing to make their manual testing less expensive. They have not figured out how to use the automated testing to change the capabilities of their organization. This requires running the automated testing on a more frequent basis much closer to the developers, and requiring them to respond to this feedback as they are writing the code. This is a huge cultural shift for most organizations, and is where I see most transformations struggle. They either don’t take this step, or when they do, they start requiring developers to respond to the test feedback before they have a stable quality signal. After seeing lots of organizations struggle with this issue, I came to the conclusion that I needed to warn people in order to avoid this mistake, and create a more systematic approach for ensuring a stable quality signal. That was one of the biggest motivations for writing this new book.     

InfoQ: You mentioned that one of the challenges of doing improvement is that software is not a physical product. How can we make products, and the way that they are developed, visible?

Gruver: We need to understand the architecture and then map out the deployment pipeline so it is visible to everyone. The architecture has a significant impact on the design of the deployment pipeline, and the deployment pipeline has a significant impact on the productivity of the software development teams. We need to make this visible if we are going to expect people to invest in making improvements, and engage the broader organization in continuous improvement.

The book has several examples of different types of architectures and applications, and the implications on the deployment pipelines. As I work with different organizations, I find that the easiest way to get people to document their deployment pipelines is to have them describe all the different environments they use in their development and delivery process. They start with code on a developer workstation, and then when they check the code in, it goes where on the path to production? Does it go to an integrated development environment next? Or QA? Maybe UAT?  Or staging? Eventually landing in production. Those steps define the deployment pipeline, and all the applications that exist in those integrated test environments right before production represent your tightly coupled architecture.  With this, you can create a simple view of the architecture and deployment pipeline.

This is really a form of value stream mapping, but a very specific form I have found most useful to highlight the issues and opportunities for improvement with software development and delivery. I started out by using industry standard approaches to value stream mapping, and found that even with manufacturing companies that had facilitators with decades of experience with value stream mapping, they weren’t highlighting the issues very well. Additionally, while the mapping process was valuable to the people in the room, it typically was so complex that it was hard to share the information beyond the people who created the map. These experiences led me to be very specific about the types of maps we want to create and the data we want to populate on the maps.

InfoQ: How can we do quality improvement for software?

Gruver: The biggest change required is moving from inspecting in quality in large batches, to building it in as the software is developed. This though requires ensuring you have a stable quality signal, which is a big challenge for most large organizations. They need to take the time to ensure they understand and address all the sources of instability before they start trying to build in quality, or else their transformation is going to struggle and lose momentum. There is a very simple exercise I recommend all my clients run in order to ensure a stable quality signal.  First, select an environment along the deployment pipeline for the exercise. Try to pick one far to the right (closest to production), as you have influence in the organization to change. Too far left and it only helps developers working in those environments. Too far right and you don’t have enough influence to implement the changes required for a stable quality signal. Next, take all your automated tests and run them 20 times in a row in that environment on the same version of code. Dump the result in a database and graph the passing rates. Most clients find they have some tests that are always passing, some that are always failing due to a defect, and a bunch that toggle between passing and failing. These flakey tests that are toggling can’t be used as a stable quality signal, and need to be set aside for rework. And the failing tests need to be set aside until the defect is fixed. Now, take all the passing tests and run them again in random order. This will highlight all the additional tests that need to be reworked before they can be used as a stable quality signal. This gets us to a set of stable tests we can use to evaluate the stability of the rest of the system.

Now, we take this set of tests and run them in parallel against the test environment to see if they can handle the load.  Most of the time, we find this uncovers more instabilities that need to be root caused and fixed. Next, we want to ensure we have a stable repeatable deployment process. Use the same environment, but this time we are going to deploy the code and run the test. Then deploy the code again and run the test. Do this 20 times in a row, and dump the results in the database so you can look for deployment issues. If you find any, they need to be fixed before moving forward. Finally, you want to run the same experiment 20 times in a row, starting with creating the environment from scratch.

This process and set of experiments forces you to fix the issues that are causing instability in the quality signal. It is only now that you are ready to start building in quality. Too many organizations skip this step, and find that running the automated tests on a more frequent basis is just too hard because you end up having to constantly triage issues associated with tests, deployments, and environments that have nothing to do with the code quality issues you are trying to find.  

InfoQ: What can be done to create a culture of continuous improvement?

Gruver: From my experience, this works best if it is led by executives, which is why I spend a lot of time running executive workshops. It was also a motivation for writing my second book, Leading the Transformation. There are a lot of resources in the industry to help engineers figure out how to implement new technology, but not many for executives who are working to create a culture of continuous improvement. Leading the Transformation, and this new book Engineering the Digital Transformation, were developed to help fill that gap.

The executives need to understand that they need to set the expectation that it is everyone’s job; not just to do their job, but also to try to figure out how to do the job better. They need to help provide the bandwidth for making improvements, and they need to provide a structured approach for driving improvements. If you can get them to fill this role, the transformation will be much easier.

If not, you can do some things to create the culture from the bottom up, but most of those efforts falter overtime, as people get tired of fighting the system. That is why I try as hard as I can with clients to get the executives to engage in leading the transformation.     

InfoQ: What's your advice for organizations that want to take a more systematic approach to managing software?

Gruver: It needs to start with a common understanding of the approach that is going to be used across the organization. People need a common framework for analyzing the issues if they are going to agree on what needs to be improved. The goal of this book is to provide that common framework that can be used to create a common understanding. It is not intended to be the final answer for the systematic approach for software, but rather a starting point that is designed to address the unique capabilities and characteristics of software that can be fine-tuned and improved by lots of great minds, as happened in manufacturing.  

About the Book Author

Gary Gruver is an experienced executive with a proven track record of transforming software development and delivery processes in large organizations. His current passion is helping as many people as possible achieve similar results through consulting, webinars, presentations, and books.