Q&A on the Book Leading Quality

The book Leading Quality by Ronald Cummings-John and Owais Peer explores how to become a leader of quality, master strategic quality decisions, and lead engineering/QA teams to accelerate company growth. The book is intended for people who lead quality inside their companies, like C-suite executives.

InfoQ readers can download chapter 3, Leading a Culture of Quality.  

InfoQ interviewed Ronald Cummings-John about the effects of poor quality, quality narratives, how pairing helps to improve quality, test automation strategies, continuous testing, testing in production, and setting a vision.

InfoQ: Why did you write this book?

Ronald Cummings-John: In our day-to-day, we run Global App Testing. At GAT we end up spending a lot of time seeing what life is like in some of the biggest companies in the world. We were constantly being asked “What should my QA strategy be?” and “How should I think about quality?” So we thought to ourselves, it won’t take long to write a book all about quality. And so, we interviewed VPs of engineering, CTOs, heads of quality etc. to understand how they look at quality inside their companies. We ended up discovering that the real question was less about “what my QA strategy should be”, and more about “how I become a leader of quality”. And it makes sense why that is more of a gap, because if you look at the content around testing and quality it is normally about the tester. But, there are people in leadership positions who have a quality-related background. So addressing how to become a leader of quality, for these people, became the ultimate question the book was answering.

InfoQ: For whom is the book intended?

Cummings-John: We wrote the book for people who lead quality inside their companies, and the title of that person can vary widely. In some companies, it could just be the CEO. In others, it’s VP of engineering, CTO or product owners. Overall, if you’re a C-suite executive or an individual contributor wanting to impact quality in your company, there is something for you in the book. 

InfoQ: What are the effects of poor quality?

Cummings-John: I spoke recently with Software Testing News about the real cost of software bugs, but the real cost of poor quality is a lot higher than you might expect. Research by CISQ found that in 2018, poor quality software cost organizations $2.8 trillion in the US. That’s a huge financial impact caused by poor quality alone, but quality can also affect what we like to call the 3C’s - your customers, company and career. We discuss this in Chapter 1 of Leading Quality, but an example would be when a bug in the American Airlines holiday scheduling system meant that all pilots were allowed to schedule time off during the Christmas holidays; this led to over 15,000 flights without a pilot assigned to them. This caused huge delays for customers trying to get home during this period. The customer was impacted because they were unable to travel home during the holidays. The company faced financial loss through having to pay pilots extra for last-minute shifts, and careers were impacted as a result of the disaster - no one wants to be in charge of a project that goes wrong publicly. The effects of poor quality are wide-reaching and damaging. 

InfoQ: What is a "quality narrative" and how can it help to improve the quality culture?

Cummings-John: In the book, we discuss the concept of a quality narrative. The idea is that in every single company right now, there is a narrative in terms of the way people think and talk about quality. There are three core types of narratives that exist. The first is the Ownership Narrative. The Ownership Narrative is really about who owns quality inside the company. Ultimately, if quality is only owned by the testing teams, then you are in for a problem. That’s because really, everybody in the company affects quality in some way, and there has to be ownership across different teams. The next narrative is the “How To Test Narrative’’ which is probably what we talk about most in the industry because everyone believes they know how something should be tested. But it is important to remember that there is no silver bullet when it comes to looking at quality. In actual fact, you need a blend of different testing types and approaches in order to get the quality level you require. The third narrative is my favourite, and it's the Value Narrative. This is all about the value that focusing on quality brings to the company. When we talk about the value of quality, it's often focused around risk mitigation. But there are also other things we can look at such as savings that are made when we invest in certain types of testing. For example, if we had implemented a certain infrastructure there is the monetary value of the team's time that is saved by that infrastructure. 

InfoQ: How can pairing help to improve quality? 

Cummings-John: Pairing is super interesting because it all comes down to building empathy between people. Companies like Atlassian have their quality teams paired with their developers so that the developers can get a deeper understanding of what it is to have that core focus on quality. It doesn’t stop there. You can also have your designers pairing with your developers, so they have a deeper understanding of the way things should look. The more interactions you have between different disciplines, the better quality product you end up having because there is a more holistic understanding.

InfoQ: What's your advice for adapting the test automation strategy when a product is maturing?

Cummings-John: The levels of automation that one implements will adjust over time. In the very early stages of a product when you are simply trying to find product-market fit, you may not have a fully automated suite. Yes, you may include things like unit testing, but you might not have high levels of automation. But, as you mature past product-market fit, your product actually reaches the Predictability Stage. Now, the Predictability Stage is when you need to ensure that you are working on a solid infrastructure. This is where automation really starts to kick in, and the level of automation usually increases at this stage. As you pass that and move to the Scaling Stage, this is when you start to see a little more levelling out between how much automation comes into play and how much exploratory testing comes into play. 

InfoQ: How would you define continuous testing and what are the benefits that it can bring?

Cummings-John: The original view of continuous testing was related to the process of executing automated tests as part of the software delivery pipeline, but, I would define continuous testing as the act of testing an application at every stage of its lifecycle. This means incorporating testing even before the moment a line of code is written and continuing to test beyond release. 

If you take this view of continuous testing, then there are some interesting benefits. Firstly, you start to pre-empt problems before they happen, and get more involved with quality early on. The skills you use to test the application are being used to test the idea, or to test the designs or to question what it is you are building. Sometimes, those earlier questions can save you a lot of time and hassle later on down the line. The other thing you start to do with TDD etc. is that you start to ensure that you have built testability into your product from the very beginning. 

InfoQ: What challenges come with testing in production? How should we deal with those challenges?

Cummings-John: Cindy Sridharan has a great article titled ‘Testing in Production, the safe way’. In it, she talks about the fact that testing in production isn’t for everyone and it requires having an advanced infrastructure to make it work. To get the best out of testing in production you need to have a good level of automation already in play, as well as a system that has been designed from the ground up that lends itself to this type of testing.

InfoQ: You mention the importance of “Setting a Vision” in your book; what do you mean by this and how can someone put it into practice?

Cummings-John: If we are taking things from the perspective of being a quality leader, one of the most fundamental things about leadership is having a good understanding of your vision for yourself and where you want to go. So, in the book, we talk about understanding and defining that vision for yourself. 

The second vision we tell you to look at is the vision for your company. You can then make an assessment if the company's direction or vision is aligned to what you want to do. 

The reason why this is important is because you can’t lead a team if you aren’t inspired by the thing you want to do yourself. So once you are clear on your personal and company vision, then you can begin to look at developing the vision for your department. 

That vision for your department is really clarifying how you would like it to look in 6 or 12 months’ time. Then you share that with your team and get everyone aligned. 

About the Book Authors

Ronald Cummings-John and Owais Peer are the co-founders of Global App Testing, a crowdtesting solution that enables tech teams to test in over 105 countries with 40,000+ professional testers using real devices and environments.

Cummings-John and Peer have gained worldwide recognition for innovations in the testing field―most notably inventing Testathons®.