In 2009 the book Agile Testing by Lisa Crispin and Janet Gregory was published. The book More Agile Testing published in 2014 reflects back on the developments that have happened in agile testing in the last five years. It covers new challenges in testing, test practices, and examples of and experiences with agile testing from teams all around the world.
You can read a chapter from the book more agile testing to get an impression of the book. Also see this for more information and some content from both More Agile Testing and Agile Testing.
InfoQ did an interview about the book More Agile Testing with the authors Janet Gregory and Lisa Crispin.
InfoQ: For those who have read your first book, can you explain how the book more agile testing is related to it? What is this second book covering?
Janet and Lisa: Our editor asked us if we’d like to do a second edition of Agile Testing. Looking at it, we felt that everything in it was still relevant and useful, and not in need of updating. At the same time, we have learned so much since 2008 when we wrote the first book. At the same time, technology has changed and grown so there are many new challenges. That’s a long way of saying, the new book contains many things that are we see agile teams struggling with over the last six years, and expands on topics covered in the first book.
We wrote More Agile Testing so that it can be read independently of Agile Testing. However, we didn’t want to repeat everything we said in the first (quite large) book, so we do refer readers back to it for certain topics.
InfoQ: Can you mention some of the more recent challenges that testing has to deal with?
Lisa: One big aha for me since writing the first book is realizing that delivery teams can and should help customers identify the features that will provide the most value to the business and slice those to the smallest viable chunks. Gojko Adzic’s and David Evan’s work with visualizing quality, impact mapping and other ways to help customers determine what software features have the best ROI has had a big influence on my own work.
Janet: I think that testing as a skill has sometimes gotten lost in teams that drink the kool-aid without considering their specific context. I see large organizations moving to agile and hoping that their outsourced test teams can keep up with development. I see small teams say we don’t need testers anymore without considering the diverse set of testing needs. There are still a lot of questions out there about where testers (or testing) fits into the process.
InfoQ: Many people have contributed to your book by sharing their experiences. What did you do to get them involved when writing the book?
Janet: I think it varied. Some of the contributors we asked specifically for their story because we heard it at a conference or during a conversation and thought it was worth sharing. Others, we knew they had experiences in an area we wanted to talk about so asked if they could write up something for us.
Lisa: Adding on to what Janet said, we were so grateful that almost everyone we asked was happy to share their stories in our book. We’ve met so many awesome practitioners at conferences and via Twitter over the years, so our “network” really came through. One sidebar resulted from a plea on Twitter about a particular type of testing.
Many of our sidebar contributors also reviewed our draft chapters as we wrote the book, and gave us so much valuable feedback. I think that, like us, they want to help other practitioners learn and improve.
InfoQ: You mentioned seeing a shift from roles and titles to competencies. How do you think this impacts the profession of testing?
Lisa: I think it means that everyone on the team gets better at building quality into software products. As testers, when we learn from teammates in other roles, we can add even more value. And as our teammates are more conscious about testing, they find ways to deliver a better product. My current teammates help with exploratory testing, and they work hard to find ways to flush out tricky bugs. We testers have more time for the activities where we add the most value.
Janet: This is one of the challenges I was speaking of earlier. If you have a title “tester”, then organizations can slot you into a spot. When you have the role of developer with specialty in test or programming, then it blurs the lines a little and people become uncomfortable. I personally think it is a good thing because it helps keep testers out of the “only good for testing” box. Testing is a skill that can be applied in many areas including testing ideas and assumptions when we flesh out requirements.
InfoQ: In your book you wrote about "thinking skills". Can you explain what they are and why they are so important?
Lisa: It’s hard to answer this briefly since we devote a whole chapter in the new book to thinking skills! It’s our experience that a whole team approach to testing and building in quality works best. This requires the skills to build relationships, understand the product, and collaborate with people in diverse roles for diverse activities. Problem solving, critical thinking, lateral thinking, time management and other so-called ‘soft’ skills are essential for testing. If we can transfer these skills to other team members, we have an even better chance of success.
Janet: Thinking skills means questioning – and testers do that well. Since testers must play many roles on agile teams, they need to escape the traditional role of tester who executes tests. They need to be leader, test lead, test analyst, test automator, as well as the person who likes to explore.
InfoQ: In your book more agile testing you updated the agile testing quadrants. Can you describe what has been changed and why?
Janet: The Quadrants is a visibility model into the types of tests we want to run. Our new version is still that with a few minor changes. I think what is important to understand is how other people have taken adapted it to their needs. We have included several in the book to show how people have changed it to meet their current needs, or used words that make more sense to them.
Lisa: We have received so much positive feedback about the Quadrants. People tell us they have a poster with them up on the wall in their team area, or a laminated copy they bring to planning meetings. But we’ve also encountered some confusion on the purpose of the Quadrants and how to use them.
We changed some of the wording in an effort to make the purpose of each quadrant more clear. For example the left-hand side used to be called “support the team”, which we changed from “support programming” which was on Brian Marick’s original version of the Agile Testing Matrix. We changed it to “guide development” to make it more clear that these tests help us know what code to write and evaluate whether we are done writing it.
InfoQ: Can you give some examples how agile teams can use exploratory testing approaches?
Janet: I like encouraging people to use different ways of exploratory testing. Most people have a good idea what session-based testing is by reading Jon Bach’s paper on this technique. People are experimenting with thread-based testing which enables a bit more flexibility to explore.
Lisa: On my own team, we testers write exploratory testing charters for new features or major updates using a template from Elisabeth Hendrickson’s Explore It! When a lot of stories have been accepted and the time feels right, two or three programmer pairs will work through the charters. The charters give them a specific area or aspect to test, while not constraining their exploration too much. They ask testers questions as needed. They’re often amazed at how many issues they find, but it helps them as they continue with coding. We also occasionally do time-boxed “group hugs”, where several people or sometimes the entire team will get together in a room and via video conference, sometimes with charters and sometimes just a general “bug hunt”.
I’ve heard of developer teams doing testing dojos together to learn more about testing techniques. I’d like to experiment with my own team trying out other techniques such as session-based testing. I encourage all teams to experiment with different approaches to exploring. It doesn’t have to take a big amount of time.
InfoQ: What can be the benefits that agile teams can get from exploratory testing?
Lisa: Exploring feature ideas even before coding starts, using simple techniques such as paper prototypes and drawing on a whiteboard has helped my own teams. Business experts often don’t know what they really need until they see it, and these activities help refine ideas and hone in on a minimum viable thin slice to start coding and getting further feedback. Having programmers help with exploratory testing helps them expand their thought process to include more aspects of quality. In my experience, that helps them create code that is easier to test and more robust. It also encourages them to have more conversations with customers so we can all learn as we go.
InfoQ: With the growing rate of test automation comes a risk of increased technical debt in test code and frameworks. Do you have suggestions how testers can deal with that risk?
Janet: Lisa talks about the collaboration between testers and programmers when actually specifying the tests. Another advantage is to talk about possible overlaps or gaps in the different layers of automation. I find that this discussion really makes a huge difference in how we design our automated tests.
Lisa: Not automating tests at all is a “shortcut” teams take all too often, and that debt is hard to pay back. But the wrong approaches to automation can also slow the team down terribly over time. I started off in software as a programmer, and I have always enjoyed automating tests. However, as a tester, I usually have at most 20% of my time to devote to writing test code. I’m never going to be as good at designing maintainable code as my programmer teammates. In my experience, close collaboration between testers, who are good at specifying test cases, and programmers, who are good at designing code, is the best way to get a good return on investment of all kinds and levels of test automation.
InfoQ: Scaling agile in enterprises can pose challenges for testing. What do you think are the main issues? How can you scale agile testing?
Lisa: One challenge I’ve seen is that when large organizations transition to agile, they go from having silos by role – QA, programming, BA, for example – to having silos by team. They have cross-functional teams, but each team gets so heads-down in what they are doing that they are all re-inventing the wheel. Establishing a testing community of practice is one way to encourage teams to share how they are overcoming various testing challenges, as well as share technologies and tools. Additional roles to help lead these communities and champion testing activities at the executive level can help.
Continuous integration across the entire organization, working towards continuous delivery if appropriate, is also a necessity. Large companies often have several big systems that need to work together. Often one team gets blocked by a problem introduced by another team.
Delivery teams in large companies may be far removed from business experts and end users, so it’s harder to collaborate with them. Consistent language such as Planguage for specifying feature behavior and quality attributes helps, as do agile business analysis practices for having conversations with customers.
Janet: Scaling is hard – no question. There needs to be communication between teams or the silos that Lisa mentions becomes a real issue. Deliberate conversations about dependencies is one way to counteract this separation.
InfoQ: People are using more and more different devices which are connected via the internet. This asks for different test focus and approaches, like for instance Gerie Owen described in Testing the Internet of Things: The Human Experience. How do you think the Internet of Things (IoT) is impacting testing?
Lisa: I’ve read Gerie’s article, but I have a lot to learn about human experience testing, so I don’t know how it is impacting testing. We’ve used personas for years and they’re valuable in helping us think about how different people will use the product. In recent years I have found techniques such as impact mapping and story mapping essential for helping customers focus on the purpose of the software, the problems they’re trying to solve, the value they need delivered. Since software is embedded in more and more products we all use on a daily basis, I think Gerie’s work to pull these different techniques together makes a lot of sense.
InfoQ: What do you expect that the future will bring for testers? How can test professionals prepare themselves?
Janet: Continuous learning. Going to conferences – seeing what’s new. Paying attention to the world around you, including what your organizations is saying. No one can predict the future but we can be aware and be prepared.
Lisa: I recently read an article that pointed out the most important thing is to learn how to learn and improve. Companies need to nurture a learning culture and let software professionals manage their own workloads, including time to learn. We also need to learn on our own time. We can learn from a lot of different sources – it doesn’t have to only be learning new tools or technology. I know testers who learn from other fields such as philosophy and psychology. We can learn from people in other roles and departments. For example, sales and marketing people have a lot to teach us about our existing and potential customers. We must work at a sustainable pace so that we can maintain our passion for what we do.
About the Authors
Lisa Crispin is the co-author, with Janet Gregory, of More Agile Testing: Learning Journeys for the Whole Team (Addison-Wesley 2014), Agile Testing: A Practical Guide for Testers and Agile Teams (Addison-Wesley, 2009), co-author with Tip House of Extreme Testing (Addison-Wesley, 2002), and a contributor to Experiences of Test Automation by Dorothy Graham and Mark Fewster (Addison-Wesley, 2011) and Beautiful Testing (O’Reilly, 2009). Lisa was honored by her peers by being voted the Most Influential Agile Testing Professional Person at Agile Testing Days 2012. Lisa enjoys working as a tester with an awesome agile team. She shares her experiences via writing, presenting, teaching and participating in agile testing communities around the world. For more about Lisa’s work, visit this, and follow @lisacrispin on Twitter.
Janet Gregory is an agile testing coach and process consultant with DragonFire Inc.. She is the co-author with Lisa Crispin of Agile Testing: A Practical Guide for Testers and Agile Teams (Addison-Wesley, 2009), and More Agile Testing: Learning Journeys for the Whole Team (Addison-Wesley 2014). She is also a contributor to 97 Things Every Programmer Should Know. Janet specializes in showing agile teams how testers can add value in areas beyond critiquing the product; for example, guiding development with business-facing tests. Janet works with teams to transition to agile development, and teaches agile testing courses and tutorials worldwide. She contributes articles to publications such as Better Software, Software Test & Performance Magazine and Agile Journal, and enjoys sharing her experiences at conferences and user group meetings around the world. For more about Janet’s work and her blog, visit this. You can also follow her on twitter @janetgregoryca.