We can learn from how animals search for food to understand testing better, argued James Bach. Over time testers find out where the buggy parts of a product are, but then it might be effective to occasionally wander off and explore other areas. Stop hoping that automation will save you, and learn to hunt for bugs.
James Bach, a consulting software tester, gave the keynote "When Animals Test" at the Fall Online Testing Conference 2017. InfoQ is covering this conference with Q&As, summaries, and articles.
InfoQ interviewed Bach about effective ways to find defects, determining how much planning or how many rules are needed, and what testers can do to sharpen their skills.
InfoQ: What are the similarities between testing and how animals forage for food?
James Bach: Both activities involve searching a large and often complex space for something valuable and rare. In the case of testers, that means finding bugs. In the case of animals, that means finding food. Through the mechanisms of natural selection and variation (in other words, Darwinian evolution) animals have developed strategies to find food without expending much energy. Those strategies might help us understand testing better. For instance, bumble bees explore their neighborhoods looking for nectar. At first, they spend a lot of energy flying mostly at random. After a little while, they learn where the flowers are, which is much like the way testers learn where the buggy parts of a product are. The part I find most interesting is that after the bees optimize their nectar collection route, they still occasionally fly in apparently random directions, which makes it possible for them to find new flowers. They sometimes collect food efficiently, and sometimes wander. Testers face a similar tradeoff problem.
InfoQ: What can we learn from this?
Bach: One thing we can learn is to stop promoting ways of testing that even animals know don’t work. Animals know that searching is an open-ended and probabilistic process. Spiders use adaptive approaches to solving the foraging problem. So perhaps scripting a zillion non-adaptive test cases is not the best way for us to find bugs, either.
Stop hoping that automation will save you. You have to learn to hunt for bugs. A hunt is a dynamic process. Yes, we can use automation to help us test, but we should use automation in a way that enhances our senses and powers, rather than using it in ways that puts us to sleep.
InfoQ: What is the value of planning, and how can we find out we have too much planning or too many rules?
Bach: Since the purpose of planning is to configure our tools and resources and actions to efficiently achieve our goals; and since planning takes valuable time and energy; and since the data and skills required for a better plan come partly from experiencing the results of a worse plan, the answer is this: "we discover we have an unnecessary planning process partly through the exploratory and adaptive process of forgetting or otherwise removing bits of the process and experiencing what happens."
Rules are expensive. But rules do good things for us. Seek a good tradeoff through ongoing experimentation and mutation. That’s what animals do.
InfoQ: If we have too many rules, how can we get rid of some of them, how to decide what to keep and what not?
Bach: This is why lack of discipline, youthful irreverence, and forgetfulness all have adaptive value. These qualities are not necessarily bugs in our processes. As I like to say: if you are going to preach a bad process, then at least don’t practice what you preach.
In other words, you may not need to shed your excess rules because maybe, already, nobody follows them.
InfoQ: What’s your suggestion for testers to sharpen their skills?
Bach: Watch yourself working. Watch others work. Occasionally take very careful notes about how problems get solved and tasks get done.
This is something other animals are not so good at doing.
Here’s an example. I once took a seven minute video of myself testing and analyzed each two-second block of it. I thought I would discover that I was doing a lot of implicit fact checking as part of my testing, and indeed I found that to be the case. What surprised me was that about 25% of my time was spent doing a particular activity that wasn’t exactly performing a test or checking a fact. That activity was sensemaking. Sensemaking is the process of building a theory to explain something that you experience. I hadn’t recognized the prominent role of sensemaking in my work until I spent three hours carefully picking apart seven minutes of testing. Having recognized that I do a lot of sensemaking, I can develop exercises and heuristics to systematically improve at it.