Michael Bolton on the Testing Mindset

Show Notes

• 00:38 Introductions
• 01:10 Michael’s role as a consulting software tester
• 02:12 Defining what testing is all about: evaluating products by learning about them through exploration and experimentation
• 02:24 What’s involved in being able to evaluate products in this way
• 02:56 The question which testing is focused on: are there problems which threaten the on-time, successful completion of the project or that threaten the value of the product?
• 03:14 Exploring why this question is often ignored or suppressed
• 04:02 Not testing is OK if the domain the product is in is not important and has no risks associated with it
• 04:24 Examples of products which can be perceived as low risk can actually have life risk impacts
• 05:21 Part of the problem is that people building the product don’t know how and where the product will be used
• 06:04 Identifying these risks and identifying potential problems is a role for skilled testers, but not restricted to them
• 06:18 The need for critical distance
• 06:36 Describing the limitations of unit testing
• 06:58 The confusion between testing a small bit of functionality and the complexity that arises when many of these small bits of software are combined into systems
• 07:54 There are situations where a software product can be reliable, but unsafe
• 08:10 Quoting examples from the book Engineering in a Safer World where problems occurred as a result of reliable, but unsafe products
• 09:23 Putting simple things together into a complex thing creates complex problems that we’re not good at anticipating *** Good soundbite ***
• 09:40 Taking an experimentation approach to exploring a system is drawing on ideas from as far back as the 1600’s: put the system into states that are not normal or standard and observing the results
• 10:01 The common approach of using unit tests and checking the output of transactions they are not looking for trouble, they are looking to demonstrate that everything is OK, and this is a dangerous perspective
• 10:40 Effective testers are professionally pessimistic – skeptics who are professionally unsure about the state of a product
• 11:11 Critical distance and social distance are linked
• 11:58 People don’t like to receive bad news, and need to help teams realize that bad news now is better than worse news later
• 13:40 Michael’s wide and diverse experiences in many different roles makes him a better tester
• 14:35 The risk with cross-functional teams is that if testing is nobody’s focus there won’t be a desire to become excellent testers 
• 15:05 There is a need to switch the perspective from the builder’s mindset to the tester’s mindset, and this switch is hard 
• 16:20 The types of things that don’t get found in peer reviews and pair programming
• 17:24 Referencing Why Software Sucks – geeks drive stick-shifts
• 18:34 The difficulty in the mindset shift is that it is tough for people to get out of their own sphere of interests
• 19:34 Making the mindset switch is easier to do when you have someone full-time in the testing role
• 20:06 Irrespective of the technology, domain or product type, testing remains testing – critical thinking applied to products and technology
• 21:20 Changes in technology and tools mean that testing has to shift its focus to different types of risk
• 23:08 The need for testing of machine learning so that it reflects what we aspire to be, rather than what we actually are, which requires testers to become ethicists, sociologists and anthropologists
• 24:04 The need for people with a liberal arts education rather than computer science to take on the tester roles in the future
• 25:58 Exploring different types of (artificial) intelligence and how that impacts testing
• 27:34 Discussing what advances in AI mean in terms of how they impact games such as chess and go and the impact of this on players
• 28:56 The value of testers being historians – able to examine the way ideas have changed over history and how that impacts on the testing approach
• 29:20 Testers also need to be artists – “the defence early-warning system for the culture”
• 30:20 The changes in the education requirements for engineers – in most universities today engineers are not required to study ethics and liberal arts topics
• 31:36 The addition of “charisma” as a quality attribute to be tested for a product, is the product aesthetically pleasing and engaging to use
• 32:42 Discussing different approaches to aircraft flight management software or autonomous driving
• 34:28 An example of how the Chinese social credit score impacts people; a photograph on a bus advertisement caused the person in the advertisement to have her score lowed and the impact that has
• 35:54 The wider implications of these types of systems and how they impact peoples’ lives
• 36:58 We’re not letting the machines help us defeat racism and bias, we’re using the machines to entrench the existing problems
• 37:58 Advice for technologists – what we are trying to do is about enhancing, intensifying and accelerating human purposes
• 39:08 We need to pay attention to the changes we are trying to effect with technology, and the side effects those changes are having both good and bad  
• 39:27 We need to examine product risk, project risk, business risk and societal risk
• 39:44 Technology can amplify what we want, and they can amplify what we are like 
• 40:14 The role of testers will be to introduce caution and reflection into the product development processes
• 40:55 Testers need to find the problems that threaten value before those problems can impact our customers

Mentioned:

• Rapid Software Testing