At the Agile Practitioners 2016 conference Huib Schoots talked about testability. He stated that low testability, anything that makes our software hard to test, slows teams down, and explored how testability can be increased.
Schoots talked about the unknown unknows in software development. We cannot capture everything that a product has to do upfront, so we have to ensure that we build insights during development. We have to learn how to deal with complexity and confusion. A waterfall approach with "command and control" will be in the way when developing insight.
Agile testing is testing in an agile context. The testing doesn’t change when we adopt agile, but the context changes. Some of the differences in agile testing are that by using an iterative approach the lead time for preparing, executing and reporting tests becomes shorter and that change will be a common thing. Also there is a change in roles, Schoots mentioned that as a test manager using agile he became more of a coach and was doing less testing.
Rapid testing is a mind-set and a skill-set of testing said Schoots. With rapid testing there will be less documents and the focus on how to do testing increases. Rapid testing is a general testing methodology, not only for agile but suitable for any kind of project or product.
Testing is about people working together in conditions of uncertainty. We can’t know everything and things will change.
Schoots stated that the purpose of testing is to discover the status of the product and any threats to its value, so that clients can make informed decisions about it. Testers see things for what they are and light the way. They tell the truth to the team and managers.
Checking (opposite to testing) is about operating a product to check specific output. All checks should be automated as soon as possible said Schoots. Checking is boring and automating them improves testability.
Schoots mentioned James Bach’s definition of testability:
The practical testability of product is how easy it is to test by a particular tester and test process, in a given context.
We need testability as it makes testing easier, faster, and cheaper, and leads to less non-reproducible bugs said Schoots.
Schoots told a story about a bank where he worked. To test the systems they couldn’t use production code. So instead they had to create text based files to test transferring money between "test banks". Schoots witnessed how a tester was manually changing a large text file for testing. Since this was taking much time they decided to make a tool to do the changes. The tool was extended to be able to do all the changes that are needs to created test files using production code as input. The tester saved about 3 hours a day using this tool, so this tool increased testability.
Epistemic testability is the gap between what we know and what we need to know. It has to do with prior knowledge of the quality of the product. Schoots explained that the longer you wait with testing, the bigger the gap becomes. An example is knowing what has been tested in unit testing, if you have this knowledge then you don’t need to test that again in system testing.
James Bach stated that the tester must ask for testability. Schoots doesn’t fully agree with this, his opinion is that testers should ask for more testability since it is a team responsibility and the whole team will benefit from high testability.
Testing is faster, easier and cheaper with high testability, everybody benefits said Schoots. He suggested that testability should be a subject for the sprint planning and that teams should do testability retrospectives to find ways to improve it.