At the Agile Testing Days 2014 in Potsdam Jan Jaap Cannegieter spoke about using different sides of our brain to optimize testing and he will redo this presentation during the Agile Testing Day Netherlands 2015. The presentation is mainly about creativity to enhance testing skills. Is creativity important for testers? And did this change due to agile?
InfoQ interviewed Cannegieter about how agile has changed testing, creativity and thinking in testing, skills of agile testers, and how testers can make steps towards agile testing.
InfoQ: Do you think agile changed a lot in testing?
Cannegieter: Definitely, it is one of the biggest changes in system development in general and also in testing in the last 20 years. If you look at it from a theoretical point of view there are no testers in agile, in agile you only have the product owner, the scrum master and team members. But because we need testing skills in most teams there are usually testers in the teams.
InfoQ: Ok, that is true. But still, isn’t testing just checking whether the system does what it is supposed to do?
Cannegieter: Yes and no. Testers, and with the testers the whole team, are responsible to check whether the system does what it is supposed to do. But at the same time end users, the product owner and team members expect more from testing. Does this system, or part of the system, solve the problem of the end user? Is the sprint goal met? Is the solution optimal? Just comparing the solution with the user stories is not enough anymore, in Agile we need thinking testers.
InfoQ: What’s a "thinking tester"?
Cannegieter: A tester that looks beyond his or her assignment, that looks beyond written requirements. Written requirements, or other forms of documentation, are only something to transfer knowledge. When a system is in line with the written requirements it doesn’t mean that it is a good system or meets the goals of the product owner and other users. So testers need to look beyond these written requirements and keep asking questions.
I was testing our new knowledge base a couple of days ago and I saw something that was not logical. The developer said that it was in line with the requirements. That may be true but it was just not right. We contacted the relevant stakeholder, who was owner of the requirement, and he agreed that the requirement was wrong.
InfoQ: What does this mean for testing process
Cannegieter: In most cases it means that testers don’t make detailed scripts upfront. The emphasis is more on actual testing. This is of course exactly in line with the agile manifesto statement "Working software over comprehensive documentation". But still it is a big shift for most testers because they were used to the guidance of scripts during test execution. Now they have much more freedom, have to decide on the spot what is best to do next. So testing is much more flexible.
InfoQ: What about test automation? Doesn’t that need detailed test scripts?
Cannegieter: Yes, absolutely. But automated tests are always conformance checks by nature. The first time you test something you can’t automate it, at least not by means of record-and-playback. This conformance checking is already automated highly but the thinking part of testing can’t be automated. The most important tools for the thinking tester are his or her eyes, to see what is actually happening, and his or her brain, to interpret what is happening and decide whether that is good or not.
InfoQ: And what does these changes in testing mean for skills of the individual tester?
Cannegieter: Having less guidance by the test scripts, testers need to be more creative. And this is difficult for some testers. More than ones I experienced that testers wanted to keep making test scripts based on comprehensive documentation. But developers kept saying: working software over comprehensive documentation. In one of these projects I introduced test sessions, in which testers, developers, end users and other stakeholders took part. The testing was based on test goals and on the fly the attendees had to design their tests. It took some time to get used to having no guidance by previously made test scripts but after a while most testers like it more. It means more freedom and testers have go get used to it.
InfoQ: How do testers make this step?
Cannegieter: First of all, for newbie’s this is no problem, creativity and flexibility is no problem for them. Some old school testers will have to enhance their flexibility and creativity. For the test process it means testers need to master different ways of testing and documenting tests. Some systems or parts of systems will still be tested my means of detailed test scripts, for instance complex calculations. Other systems will be tested by means of session based testing, whereby the testing is guided by test goals, statements of what the user want to do with the system. And there are systems that will be tested using freestyle exploratory testing in which testers have no guidance upfront what so ever. So testers will have to master all these ways.
Another point is that testers will have to enhance their creativity skills. That is the subject of my Agile Testing Days Netherlands presentation is about. I will share 12 ways to become more creative. One example is what we call perspective based testing in which each tester, or user, tests with a different perspective in mind. Examples of perspectives are: does the system support the process, what are the good things about the system, is it possible the users would refuse the use the system et cetera. This way of acceptance testing enhances the creativity of the testers very much. And this is only one of the twelve tips I give the attendees to become more creative.
InfoQ: Is the old school tester dying?
Cannegieter: No, we will still need their skills and surely in some projects we will make test scripts upfront. But not always and often not as detailed as we used to. And I see a some testers with a lot of experience specializing in performance testing, security testing and test tooling. We will need these testers very badly in the coming years!
InfoQ: What did the change to agile testing mean for you?
Cannegieter: LIBERATION! A couple of years ago I got a bit bored as a tester. Most of the times I was reading documents or writing test cases. The more action focused test execution was only a part of the work and I liked that part more.
At first I shifted my career towards requirements, great discipline as well! But last year I was asked to test a platform for virtual desktop infrastructure; VDI’s. We used session based testing and freestyle exploratory testing and it was fun with a capital F. I used my instinct as a tester, had to react instantly on unexpected things and we worked with a very committed team towards a common goal. I felt I was freed from the limitations set by the production of detailed test scripts upfront and it gave me much more energy.
