Applying Stoicism in Testing

Agility is an interesting word. Quite a lot of people relate to it as being “flexible,” as if you will adjust to every new situation that you encounter because you need to have a continuous focus on delivering software (and therefore value) to your customers. Delivering value can be threatened by “all” that can go wrong, better known as risks. These risks can be investigated by testing for example. But in the rush to becoming agile and shortening the time-to-market, testers can be seen as a hindrance.  For this reason, test automation (as in mostly regression testing)  is very popular, from the managing and non-testers point of view, because testing will be transferred to a controllable process that can be executed within a set time frame. 

In this article, I want to explain that agility is about much more than being flexible, that it stands for being aware of your environment, and the specific challenges that you will face in trying to be in control of quality. This means that there is a specific set of values for a tester that you should stick to. These values will set limits to what you can deliver as a tester, and within those limits, you can keep your agility. But also you can cause a “collision” with agile people around you, because the set of values for a tester stand on their own and don’t have to be perfectly in line with how people apply the agile principles.

The limitations of testing perspectives

Testing has its limitations, as stated in some well-known testing principles. And as a tester, you have to deal with these limitations. Look for example at the testing principles such as,  “Testing shows the presence of defects, not their absence,” and as a result, “Exhaustive testing is impossible” . As a tester, you are dealing with open questions; you never know if you have found “all'' the defects, so how to know when you are done? 

To help testers perform better, I added falsifiability to the well-known principles of verification and validation. Falsifiability comes from the philosopher Karl Popper whose ideas of course attract me as a qualisopher. He says that you not only have to look for proof that a hypothesis is true (‘the software works’’), but you have to look for proof that the hypothesis can be falsified, or in our language, we need to look for proof that (and when) the software isn’t working. You will go beyond the confirmation bias (which is easy to fulfill, automated tests will help you with this). 

But looking for proof that the software isn’t working is an ongoing assignment. I once worked with a tester who understood what I meant and said to me, “I’ve tried everything that I could but I haven’t found any evidence that the software doesn’t work“. Falsifiability works on the base of a hypothesis, or translated to testing, risks. As testers, we have to explore the software and we need to look for evidence of what can go wrong in the software, and we use the risk as a hypothesis. 

Risks mean that a minor adjustment in the software can have an immense impact: e.g. a software change in an x-ray machine can lead to the death of a patient in the machine. So do we allow a tester in a team to take as much time as they need to test the x-ray while the developer is done in five minutes? This can create a lot of stress; should the deadline of a sprint be adhered to, or do they allow you to spend more time testing? Or if you have found an issue, do they grant you more time to explore the issue (because mostly there are more issues in that area)? 

Collisions in your Scrum team

As stated above, testing is an activity that never ends: it has no hard-defined ending. This puts quite a bit of time pressure on testing in an Agile/Scrum context. Isn’t agile focused on delivering value as quickly as possible? But as stated before, the counterbalance of value are risks, and if we don’t balance those two in the right way, as a team we will probably find ourselves “in trouble”.

Looking at it from the point of view that we as people tend to look for certainty (especially in these times when we tend to try to do things more quickly in our daily lives, it seems as if we are also looking for an increase in what we want to control, but that is a different philosophical discussion), it puts additional pressure on testing. The fact that we can never prove that the software really works, means that you can still have a lot of uncertainty during and after testing. The team wants to have results (“Is the testing done? Are you finished with your test cases?”), while the tester in the team from a testing point of view can have doubts, He doesn’t feel confident that the software is good enough, according to his standards of good testing. His standards can collide with the testing standards that the non-testers have and apply.

Besides that, stakeholders can put pressure on the team and on you as a tester. For example, at the end of a sprint, a team can commit to the product owner that they will meet the spring goal. Team members can raise the question, is it a problem that there are still tests (and risks) open? Or, you may encounter the case where a stakeholder’s pressure pushes the team to deliver more and more software per sprint, because the time to market is a higher priority than the quality of the product. What will that do with the quality of testing in the team? 

How Stoicism can help you as a tester

All these kinds of situations can be quite stressful because as a tester you want to keep up your own quality standards. To deal with this, you should also be aware of how philosophy can help you in these situations. To be more specific, Stoicism can help because it stands for “Only worry about those things that you have under your control”.  And if you, from a testing point of view, want to have control of the situation, what are the assets that you have in your hands? 

Stoicism makes use of the four virtues, which are derived from the teachings of Plato. The virtues can help you in performing without carrying the burden of unrealistic expectations, where too much flexibility becomes a burden. The four virtues in Stoicism are: 

• Courage 
• Justice, 
• Wisdom 
• Moderation 

To start with courage, in terms of testing this means: do you have the confidence that you have done the right things? This also means that you must be critical of how you perform your tests. And in testing there can be shallow testing: “Run some tests proving the software works (confirmation bias) and you are done!” So, how critical are you about yourself? Do you put too much trust in automated tests, for example? Automated tests only cover some known functionality (at least mostly), and even test-driven development (in the broad sense of this principle) is more about looking for evidence that it works. 

I stated before that testing is also about falsifiability; explore the software and look for proof that the software -in specific situations- doesn’t work and won’t meet the expectations.  And let's be clear about this: you need confirmation that it works, for sure, but you also need to have the courage to say, “I want to look further than what we know and I want to look for the unknown unknown.” This requires the courage to stand up to your team or your stakeholders. 

To consider and look for the unknown information about a system, you need to have justice. In Stoicism, this stands for “showing kindness and generosity in our relationships with others”. And because you don’t know everything, you need other people to help you out. Gathering information is also about creativity, so you have to gather inspiration from past experience, and with your colleagues must be able to connect dots that weren’t connected before. Once I even stated, “The knowledge (and information) you gather as a tester about the software can be an interesting input for new software products and innovations”. But as a Stoic, stay humble ;-). 

After gathering all the information you need, you should use your wisdom (“based on reasoning and judgment”) to come to conclusions so that you can answer the question “Is this software ready to be used?” Although our customers are the best testers, we as testers are in the position that we are (or at least should be) able to answer the question at every step in the software development: if the software is going to production, what can happen? 

The information you put on the table for your stakeholder should be based on facts. The information shouldn’t be about how many test cases you’ve performed, but objective information on the value and the risks that can still be in the system. Stakeholders mostly don’t want to have the confirmation that the software works, for the simple reason that they already presume that you would never start a discussion about going to production if the software didn't work. You can help your stakeholders to determine these risks in the specific situation they can occur in, and also tell them how to deal with these risks, e.g. extra monitoring, rollback, roll out to a limited group of customers, run some extra tests, etc. 

Finally, if you have done your tests and drawn conclusions based on information, then from that point on it is outside your control and the stakeholders have to make up their minds. Here you need moderation or self-control (also a virtue in Stoicism). Stakeholders can make choices you don’t understand or which can frustrate you. If you have bad thoughts about these people, then you need to practice self-control. Aren't we often misled by the judgment that we form about other people? We should keep our focus on the facts and content, also since we are not good at judging other people (read the book “Talking to Strangers: What We Should Know About the People We Don't Know” by Malcolm Gladwell). 

How to apply Stoicism in your daily work

I will give you some examples of how I incorporate these four virtues into my daily work.

As a tester, I am mostly in a position of not being the one to have created the software. This is a good thing because I have fresh eyes to look at the software. But this also means that I am not in full control of the situation. The developers never tell me where the issues lie ;-). The metaphor I use for this situation of uncertainty is that “I am looking for a black cat in a dark room and I don’t know if there even is a cat in the room." I don’t know where the cat is, so I need Stoicism to guide me through this uncertain situation. 

If we look at the justice virtue: I am aware that I need other people to help me, to define the risks we should focus on, or for analysis when I have encountered something I didn’t expect. In these situations, I start asking a lot of questions: first of all, because I am aware that I could have made a mistake, but also so as not to offend a developer since “his baby isn’t perfect”. Don’t offend anyone; be nice and work together. 

I have also used moderation (or self-control) quite a lot in my career. I once had a pretty emotional discussion with a stakeholder who was of the opinion that I was the main person responsible (and accountable) for the decision on whether the software was good enough to go to production. I gave thorough recommendations and I also identified some risks along with appropriate measurements to take if the risks occurred. And you know what happened? In production, a specific risk that I had mentioned popped up. The measurement was taken and we were back in control. The stakeholder was mad at me because I had given positive recommendations, when there were also major risks. It took me quite a bit of self-control to explain to her that I wasn’t the person who was accountable for that decision, and that in fact, she was! An interesting and respectful discussion led to a positive redefinition of our positions, and following this incident we were then aware of what to expect from each other.

If you are working closely together to get the software to production, it can be hard to stay unbiased and have an independent judgement on the results of testing. The collective mind of my team and the virtue of wisdom can help you here. For every piece of advice I wrote about, I asked for feedback from my team, and my team members also helped me to keep the emotion out and the facts in mind. I needed their wisdom in discussions and that helped me to improve the quality of testing. It is also important that you are motivated to keep that independent opinion on the quality of the software. Marcus Aurelius had a great quote that helps me to keep an independent view on things and not just follow the general opinion: “The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane”.

Another example of how moderation has helped me was in a situation where I lost an assignment because the project manager lost trust in me. This was quite an emotional experience and I wondered, what can I learn from this experience? First of all, I was aware that there was a judgment about things that happened, in which I didn’t have full control (as Stoics know), and I built up the courage to ask other testers what I could do differently next time. I explained the case, and wrote down all kinds of questions I could use next time to be more aware of what is happening around me and to avoid finding myself in the same situation. 

These are just some examples where the virtues of Stoicism have helped to put situations into perspective, and which have helped me a lot!

Conclusion

The four principles (or virtues) of Stoicism are incredibly useful for a tester in an agile organization. Agile organizations (hopefully) became agile because we need to provide value in less time; we need to be able to rapidly anticipate developments that are happening around us. Speed is a leading principle of these times. This puts more and more pressure on testing. Of course, there are a lot of new developments that help testers to deliver the information faster. 

On the other hand, you should stick to your own quality standards. You provide value by delivering information and for that reason, the core principle of testing hasn’t changed (and will never change!); we have to look for confirmation that software works as well as gather information that can be a threat (or create new possibilities) while using the developed software (also known as risks). 

After gathering this information, a judgment needs to be made by humans, e.g. testers. Be aware that you will never be put in the position of making the decision on whether software is good enough to go to production. This is a decision made by your product owner because he or she has an overview of all the factors that need to be taken into account, e.g. the technical as well as the business points of view. 

In this fast-changing agile world, testers can use the virtues of Stoicism to stand firm when under pressure to make a call on the quality of a product; testers can practice the agility of testing using the four Stoic virtues: moderation, courage, wisdom, and justice.

About the Author

Ard Kramer is a software tester from the Netherlands who works at OrangeCrest. He calls himself a qualisopher, which stands for someone “who loves truth and wisdom and at the same time is decisive to improve man and his environment”.