Try to keep units small, use appropriate tools, and pair-up programmers and tester; these are suggestions from Adrian Bolboacă for writing good unit tests. Unit testing is a mixture of programming and testing; programmers can work together with testers to learn from each other and broaden their knowledge horizons.
Adrian Bolboacă, Organizational and Technical Coach and Trainer at Mozaic Works, will talk about different types of automated tests at the European Testing Conference 2017. InfoQ will cover this conference with Q&As, summaries and articles:
[The European Testing Conference] is about getting experts and practitioners together to talk, learn and practice the art of testing. We’re looking into advanced new methods into making our testing more effective, as well as enrich our understanding of fundamental methods to grow a stronger community.
In his blog post Automated Test Purposes, Bolboacă defines what a unit test should do:
A unit test is focused on a method or a class. It should be very small, a few lines of code at the most. There are many ways one can make mistakes when writing unit tests, so it is not a trivial thing. Because they are small they should run in memory and a unit test should run in milliseconds. Any test that touches external dependencies (database, webservice, file system, any I/O) is not a unit test, it is something else (integration test, integrated test, acceptance test, end-to-end test, etc).
InfoQ interviewed Bolboacă about writing good unit tests and using automation in unit testing.
InfoQ: Does it matter who writes unit tests- developers or testers?
Adrian Bolboacă: Unit tests are more technical, they often focus on coding details or even on program language specific concepts. Unit tests look different in a static programming language like Java or in a dynamic programming language like Ruby. That is why the main responsible for unit tests should be a programmer.
On the other hand, testers know a lot better how to make a test plan, to identify interesting values for tests by specific analysis like Equivalence Partitioning or Boundary Value Analysis. So a programmer needs to "steal" this kind of knowledge from testers, or they could pair-up and work together on the tests to write, but then the programmer would implement them.
In my experience pairing is the best option, as both testers and programmers learn more from each other and they broaden their knowledge horizons.
InfoQ: When should you do unit testing?
Bolboacă: The team can write unit tests after the production code was written. We call that test after. But that is often difficult, because the production code needs to be written having testability in mind. If we chose this approach, we need to pass the production code through a process of code review and make sure it is testable. Only after that we can continue with creating the tests in the programmer-tester pair.
There is also the test-first approach, which I recommend to programmers who "stole" a lot of testing knowledge from their peers. Using this approach we start with an analysis of the problem and then write one unit test, the simplest implementation code for it, one unit test, implement it, and so on. When the team reaches this level, I would say that the programmer who is writing the unit tests is half-tester, because doing test-first requires a lot of testing knowledge. In this case the testers would focus on reviewing the unit tests at the end, and writing acceptance tests.
Gil Zilberfeld talks about the benefits that a test first approach can bring in the interview Test First Approaches:
Test First defines what needs to work. It defines what code we need to write to solve specific problems, because we have a definition in a form of a test. It’s fairly simple to know if we have working functionality just by running the tests.
Working this way results in much more coverage, because testing becomes a first-class development activity, rather than getting pushed to the end.
In addition, when writing these tests, and specifying the scenarios, we explore the problem space more, because many questions will come up. In Test-After these discussions sometimes never happen, and the developers code what they think, rather than what the solution needs.
Eli Lopian explained in the article The Day the QA Department Died how unit testing can be a possible QA killer:
Unit testing is a method of testing your specific piece of code to make sure that it is working properly and will fit correctly into the software puzzle. It has been shown that with unit testing you can properly check over 90 percent of your code, and, unlike QA’s manual testing tools, unit tests that are built correctly and tested automatically can evolve with your codebase, testing the code in real-time.
InfoQ: How can you use automation in unit testing?
Bolboacă: Unit testing is a process of analysis that leads to a test plan, and then that test plan can be automated.
In order to automate these tests there are some useful things to have in mind:
- Use domain language for tests names
Often we write the test and that’s all. But code is more often read than written. So be nice to your colleagues and your future self and make the test name clear.
Use names from the domain, not technical names. A test name like "ExceptionOnOverflow" or "TestThree" or "CustomerTest" is not clear. Instead we should write clear names like "WhenTooManyPlayersAreAddedAnErrorIsReturned", "ACustomerNeedsAllFieldsValidated", "ValidCustomerCanBeUsedByOrder", "InvalidCustomerIsRejectedByOrder". In this way all the people who know the business domain will understand what you are testing there. Even your customer.- Have short tests
A unit test is short, clear, with a single purpose. The best test has 3-4 lines of code, and that makes it very clear for anybody who reads it.
We need to have many small unit tests like this, and they will work together like a Product Immune System, where each test is like an immune cell . If a bug appears, one small test should tell you exactly where the problem is. In this way you will have a fast feedback and an easy development and testing cycle.- One assert per test
If you have more than one assert per test you are testing more than one thing. In this case test names become weird and unclear, tests become too long and the feedback in again unclear; you never know which of the asserts passed or failed.
Let’s say you have three asserts one after the other. If the first assert fails, the last two will never be checked. You change some production code, and you are not protected by the last two asserts when changing the code. In this case you just have the illusion of a safety net and regression testing.- No chained tests
I often see the habit of chaining the tests. The reason usually is that the arrange part is very difficult to set up. But this is not a solution. The tests that depend on the other ones in the chain will fail to run most often because of test on the top of the chain. In this way you might change code to make the tests pass, but you might introduce defects that are not validated by the incapacitated tests.
Always tests should be independent of each other, like an immune cell in the Immune System. They all depend on the Product, and not between each other.- Use the appropriate tools
There are many tools there: testing frameworks, mocking frameworks, test runners, performance testing tools, security testing tools, etc. Make sure you use the appropriate tool for the job. Don’t just use the tool you know. Often the xUnit frameworks are very good for doing most types of automation, but use specialized frameworks for performance and security. If you want good tests that can become executable specifications, you can use xUnit as well, but maybe you want to use some BDD frameworks that make your life easier.
Remember that choosing a testing framework is a decision that has to be made on medium and long term. Imagine the costs of learning, and also the costs of usage and maintenance.Because in the current market there is the need to deliver features faster and faster, we need to automate in a smart way the validation process of the product. This is why automation is essential nowadays.
InfoQ: Which tips do you have for writing good unit tests?
Bolboacă: Unit testing comes from industrial production, and most people forget that. In industry we need to test each small part, and if it is quality compliant we can use it in the next step to assembly the parts. In software we don’t have such a clear definition of what a unit is; there are several opinions- method, class, module. The first difficulty is making sure all the team members decide together what a unit is, and write the test accordingly. For me a unit is very small, the size of a screw in an engine or a nail for furniture. So my tip is to define what a unit is, and focus during the tests review phase on enforcing that decision.
Unit testing is a mixture of programming and testing. Programmers need to know many testing concepts in order to write simple, fast, maintainable tests. My recommendation for programmers is to learn from their colleagues testers about some essential concepts: Equivalence Partitioning, Boundary Value Analysis, Test Coverage, Positive Testing, Negative Testing.
Another very often forgotten part about unit testing is analysis. We don’t start writing tests immediately without thinking. We need to analyze the problem, split it if possible and then think about what unit tests we need to write. A good piece of advice here is to always start with the outputs of the system, and then identify the possible inputs that generate that output. Thank you Chris Matts, for teaching me that.
When writing a unit test, we should use words from the domain of the problem. The test should represent the reason why a feature exists in your product. A person that knows the business domain, but does not know programming, should be able to read your test. Pairing with analysts is very useful in that direction.