Writing assertions for tests seems simple: all we need do is compare results with expectations. This is usually done using the assertion methods – e.g. assertTrue() or assertEquals() – provided by testing frameworks. However, in the case of more complicated test scenarios, it can be rather awkward to verify the outcome of a test using such basic assertions.
The main issue is that by using them we obscure our tests with low-level details. This is undesirable. In my view we should rather strive for our tests to speak in the language of business.
In this article I will show how we could use so-called "matcher libraries" and implement our own custom assertions to make our tests more readable and maintainable.
For the purposes of demonstration we will consider the following task: let us imagine that we need to develop a class for the reporting module of our application that, when given two dates ("begin" and "end"), provides all one-hour intervals between those dates. The intervals are then used to fetch the required data from the database and present it to the end user in the form of beautiful charts.
Standard Approach
Let us begin with a "standard" way of writing assertions. We're using JUnit for this example, though we could equally use, say, TestNG. We will use assertion methods like assertTrue(), assertNotNull() or assertSame().
Below, one of several tests belonging to the HourRangeTest class is presented. It is quite simple. First it asks the getRanges() method to return all one-hour ranges between two dates on the same day. Then it verifies whether the returned ranges are exactly as they should be.
private final static SimpleDateFormat SDF
= new SimpleDateFormat("yyyy-MM-dd HH:mm");
@Test
public void shouldReturnHourlyRanges() throws ParseException {
// given
Date dateFrom = SDF.parse("2012-07-23 12:00");
Date dateTo = SDF.parse("2012-07-23 15:00");
// when
final List<Range> ranges = HourlyRange.getRanges(dateFrom, dateTo);
// then
assertEquals(3, ranges.size());
assertEquals(SDF.parse("2012-07-23 12:00").getTime(), ranges.get(0).getStart());
assertEquals(SDF.parse("2012-07-23 13:00").getTime(), ranges.get(0).getEnd());
assertEquals(SDF.parse("2012-07-23 13:00").getTime(), ranges.get(1).getStart());
assertEquals(SDF.parse("2012-07-23 14:00").getTime(), ranges.get(1).getEnd());
assertEquals(SDF.parse("2012-07-23 14:00").getTime(), ranges.get(2).getStart());
assertEquals(SDF.parse("2012-07-23 15:00").getTime(), ranges.get(2).getEnd());
}
This is definitely a valid test; however, it has a serious drawback. There are a lot of repeated fragments in the //then part. Obviously they were created using copy & paste, which – so experience has taught me – inevitably leads to errors. Moreover, if we were to write more tests like this (and we surely should write more tests to verify the HourlyRange class!), the same asserting statements would be repeated over and over again in each of them.
The readability of the current test is weakened by the excessive number of assertions, but also by the complicated nature of each assertion. There is a lot of low-level noise, which does not help to grasp the core scenario of the tests. As we all know, code is read much more often than it is written (I think this also holds for test code), so readability is something we should definitely seek to improve.
Before we rewrite the test, I also want to highlight another weakness, this time related to the error message we get when something goes wrong. For example, if one of the ranges returned by the getRanges() method were to have a different time than expected, all we would learn would be the following:
org.junit.ComparisonFailure: Expected :1343044800000 Actual :1343041200000
This message is not very clear and could definitely be improved.
Private Methods
So what, exactly, could we do about this? Well, the most obvious thing would be to extract the assertion into a private method:
private void assertThatRangeExists(List<Range> ranges, int rangeNb,
String start, String stop) throws ParseException {
assertEquals(ranges.get(rangeNb).getStart(), SDF.parse(start).getTime());
assertEquals(ranges.get(rangeNb).getEnd(), SDF.parse(stop).getTime());
}
@Test
public void shouldReturnHourlyRanges() throws ParseException {
// given
Date dateFrom = SDF.parse("2012-07-23 12:00");
Date dateTo = SDF.parse("2012-07-23 15:00");
// when
final List<Range> ranges = HourlyRange.getRanges(dateFrom, dateTo);
// then
assertEquals(ranges.size(), 3);
assertThatRangeExists(ranges, 0, "2012-07-23 12:00", "2012-07-23 13:00");
assertThatRangeExists(ranges, 1, "2012-07-23 13:00", "2012-07-23 14:00");
assertThatRangeExists(ranges, 2, "2012-07-23 14:00", "2012-07-23 15:00");
}
Is it better now? I would say so. The amount of repetitive code has been reduced and the readability has been improved. This is definitely good.
Another advantage of this approach is that we are now in a much better position to improve the error message that gets printed in the event of failed verification. The asserting code is extracted to one method, so we could enhance our assertions with more readable error messages with ease.
The reuse of such assertion methods could be facilitated by putting them into some base class, which our test classes would need to extend.
Still, I think we might do even better than this: using private methods has some drawbacks, which become more evident as the test code grows and these private methods then come to be used within many test methods:
- it is hard to come up with names of assertion methods that clearly state what they verify,
- as the requirements grow, such methods tend to receive additional parameters required for more sophisticated checks (already the assertThatRangeExists() takes 4 parameters, which is too much!),
- sometimes it happens that in order to be reused across many tests, some complicating logic gets introduced into such methods (usually in the form of boolean flags which make them verify - or ignore - some special cases).
All of this means that in the long run we will encounter some issues with the readability and maintainability of tests written with the help of private assertion methods. Let us look for another solution which would be free of these drawbacks.
Matcher Libraries
Before we move on, let us learn about some new tools. As mentioned before, the assertions provided by JUnit or TestNG are not flexible enough. In the Java world there are at least two open-source libraries which fulfil our requirements: AssertJ (a fork of the FEST Fluent Assertions project) and Hamcrest. I prefer the first one, but it is a matter of taste. Both look very powerful, and both allow one to achieve similar effects. The main reason I prefer AssertJ over Hamcrest is that AssertJ's API - based on fluent interfaces - is perfectly supported by IDEs.
Integration of AssertJ with JUnit or TestNG is straightforward. All you have to do is add the required imports, stop using the default assertions provided by your testing framework, and start using those provided by AssertJ.
AssertJ provides many useful assertions out-of-the-box. They all share the same "pattern": they begin with the assertThat() method, which is a static method of the Assertions class. This method takes the tested object as an argument, and "sets the stage" for further verification. Afterwards come the real assertion methods, each of them verifying various properties of the tested object. Let us take a look at a few examples:
assertThat(myDouble).isLessThanOrEqualTo(2.0d);
assertThat(myListOfStrings).contains("a");
assertThat("some text")
.isNotEmpty()
.startsWith("some")
.hasLength(9);
As can be seen here, AssertJ provides a much richer set of assertions than JUnit or TestNG. What is more, you can chain them together – as the last assertThat("some text") example shows. One very convenient thing is that your IDE will figure out the possible methods based on the type of object being tested, and will tip you off, suggesting only those which fit. So, for example, in the case of a double variable, after you have typed assertThat(myDouble). and have pressed CTRL + SPACE (or whatever shortcut your IDE provides), you will be presented with a list of methods like isEqualTo(expectedDouble), isNegative() or isGreaterThan(otherDouble) - all making sense for double value verification. Which is actually pretty cool.
Custom Assertions
Having a more powerful set of assertions provided by AssertJ or Hamcrest is nice, but this is not really what we wanted in the case of our HourRange class. Another feature of matcher libraries is that they allow you to write your own assertions. These custom assertions will behave exactly as the default assertions of AssertJ do – i.e. you will be able to chain them together. And this is exactly what we will do next to improve our test.
We will see a sample implementation of a custom assertion in a minute, but for now let's take a look at the final effect we are going to achieve. This time we will use the assertThat() method of (our own) RangeAssert class.
@Test
public void shouldReturnHourlyRanges() throws ParseException {
// given
Date dateFrom = SDF.parse("2012-07-23 12:00");
Date dateTo = SDF.parse("2012-07-23 15:00");
// when
List<Range> ranges = HourlyRange.getRanges(dateFrom, dateTo);
// then
RangeAssert.assertThat(ranges)
.hasSize(3)
.isSortedAscending()
.hasRange("2012-07-23 12:00", "2012-07-23 13:00")
.hasRange("2012-07-23 13:00", "2012-07-23 14:00")
.hasRange("2012-07-23 14:00", "2012-07-23 15:00");
}
Some of the advantages of custom assertions can be seen even in such a tiny example as the one above. The first thing to notice about this test is that the //then part has definitely become smaller. It is also quite readable now.
Other advantages will manifest themselves when applied to a larger codebase. Were we to continue using our custom assertion, we would notice that:
- It is very easy to reuse them. We are not forced to use all assertions, but we can select only those which are important for a specific test case.
- The DSL belongs to us, which means that for specific test scenarios we could change it according to our liking (e.g. pass Date objects instead of Strings) with ease. What is more important is that such a change would not affect any other tests.
- High readability - there is no problem with finding the right name for a verification method, because the assertion consists of many small assertions, each of them focused on just one very small aspect of the verification.
Compared to private assertion methods, the only disadvantage of the custom assertion is that you have to put more work in to create them. Let us have a look at the code of our custom assertion to judge whether it really is such a difficult task.
To create a custom assertion we should extend the AbstractAssert class of AssertJ or one of its many subclasses. As shown below, our RangeAssert extends the ListAssert class of AssertJ. This makes sense, because we want our custom assertion to verify the content of a list of ranges (List<Range>).
Each custom assertion written with AssertJ contains code which is responsible for the creation of an assertion object and the injection of the tested object, so further methods can operate on it. As the listing shows, both the constructor and the static assertThat() method take List<Range> as a parameter.
public class RangeAssert extends ListAssert<Range> {
protected RangeAssert(List<Range> ranges) {
super(ranges);
}
public static RangeAssert assertThat(List<Range> ranges) {
return new RangeAssert(ranges);
}
Now let us see the rest of the RangeAssert class. The hasRange() and isSortedAscending() methods (shown in the next listing) are typical examples of custom assertion methods. They share the following properties:
- Both start with a call to the isNotNull() which verifies whether the tested object is not null. This guarantees that the verification won't fail with the NullPointerException message (this step is not necessary but recommended).
- They return "this" (which is an object of the custom assertion class – the RangeAssert class, in our case). This allows for methods to be chained together.
- The verification is performed using assertions provided by the AssertJ Assertions class (part of the AssertJ framework).
- Both methods use an "actual" object (provided by the ListAssert superclass), which keeps a list of Ranges (List<Range>) being verified.
private final static SimpleDateFormat SDF
= new SimpleDateFormat("yyyy-MM-dd HH:mm");
public RangeAssert isSortedAscending() {
isNotNull();
long start = 0;
for (int i = 0; i < actual.size(); i++) {
Assertions.assertThat(start)
.isLessThan(actual.get(i).getStart());
start = actual.get(i).getStart();
}
return this;
}
public RangeAssert hasRange(String from, String to) throws ParseException {
isNotNull();
Long dateFrom = SDF.parse(from).getTime();
Long dateTo = SDF.parse(to).getTime();
boolean found = false;
for (Range range : actual) {
if (range.getStart() == dateFrom && range.getEnd() == dateTo) {
found = true;
}
}
Assertions
.assertThat(found)
.isTrue();
return this;
}
}
And what about the error message? AssertJ allows us to add it quite easily. In simple cases, like a comparison of values, it is often sufficient to use the as() method, like this:
Assertions
.assertThat(actual.size())
.as("number of ranges")
.isEqualTo(expectedSize);
As you can see, as() is just another method provided by the AssertJ framework. Now, when the test fails, it prints the following message so that we know immediately what is wrong:
org.junit.ComparisonFailure: [number of ranges]
Expected :4
Actual :3
Sometimes we need more than just the name of the tested object to understand what has happened. Let us take the hasRange() method. It would be really nice if we could print all the ranges in the event of failure. This can be done using the overridingErrorMessage() method, like this:
public RangeAssert hasRange(String from, String to) throws ParseException {
...
String errMsg = String.format("ranges\n%s\ndo not contain %s-%s",
actual ,from, to);
...
Assertions.assertThat(found)
.overridingErrorMessage(errMsg)
.isTrue();
...
}
Now in the event of failure we would get a very detailed error message. Its content would depend on the toString() method of the Range class. For example, it could look like this:
HourlyRange{Mon Jul 23 12:00:00 CEST 2012 to Mon Jul 23 13:00:00 CEST 2012},
HourlyRange{Mon Jul 23 13:00:00 CEST 2012 to Mon Jul 23 14:00:00 CEST 2012},
HourlyRange{Mon Jul 23 14:00:00 CEST 2012 to Mon Jul 23 15:00:00 CEST 2012}]
do not contain 2012-07-23 16:00-2012-07-23 14:00
Conclusions
In this article we have discussed a number of ways of writing assertions. We started with the "traditional" way, based on the assertions provided by testing frameworks. This is good enough in many cases, but as we saw, it sometimes lacks the flexibility needed to express the intent of the test. Next we improved things a little by introducing private assertion methods, but this also proved not to be an ideal solution. In our final attempt we introduced custom assertions written with AssertJ, and achieved much more readable and maintainable test code.
If I were to offer you some advice regarding assertions, I would suggest the following: you will greatly improve your test code if you stop using assertions provided by testing frameworks (e.g. JUnit or TestNG) and switch to those provided by matcher libraries (e.g. AssertJ or Hamcrest). This will allow you to use a vast range of very readable assertions and eliminate the need to use complicated statements (e.g. looping over collections) in the //then parts of your tests.
Even if the cost of writing custom assertions is very small, there is no need to introduce them just because you can. Use them when the readability and/or maintainability of your test code are endangered. From my experience, I would encourage you to introduce custom assertions in the following cases:
- when you find it hard to express the intent of the test with the assertions provided by matcher libraries,
- in place of creating private assertion methods.
My experience tells me that with unit tests, you will rarely need custom assertions. However, I'm pretty sure you will find them irreplaceable in the case of integration and end-to-end (functional) tests. They allow our tests to speak in the language of the domain (rather than that of the implementation), and they also encapsulate the technical details, making our tests much simpler to update.
About the Author
Tomek Kaczanowski works as Java developer for CodeWise (Krakow, Poland). He is focused on code quality, testing and automation. Test infected TDD enthusiast, open-source proponent, agile worshipper. Strong inclination towards sharing his knowledge. Book author, blogger and conference speaker. Twitter: @tkaczanowski