Requirements and test cases, in particular acceptance tests, are strictly related. Agile methodologies highlight this aspect basing themselves on test driven approaches. The notation of UML use cases may be enhanced in order to allow enhanced UML tools to properly handle links between use cases and tests.
Acceptance tests are part of the design
One of the best practices of the agile paradigm is the “Test Driven Development” or – in the perspective of Dan North1 - the “Behaviour Driven Development”. When I read for the first time the introduction to BDD, I was particularly impressed by the concept of “executable specifications” built upon the text template:
(T1) Given <context> {and <context>}, when <event>, then <outcome> {and <outcome>}
“Executable specifications”, what a wonderful idea! Regardless of the usability of the concept in any context - for example it is very difficult to imagine how to apply it in a literal sense inside a contractual document -, it was the implicit conveyed message that changed the way I see the tests:
- the acceptance tests are part of the design!
Why only acceptance tests? What about unit testing, integration testing, system testing? Well,
- Unit tests are typically defined to check low level building blocks - such as class methods - that are subject to many implementation decisions to be seriously taken into consideration: I don’t see unit tests as design components, but as aids automatically generated by tools from source code.
- System (or subsystem) tests are defined to check readiness to the delivery of high level components, so I regard system tests as equivalent to the acceptance tests.
- Integration tests raise the question: my simple answer is that they should be considered acceptance testing. Perhaps the word “should” requires a deeper understanding and it is related in some way to “black box” and “white box” tests. A “black box”2 test focuses on what a system does, while a “white box”3 test on how4. More a test falls within the category of “white box”, the further away from the category of “acceptance”. In other words the more an integration test falls within the category of “black box”, the closer to the category of “acceptance”.
Some agile methodologies, among them XP (eXtreme Programming) and Scrum, suggest using user stories as artifacts for requirements. Some authors, see for example Mike Cohn or Scott Ambler say that the user stories are particularly effective if concise and schematic suggesting the adoption of the following template5:
(T2) As a <role>, I want to <goal/desire>, so that <benefit>
The first impression is that the “so that” clause is optional6, but Chris Matts7 thought that it was so fundamental to propose the following template:
(T3) In order to <receive benefit> as a <role>, I want to <goal/desire>
My point of view is to accept the Chris’s thesis and to regard as essential the “in order to” clause or the equivalent “so that”. As a matter of fact any requirement needs at least an acceptance test and such clauses - in my opinion - shall be used to communicate what is expected by the acceptance tests: the <goal/desire> implies a test whose outcome shall comply with the <benefit>.
In a hypothetical tool that manages both user stories and acceptance tests, each acceptance test should be linked to a specific “so that” clause of a user story and not directly to the user story.
To hit the nail on the head, my vision of the user story template is:
(T4) As a <role>, I want to <goal/desire>, so that <benefit> {and <benefit>}
Each “benefit” – more than the entire “so that” clause - shall have at least a related acceptance test.
UML use cases and acceptance tests
I wonder if the same concepts can be conveyed through UML use cases. In this paper “use case” shall be always be interpreted as “UML use case diagram” or “UML use case artifact”, I avoid to deal with use case descriptions as they have both desirable and undesirable features compared with UML. My aim is to present an idea that fits very well with UML diagrams.
A kind of bijection may be established between a part of a user story and a use case diagram as highlighted by the Figure 1: the “As a <role>” clause corresponds to the “UML actor”, the “I want to <goal/desire>” clause corresponds to the “UML use case”. See it as a structural relationship, a theoretical possibility, not as a good or recommended practice: there are examples that make a mockery of the translation, see Alistair Cockburn8 for a sharp discussion.
Figure 1 : map between user stories and UML use cases
The UML use case artifacts are not strictly able to express the “so that” clause, they can only introduce it as a comment (see Figure 2):
Figure 2 : map between user stories and UML use cases
In this light the UML use case is less expressive than a user story: a hypothetical tool that manages both UML use cases and acceptance tests is forced to connect an acceptance test to a use case, as connecting a primary element (the acceptance test) to an auxiliary element (the comment) is logically wrong and shall be forbidden.
The situation would be quite different if the UML use case artifacts allow to define the acceptance tests as part of the artifact. I figure out that the acceptance tests are for use cases what methods are for classes (see Figure 3).
Figure 3 : Acceptance Tests are for UML Use Cases what Methods are for Classes
Obviously I’m abusing of the current UML notation to explain my vision.
Simple and complex requirements
The requirements may be divided in two categories: simple and complex. A simple requirement involves only a single piece of business functionality, while a complex requirement involves more than a single business function. A single function does not necessarily map to only a single UML use case, as a main use case decorated with included and extended use cases shall not be intended as a complex requirement. An example will clarify the distinction.
Suppose you have the following stakeholder request:
(SR-1) The system needs to manage a list of products by adding, updating, retrieving items.
It may be rendered by the UML use case diagram in Figure 4 that specifies five simple requirements:
Figure 4 : Product management use cases
As you can notice the included “select product” use case has not been counted as simple requirement.
Suppose you have now the following stakeholder request:
(SR-2) All operations shall be logged for auditing purposes.
The naïve approach (see Figure 5) cannot be satisfactorily: in effect, how can we define acceptance tests?
Figure 5 : naive approach to complex requirements
The question shows one of the reasons why coupling UML use cases with acceptance test cases is strongly desirable: the most important attribute of a requirement is testability, so must be also for UML use cases.
The UML notation allows to define abstract use cases and it allows to define use cases that specialize other use cases and this enables you to draw the remarkable diagram shown in Figure 6.
Figure 6 : abstract use cases
This diagram is exceptionally clear to all object oriented proficient people and easily explained to all people. I would emphasize that the abstract nature of the “logged operation” use case delegates the specialized concrete use cases to supply the required acceptance tests.
Be aware of the big difference between what I called simple requirements and what I called complex requirements: the first ones allow a direct link with tests, while the latter ones do not have such facility and depend on other features.
The problem is how to ensure that all (acceptance) tests are defined.
Tests as UML use case methods
Let me change the Figure 6 in the Figure 7 (also here abusing of UML notation).
Figure 7 : tests as use case methods
The hypothetical tool introduced earlier could handle automatically the abstract “test logged operation” for example generating the required test for all concrete derived use cases. Be aware that the use case shall anticipate the test case, but it is not its responsibility to detail it: the “use case test method” shall include the name, perhaps the input parameters, perhaps the mode (i.e. for inspection, demonstration, log analysis, etc).
Conclusion
I consider user stories a valuable tool for gathering stakeholder’s requests, but they require further elaboration to detail requirements. Moreover they are not entirely suitable for incorporation in graphical tools: if so they would be more or less equivalent to simple UML use cases. UML use cases offer richer capabilities and, although they are aging, they still retain hidden possibilities: there is room for enhancements. Many modern design tools add to UML diagrams other kind of diagrams in order to support requirements, test cases and some sort of traceability between those elements. I think that a better solution is to make the notation of the UML use cases more powerful as sketched in the previous paragraphs: I see only advantages to encapsulating (acceptance) tests inside the UML use cases, this may also open the door to the introduction of a new UML classifier, the “acceptance test” with the test details. I will end by restating that, in my humble opinion, acceptance tests are part of the design as implicitly or explicitly preached by the agile practices. In a more complete frame, the definition of a system - a high quality definition - conforms to the following equation "SYSTEM DEFINITION" = "REQUIREMENTS" + "ARCHITECTURE" + "ACCEPTANCE TESTS", but its full explanation is matter for another article.
References
- See here or also here for Behaviour Driven Development.
- Black-Box Test
- White Box Testing
- “black box” testing is a synonym for functional testing, and “white box” testing it is for structural testing: see this example
- It was developed by a team at Connextra in 2001.
- Perhaps Mike Cohn was the first to suggest this thesis.
- To learn about this author please read this article.
- Alistair Cockburn's discussion.
About the Author
Raul Rugiero is a Capgemini Enterprise Software Engineer with more than twenty years of experience in the production and delivery of information systems. He worked in several industries including public administration, telco and - in recent years - aerospace and defense. Raul has held and continued to cover many roles - his favorite is software architect -: nowadays he is dealing mainly with requirements analysis, architecture design, project technical leadership and where appropriate also component development. Not a fan of any particular technology or methodology, but he follows with interest the evolution of many of them.