Although Behaviour-Driven Development (BDD) can help in improving how business stakeholders and software developers communicate with each other, as Kevin Smith recently claimed, there are some common anti-patterns when using Cucumber to run the automated tests, which Aslak Hellesøy, (co-creator of Cucumber), Matt Wynne and Steve Tooke described in a recent discussion.
Many Cucumber anti-patterns concern scenarios, which is a description of some behaviour at the feature level. A scenario should normally be phrased declaratively using the language of the domain, to describe what should happen. The structure starts with an initial condition, followed by an event which triggers the scenario, and finally, the expected outcome in one or more clauses. A common template is the Given-When-Then format:
Scenario: Successful withdrawal from an account
Given I have €100 in my account
When I request €20
Then €20 should be dispensed
Writing a scenario after code assumes that Cucumber is a testing tool, and although it can be used as such, it foremost is a tool for testing your understanding of the problem domain and finding potential misunderstandings together with domain experts, before the code is written.
Creating scenarios by domain experts in isolation will not represent a common understanding and miss the important input from developers and testers. Lacking a technical input, these scenarios also tend to be harder to automate.
Testing through the UI can be problematic. User interfaces tend to change much faster than the business logic which often lead to failing tests. When the scenarios or business logic hasn’t changed, the effort to get the tests running again is unnecessary work. They are also very slow to run since they have to interact with all parts of the application, all the way into data storage and back again. This way of testing may also lead to a lack of understanding of the domain. Describing the usage of an UI is mainly a generic language common to many domains, which may lead to scenarios not written as a reflection of the domain they are supposed to describe.
Thomas Sundberg, a seasoned BDD practitioner in Sweden, refers to the agile testing pyramid, and argues that BDD should be used in all places where the business has reason to have opinions about the behaviour. He prefers to focus on integration tests using BDD and as little as possible through the UI. He emphasizes that Cucumber is not a testing tool primarily, but a tool for capturing a common understanding of how a system should work.
Keeping noisy scenarios like checking for an empty bank account, can obscure the relevant parts of the documentation. They are obvious but still needed to get the first iteration running. The advice from Hellesøy and the others is to delete these scenarios after a while, or at least rephrase them into something more useful.
Overuse of scenario outlines tends to lead to slow tests. With scenario outlines new scenarios are added using a template, making it very easy to add a lot of scenarios. The recommendation when using outlines is to avoid testing through the UI or other ways that are slow.
Other anti-patterns discussed include Testing many rules together and Poor naming of a scenario which both lead to misunderstandings about the purpose of a scenario. Incidental details and Too vague scenarios add no real value, either by describing too many irrelevant details or by being too abstract not containing any details at all.