Interview With Aslak Hellesøy on Cucumber For .NET

InfoQ has interviewed Aslak Hellesøy, the creator of Cucumber on its recent support for .NET. Cucumber is an acceptance testing tool for Behavior Driven Development (BDD). At Agile2009, InfoQ’s Mark Levison reported from the Functional Test Tools Workshop that Matt Wynne and Richard Lawrence started to work on a .NET solution for Cucumber, later to be named Cuke4Nuke.

InfoQ: Could you briefly describe Cucumber and what it is?

Aslak: Cucumber is an acceptance testing tool that is designed for Behaviour Driven Development (BDD). It lets people with minimal technical knowledge describe requirements for a system. This is done by providing concrete examples of how they want the system to behave, in plain text. This is expressed in plain language such as "Then there should be $5000 on Bob's savings account".

This plain text starts its life as a requirements specification for code that has yet to be written. As the code is written, the same text also becomes an automated regression/acceptance test. Some people call it executable requirements.

BDD tools come in two flavours - the "unit testing" flavour (such as NSpec and RSpec) and the higher level acceptance testing flavour such as Cucumber, JBehave or other clones of these tools. Cucumber's main goal is to involve more people and roles in acceptance testing and requirements, and make it an enjoyable (addictive) experience.

InfoQ: How would you say Cucumber differ from tools like Fit/Fitnesse?

Aslak: I'll start by explainging how Cucumber is similar to FIT. Both FIT and Cucumber are command line tools. (Fitnesse is essentially a wiki on top of FIT).

Test language
Both FIT/Fitnesse and Cucumber execute acceptance tests written in a high level language. FIT only understands HTML, while Fitnesse simplifies this by offering Wiki syntax to write tests. In FIT/Fitnesse all tests have to be in a table form.

Cucumber on the other hand lets users write tests in plain text files in almost free form English. Ou Français. Eller norsk. или Русский. It supports almost 40 spoken languages. In essence, Cucumber recognises lines that start with the keywords "Scenario", "Given", "When" or "Then". This kind of language lends itself better to expressing *behaviour* than tables, which tend to be more about *data*.

Of course, Cucumber also lets you use tables interspersed with your plain text, so you can write tests that are a mix of behaviour oriented and data oriented. The funny thing is, Fitnesse now lets you use Scenario, Given, When and Then in tables, so there is definitely inspiration going both ways. It's friendly competition.

Here is an example of a Cucumber .feature file mixing tables into the text:

# language: en
Feature: Addition
  In order to avoid silly  mistakes
  As a math idiot
  I want to be told the sum of two numbers

  Scenario Outline: Add two numbers
    Given I have entered <input_1> into the calculator
    And I have entered <input_2> into the calculator
    When I press <button>
    Then the result should be <output> on the screen

  Examples:
    | input_1 | input_2 | button | output |
    | 20      | 30      | add    | 50     |
    | 2       | 5       | add    | 7      |
    | 0       | 40      | add    | 40     |

Programming language
Both Fit and Cucumber needs "glue" code that sits between the high level language and the code under test. In FIT this is called fixtures, in Cucumber it is called step definitions. Cucumber lets you write step definitions in C#, Java, Scala, Clojure, Groovy, Javascript, Ioke and Ruby, so Cucumber is more versatile than FIT in this regard.

Here is a step definition in C# that matches entering numbers into a calculator - from the example above:

[Given(@"^I have entered (\d+) into the calculator$")]
public void EnterNumber(double n)
{
  _calculator.Push(n);
}

Editor
FitNesse's big advantage over Cucumber right now is the Wiki. Cucumber doesn't (yet) have a web based editor that lets non-technical people edit the acceptance tests. There is a tool called Lowdown, but it doesn't support tables, so it quickly falls a little short. JetBrains' RubyMine has a decent editor, but it's geared towards programmers.

InfoQ: As I understand it, Cucumber now supports two different ways of using the tool on .NET. Could you briefly describe these two and their pros/cons?

Aslak: That's right. Cucumber itself is written in Ruby, so it needs a Ruby interpreter to run. The first option for .NET is to run it on IronRuby. If you do this, you currently have to write your step definitions in Ruby code (that will talk to your .NET code). However, there are plans to change this so you can use C# as shown above.

The other option you have is to use something called Cuke4Nuke. This is the tool that lets you write step definitions in C# and forget all about ruby code. This tool actually uses MRI (Matz Ruby Interpreter, the original, which is written in C) and sends commands to a local C# socket server, which will run your step definitions. But to the user that's all transparent - they will only ever deal with .feature files, .cs files and the cuke4nuke command line tool.

I'd also like to mention a third project - SpecFlow. This project is 100% .NET and doesn't share any code with Cucumber, but it tries to replicate as much of the behaviour as possible. Plus - it has Visual Studio integration. I'm talking to the SpecFlow team about collaborating with Cucumber, and it looks like we're going to use the same feature file parser. This would make Cucumber's test language (which is called Gherkin) more consistent across tools.

InfoQ: What is on top of your wish list for Cucumber?

Aslak:
Short term:

• Speed, using a faster parser. We have done great progress on this in the Gherkin project, which uses Ragel.
• A new website. The wiki is a great resource, but the information needs to be restructured to be more accessible to newcomers, non-ruby programmers and non-programmers in general.

Long term:

• An easy to use collaborative environment for editing and running features. There is already RubyMine and LowDown, but I have some different ideas myself.
• Ability to edit and store tests in more formats and more mediums. For example on a set of wiki pages on your intranet. Or Word documents. What made FIT popular in its early days was Excel support, and in order to win the non technical side over we need to support their preferred tools. Or give them better ones.

InfoQ: Thanks for taking the time to talk to InfoQ.