Last week Jeremy D. Miller announced a preview release of his StoryTeller project: an open source .NET project for “Executable Specifications”. InfoQ sat down with Jeremy and asked him about what StoryTeller is, how it differs from other tools like Fit/FitNesse and Cucumber, and what the future looks like for the project.
InfoQ: How would you describe StoryTeller?
Jeremy: StoryTeller is a toolkit for creating "Executable Specifications" with .Net for systems built with .Net. Imagine a world where you as a developer get detailed requirements in a format and mechanism that is readable by the business facing people, completely devoid of ambiguity in terms of what the correct behavior is, and can also be used as an automated test within your team's continuous integration processes. StoryTeller makes this possible in the .Net ecosystem.
InfoQ: What motivated you to write StoryTeller?
Jeremy: I had a lot of experience using FitNesse to create automated tests over a couple different projects and at least four different employers. I loved the concept of using FitNesse to write human readable automated tests, but invariably my teams incurred quite a bit of friction with FitNesse. I originally set out to build a different editor and test management infrastructure for the FitNesseDotNet library. I made some progress but eventually scrapped that approach and wrote a completely new testing engine from the ground up that is optimized for .Net usage. Along the way I think I've created the better tooling experience that I was looking for from the beginning. Specifically, I wanted a tool that did what FitNesse did but allowed for cleaner testing language, easier mechanics for the developers writing the test automation code, and easier integration with source control and continuous integration than I felt was possible with FitNesse.
And let's not lie here, it's been an absolute blast to build and like any OSS side project it's made me a better developer for the experience.
InfoQ: How would you say StoryTeller is different from tools like Fit/FitNesse and Cucumber?
Jeremy: Let's be very clear about this, StoryTeller was built specifically to be a better Fit/FitNesse for the .Net ecosystem and owes quite a bit of inspiration to my prior experiences with FitNesse. StoryTeller is very similar to FitNesse in the sense that you code "Fixture" classes that implement and contain "Grammars" that you can use to express test specifications in an English like way. The first difference is that StoryTeller does "projection editing" that renders an html display instead of parsing html. The other difference is in tooling. FitNesse requires you to edit tests in a Wiki with a format that's a frequent source of complaints. StoryTeller tests are authored with a client written with WPF that greatly accelerates the process of creating tests.
Cucumber is a Ruby tool that seeks to solve the same problems as StoryTeller, but with different mechanics. While you can use Cucumber with IronRuby to write executable specifications for .Net code, there are immediate benefits to writing test automation code in the same language as your production code. Today you might use Cucumber and Ruby to code your tests because it creates readable tests, but with StoryTeller your developers can use C# to craft human readable acceptance tests.
InfoQ: Do you see StoryTeller as a developer tool only?
Jeremy: At this time StoryTeller is a bit developer centric in the way that you have to create tests, but in the long term I see StoryTeller as a tool for testers and business partners to express requirements as automated acceptance tests. Our (one) tester uses StoryTeller quite extensively.
InfoQ: Would StoryTeller easily integrate with CI tools?
Jeremy: Absolutely yes, and this was one of the main drivers for the architecture of StoryTeller. Everything about StoryTeller is "XCopy deployable." The StoryTeller tests are just Xml files stored on a file system and the tool makes no assumptions about where the actual binaries for the system under test will reside. For CI builds, StoryTeller includes a command line runner that can be executed from any build tool that supports shelling out to the command line. We run our StoryTeller test suite in a cascading build using JetBrains Team City CI tool and have integrated the StoryTeller results into our Team City website.
InfoQ: I understand you have dogfooded StoryTeller at Dovetail. How much have this helped in shaping the application?
Jeremy: I couldn't possibly overstate the importance of dogfooding StoryTeller. I've eliminated performance bottlenecks due to our experiences. I've found several ways to streamline the mechanisms for coding the Fixtures and Grammar building blocks for developers based on negative feedback from our first round of testing. Early on we noticed quite a bit of repetitive coding just to generate test grammars, so we found ways to eliminate duplication in the test automation code. I've routinely needed to improve the way that StoryTeller presents test errors and results to make failing test errors easier to diagnose due to feedback from our usage of the tool. Recently, I've had to invest more time into hardening StoryTeller against coding errors in a way that lets the developers writing the test automation code know exactly what's going on when things don't work correctly. Most recently, our team has finally hired a tester. Her usage of the editing tool and the friction that she encounters with StoryTeller has driven the last wave of functionality specifically to improve the ease of editing tests.
InfoQ: What can we expect from StoryTeller going forward?
Jeremy: Probably not too much in the immediate future. I'm hoping for some feedback from early adopters. Right now, StoryTeller is very developer centric and requires you to code at least a skeleton of the Fixture first before editing tests -- which was a conscious decision on my part because Dovetail was a "developer only" team for a long time and it is of course very optimized for my team. Going forward, I want to add ways for testers and business facing users to create the testing language directly in the tool without developer intervention so that it can really become a way to write executable specifications ahead of development.
InfoQ: Thanks for taking the time to talk to us Jeremy. More information and the source code can be found at the StoryTeller project page.