In any programming language and platform, having a set of well-though unit tests, is a commonly accepted practice in order to deliver maintainable code. This is especially true for a dynamic language like JavaScript and there are currently several frameworks and libraries for a team to choose from.
InfoQ had a Q&A with the creators of some of the leading JavaScript unit testing frameworks about their projects and what they offer to developers.
The participants were:
- Jörn Zaefferer for QUnit,
- Davis Frank for Jasmine,
- Tommy Montgomery for Jarvis,
- Felipe Nascimento de Moura for jfUnit.
InfoQ: What does your project actually do and how is it different than the other JavaScript Testing Frameworks?
Jörn Zaefferer : QUnit is a JavaScript unit testing framework. It provides the most value when used for unit tests that run in a browser. While it is a project under the jQuery umbrella, it doesn't depend on jQuery and not even on the DOM parts browser's provided, so you can run QUnit tests in node.js or Rhino. QUnit is very easy to get started with, as you only need to include two files in a html page, no need to install and build anything. A minimal functional testsuite is a html file with just 11 lines of code.
Davis Frank : Jasmine is a JavaScript testing framework, meant to bring a Behavior-Driven Development style to test-driving JavaScript code. How are we different? We aim to be BDD (versus standard TDD), so we try to help developers write more expressive and well-factored code than xUnit frameworks. And we strive to have few dependencies - so you should be able to write Jasmine specs for Node.JS or browser-based apps or mobile-apps.
Tommy Montgomery: Jarvis is a unit testing framework for JavaScript, based on the API of NUnit, which is a unit testing framework for the .NET stack. A couple of things that differentiate it from other frameworks are stack trace dumps, colored diffs for string comparisons, and a more readable API.
Felipe Nascimento: The main characteristic of jfUnit is the way it is loaded, and written. The way you write your unit tests is easier, with less typing and you can run it in your development environment even fro a console, whenever you want.
InfoQ: Can you elaborate a little on its architecture? What are its main components?
Jörn Zaefferer: The QUnit API is very straightforward: You call the test method, pass a name and callback, then implement the actual test inside that callback, using various assertion methods. Around that QUnit provides some more options to organize testsuites, like modules with setup and teardown callbacks. Internally these calls are mapped to a queue. Dependening on the enviroment, QUnit starts executing tests from that queue on window.load or some other events (or when triggered by the user). Queue execution is stopped when a test indicates that it needs to run asynchronous, and restarted once that test indicates its done. The order of queue execution is variable: An optional feature, on by default, will reorder test execution based on results of a previous run, yet still showing the results in order. Along with the option to hide passed tests, you can rerun a suite that had failed before and verify that bad tests now pass or still fail in an instant, instead of having to wait for the full suite to run. Most of QUnit's features are tested using a QUnit-based testsuite.
Davis Frank: We try to keep it simple. Jasmine, at it's core, is one JavaScript file that if included provides a DSL for laying out suites, nested suites, and specs. The DSL builds up a tree of spec-related functions and executes them. That part is pretty simple.
There are layers on top of this to allow reporting of results in different environments and to make it simpler to manage source and specs files. We can work with Ruby on Rails, or any web framework. Some folks are test-driving Node.js projects. Really, anywhere that JavaScript runs, Jasmine should run. If not, then we need to make it run there.
Tommy Montgomery: Jarvis, like other XUnit frameworks, is constraint based. Each assertion creates a constraint object under the hood which evaluates the expected value against the actual value. For example, running something likeAssert.that("foo", Is.equalTo("bar")); will perform an equality comparison with "foo" on the left and "bar" on the right. This API (again, borrowed from NUnit) is very easy and intuitive to read, which I think is a very important part of unit testing. At the user level, the main components are the Assert object, the Is object, and the runner. Assert exposes a couple of functions to make assertions on values, while Isexposes the constraint-based architecture of Jarvis. To run tests, you would call Jarvis.run(), passing in your test function. The way Jarvis reports the test results is configurable, and ships with two different reporters: one for the console (e.g. Firebug in Firefox) and one that generates HTML.
Felipe Nascimento: It uses a structure with a global element, called jfUnit, in which you will be able to add or remove your tests. The tests are not executed when you describe them, but when you tell this main object to run them, what may be done even from your console or the browser address bar. What I would advise you would be to load it(the library itself, and the scripts with your tests) automatically in a development and test environments, removing them when in production. It will allow you to execute your tests whenever you want, retrieving its results right in the screen.
InfoQ: Also what is the best way for a developer to get started with your framework, and what is the typical workflow?
Jörn Zaefferer: There's a good article introducing QUnit on Script Junkie - it covers a basic introduction, most of the QUnit API as well as some advanced features of the testrunner that can make test-driven development with QUnit a lot more efficient.
Davis Frank: There are a bunch of different screencasts and tutorials popping up. The Railscasts one is pretty nice. As is PeepCode's CoffeeScript cast. The program is about CoffeeScript, but Geoffrey does a great job of representing a nice test-driven flow withe Jasmine.
Tommy Montgomery: The best way for a developer to get started with Jarvis is to visit the website, take a look at the example code, and then run the tests right from the site to get an idea for exactly what tests look like and what the test result report looks like. Jarvis itself is tested using Jarvis, and the test results are available to look at as well here. From there, it's as simple as downloading the framework and plugging it into your project. The typical workflow for writing tests is dependent on the developer, and Jarvis makes no assumptions about that. Since the main test runner is HTML based, having a browser open is a necessity. My personal workflow is write code, write test, refresh the browser, view test results, repeat.
Felipe Nascimento: Well, you can find some more detailed data about it here. Basically, the developer will have to load the library and then use the jfUnit object setting as many tests as the developer may want to use. Once this is ok, the developer can execute the command jfUnit.run() to see its results. There is, also, a jfUnit.config({...}) with which you can set(optionally) some parameters or environmental variables. The only thing that I might call your attention would be that it uses the document body element to build the report. To do so, it will just require that your page have at least the body element already rendered.
InfoQ: What features would you like to see implemented in the future and how do you see JavaScript testing frameworks evolve as a whole?
Jörn Zaefferer: A priority right now is better command line integration, e.g. within node.js. While that already works, you currently need a additional script that isn't yet part of the main qunit.js file. Part of that integration would be publishing QUnit via npm. Other then that the focus is on improving existing features. Eventually we may pull in a BDD wrapper like Pavlov - until then anyone wanting to use QUnit but preferring BDD should just use Pavlov in addition to QUnit. Not a code feature, butqunitjs.com should launch sometime this year.
Davis Frank: We have a very public backlog on Pivotal Tracker, so you can see what our priorities are through the next few releases. We welcome contributions from the community, but we're admittedly picky. We want to keep Jasmine small and essential. Keep your fixes/features simple and please make sure the include specs for your work. Jasmine is, after all, a test framework. So test drive your test framework contributions!
Tommy Montgomery: Right now I'm working on nodejs support for server side testing. A couple of other future features I'm thinking about are an ant task and a command line executable. JavaScript testing frameworks are severely hindered by requiring a browser to be running (in most cases). To that end, making bootstrapping browsers easier for users will become a staple of any JavaScript unit testing framework. I also see JavaScript unit testing frameworks getting better at bridging the gap between server-side and client-side testing.
Felipe Nascimento: I'm a javascript lover and I believe this language will be in the future of most of developers in the word. I like the way things are going with javascript, with new tools and techniques. I am one of BrazilJS(the Brazilian JavaScript conference) and I can wonder how strong the future of javascript is. I am really up to create the new features to this unit test tool evolve enough to deal with Ajax calls in an easier way, plus, allowing the user to write in a extremely easy way, scripts to test the graphic interface itself, like filling forms in, submitting elements, clicking in buttons(what is already possible, but a bit harder to write), etc. I'd also like to add here that, if you're a developer and may want to contribute to this tool, please, feel free to talk to me and send your criticisms, ideas, suggestions, bug reports or code. To help you with that, you may want to follow the project in my github account.
About the Panelists
Jörn Zaefferer is a freelance web developer, consultant and trainer, residing in Cologne, Germany. Jörn evolved jQuery’s testsuite into QUnit, a JavaScript unit testing framework, and maintains it. He created and maintains a number of popular plugins. As a jQuery UI development lead, he focuses on the development of new plugins, widgets and utilities.
Davis Frank is a software engineer, dad and baseball fan living in Northern California. By day he is a software engineer and manager for Pivotal Labs, Inc. coding for clients. Other times he reads to his kids, makes his wife laugh, grills flank steaks for anyone who comes by, and ensures that the Internet is up at home with roughly three 9s.
Tommy Montgomery graduated from college with a BS in math, minor in computer science. Moved to San Diego in 2007 to do .NET work. Since then I've been primarily working on LAMP-related projects, but in the last year or so got more into .NET. My programming interests are centered around PHP, C# and JavaScript. Recent projects include Sunlight, a client-side syntax highlighter, Jarvis, and Butterfly, a markup-to-html transformer in .NET and JavaScript.
Felipe Nascimento de Moura is a Senior developer and System Analyst in Brazil. He works with web development for almost seven years and is the creator of some open source projects, such as theWebMind.org, PHPDevBar and jfUnit, besides being one of the organizers of BrazilJS, the Brazilian Javascript Conference.
You can find more interesting articles about JavaScript and Unit Testing, right here on InfoQ!
