An important property of a good test suite is that it runs fast, the faster the better. But over time, as tests accumulate, the test suite will take longer to complete. A countermeasure would be to run tests in parallel, maybe even on several systems (after all, tests should not have any interdependencies). This is were Devver comes into play: Devver (pronounced like "developer") will run your tests on their cloud in parallel. For example, the CruiseControl.rb's integration test suite runs in just 17 seconds, compared to the 59 seconds a test run needed on their local machine.
Devver comes in the form of a gem you install and a Rakefile for your project that contains the new tasks, like rake devver:test to run all unit, functional, and integration tests on their infrastructure.
InfoQ talked to Benjamin Brinckerhoff from the Devver team to learn how exactly they can speed up your tests:
We partition the tests and then distribute the tests among many machines. Each machine has a full copy of the test environment, including the test database and gems.
But what happens when a user needs an additional gem to run the tests?
That's no problem. Since most of our users have Rails projects, by default we run 'rake gems:install' to install the required gems (each user has their own separate set of gems). If that doesn't work for any reason, a user can configure the step that installs gems. So, for instance, a user could run 'geminstaller' instead.
Running the tests faster is certainly very helpful, but what about running them with different versions of Ruby in parallel?
Our current version does not yet support multiple versions of Ruby, but it is a feature we're planning on supporting in the future, especially if we hear a lot of demand for it.
Devver is currently in a beta and during this time free of charge. Afterwards, they "plan to offer Devver free of charge for open source projects but will charge closed source projects a monthly fee". Visit the Devver knowledge base for more information on Devver or just sign up.