Paul Duvall, CTO, Stelligent Inc. describes in the "Automation for the people" series, how JMeter and Ant can be integrated with a continuous integration server to allow developers to quickly determine the behavior of the developed system under certain load conditions and adapt to changes. Duvall starts by enumerating some of the metrics that can be measured by these tests:
- How many concurrent users can access your software system?
- How much data can be loaded without performance degradation?
- What are your system's throughput requirements?
- How often are these requirements tested?
- What if you could specify and validate that these load and performance requirements are being met at least once a day?
Duvall adds that the automated tests scheduled by a CI server can be run at those times during the day when machine usage is lower as well as allow to "reduce bottlenecks and errors that can occur when you depend on a single person to configure and run the tests". He describes the main steps needed to setup such an environment:
1. Run JMeter and Create a Test Plan. An Apache project, JMeter is a Java tool designed to load test functional behavior and measure performance. It provides both GUI and command-line invocation facilities and allows the execution of various load tests which can be defined by using a test plan. Sample test plans include Web test plans, FTP test plans, Web service test plans, etc. These test plans can be setup through a GUI interface and then saved in an XML format. This accommodates the parameterization of configuration values in order to customize the way the tests are run.
2. Drive JMeter Tests with Ant
Define a JMeter Ant task, based on a implementation already written, ie: "
<jmeter
jmeterhome="${jmeter.home}"
resultlog="${basedir}/target/JMeterResults.xml">
<testplans includes="BreweryTestPlan.jmx" dir="${basedir}/tests/load">
</testplans>
</jmeter>
3. Render test results reports using XSLT AND HTML. He then presents how the output of the test execution - "JMeterResults.xml" - can be transformed into an HTML report using an XSLT script provided in the JMeter extras directory. The generated report displays each of the load tests run, along with test status, time, and aggregate status and time of all of the tests.
4. Customize JMeter tests. The way tests are run can be customized through the use of parameters and proprieties. In order to adapt the tests to a specific environment (e.g. testing or staging), tokens can be placed in the .jmx file and later on filtered and modified before the tests are run against a specific environment. Duvall demonstrated an example of how to increase the JVM memory and specify the number of threads and loops.
5. Integrate JMeter tests within a CI environment. Duvall provided an example of integration with the Continuous Integration server CruiseControl to schedule a build to run on a daily basis at 11PM:...
<modificationset>
<svn repositorylocation="${svnrepo.location}">
<timebuild time="2300"/>
...
</svn>
</modificationset>
JMeter test results reports can then be made available from the CruiseControl project dashboard by adding a few lines to CruiseControl's config.xml file.
