Testing Ajax Applications with Selenium

Selenium at a Glance

Q: In a typical online store, many operations require user input in order to complete a purchase. As software developer, how can you ensure the quality and correctness of your implementation?

For those of you who develop web applications for a living, it's a question you might encounter often. It would be really useful to be able to test the functionality of your application. But how?

Selenium is an functional testing tool written by ThoughtWorks specifically for Web applications. Selenium tests run directly in a browser, just like real users do. Selenium works in Internet Explorer, Mozilla, and Firefox on Windows, Linux, and Macintosh.

Take a look at the online demo to see Selenium in action. Click the "All" button in the top-right corner to start the test cases running, and with any luck everything should pass with beautiful green bars. Note that actions are a lighter shade of green than assertions, this is because they don't actually test anything like verify or assert commands. If an assertion fails, the failing command will go red and Selenium will stop executing. If on the other hand a verify command fails, it will go red, but won't bring the test to a grinding halt.

Test suites and cases in Selenium are written in HTML - actually they are just HTML <table>s. Every row in test suite is a reference to a test case, such as:

 <tr><td><a href="MyTest.html">MyTest</a></td></tr>

A test-case is represented by an HTML document written in "Selenese", containing a table with 3 columns, room enough for a command and two arguments. A typical test case looks like:

When you start a test-run (e.g. by pressing "All"), the Selenium TestRunner interprets the HTML test-case document, and drives your web-application, which appears in the bottommost frame.

By simulating user actions, Selenium allows you to test your application through it's user-interface. It most certainly does not replace unit-testing, but we often use it to automate functional acceptance tests for web-applications. It can be integrated into a continuous-integration build, for regular automated regression testing. For an in depth look at Selenium, please refer to the online documentation "Selenium: Usage".

Here Comes Ajax

Selenium is particularly useful when some of your web-application functionality is implemented on the browser, in JavaScript.

Ajax, short for Asynchronous JavaScript and XML, is a web development technique for creating interactive web applications. The intent is to make web pages feel more responsive by exchanging small amounts of data with the server behind the scenes, so that the entire web page does not have to be reloaded each time the user makes a change. This is meant to increase the web page's interactivity, speed, and usability.

The "Loading" flag is one of the indications of Ajax

That's the technical definition, but to most of us, Ajax means web pages that act like GMail or Flickr. When you click a link, it will not cause a page reload; instead the browser will converse with the server and refresh just a part of current page. The time delay between clicking a link and seeing the results is what makes things tricky.

Let's say we have a web page containing a text field and a button. The initial value of the text field is "oldValue". If you click the button, some Ajax magic will happen and the value of the text field will be changed to "newValue", without reloading the page. How do we go about testing this? The obvious approach approach is to open the page, click the button, then check the value of the text field. However when we did that in Selenium, the test case failed!

The reason for the test failure is perhaps not so obvious. What happens is that the asynchronous nature of the Ajax call means that the result doesn't come back from the server immediately. So the value of the field changed a few seconds after the button click, but Selenium checked the value immediately and got the old value! So if Selenium doesn't know to wait for the result, how can we get the brilliant Ajax effect tested?

Wait For It...

So how do we get Selenium to wait for the result? One way to solve the problem is to use the clickAndWait command instead of click; the "AndWait" suffix indicates that Selenium should wait for a page-reload. This might seem like it would work, but there's a catch: because the page is not reloaded, clickAndWait will make Selenium keep waiting forever. This clearly isn't going to work.

Another possibility is to insert a pause between the click and assertValue. So lets pause for 5 seconds in between, to give the server enough time to respond. This approach works most of the time, but it may fail if the server takes more than 5 seconds. This might happen for any number of reasons, slow network, overloaded test machine, and so on. You can keep increasing the pause before checking the result, but this will make your tests run slower and slower. So this clearly isn't the best solution because it's not guaranteed to work and makes your tests run much slower than necessary.

Fortunately Selenium has provided support for doing exactly what we want. When a field value changes in the current page, you can use the waitForValue command to make Selenium wait until the expected value appears.

So in order to make the failing test pass we need to replace the failing assertValue command with the following:

When executing this command, Selenium will suspend the execution of current test case and wait for the expected value. When the string "newValue" appears in the text field, the suspended test will go on. One thing to note: if you misspelled the expected value, Selenium will wait for 30 seconds before timing out.

As you may have already guessed, there are lots more commands that can be used to test Ajax effects. For example, if you want to check some plain text updated with Ajax, there's waitForText; if you want to check the title of current page, there's waitForTitle; if you want to verify that an HTML element is removed, there's waitForElementNotPresent. Actually, for every Selenium Accessor, there is a waitForXxxx and a waitForNotXxxx commands corresponding. Whenever you use verifyXxxx or assertXxxx to check something, there will always be a waitForXxxx for testing asynchronous effects.

What happens if the predefined waitForXxxx and waitForNotXxxx commands don't meet your needs? Well for this there is the waitForCondition command which allows you to specify a Boolean expression in Javascript which causes Selenium to wait until the expression evaluates to true. The declaration of waitForCondition is:

waitForCondition		script		timeout (in ms)

This could come in handy when testing more complicated Ajax effects.

Summary

In this brief article, we introduced Selenium, a web acceptance testing tool. Also we discussed how to test Ajax applications with waitForXxxx Selenium commands, as well as demonstrated how to test a simple Ajax effect - an asynchronous text update - with Selenium.

If you want to know more about the waitForXxxx commands, Selenium developers provided a few sample test cases which will show you how to test some common Ajax effects, such as in place editing, auto completion and drag-drop. These samples are based on script.aculo.us, a popular Ajax library which you maybe already be familiar with.

Biography

Jeff Xiong serves ThoughtWorks as an Application Developer with four years of experience in the creation of large-scale enterprise applications and the integration of disparate systems.

Mike Williams is a Senior Developer with fifteen years of industry experience. Besides his Java experience, Mike is an enthusiastic user of dynamic/scripting languages.

Josh Price serves ThoughtWorks as a Developer. He has eight years of experience building enterprise systems using Java and J2EE.