Writing Automated Tests with Jazz Automation

Introduction to Jazz Automation

Jazz Automation is a testing framework built to automate and speed up acceptance/functional testing for all types of web based systems or static websites and in any industry. It also lends itself to easily implement automated integration testing. Historically this type of testing has been all manual, labor intensive, and inaccurate. With automated testing, companies are producing higher quality applications, producing fewer production bugs, and increasing their speed to market. Jazz Automation supports the following web browsers: IE, Firefox, Safari, Chrome.

Jazz Automation differs from other automated testing frameworks by allowing test writers to describe how software features work in plain and understandable English.

Most automated testing frameworks, such as Selenium and Cucumber, require programming in languages like Java and Ruby before tests are executed. Jazz Automation provides an abstraction layer, removing programming requirements from test writers and configurators.

Writing Your First Test Script

Let’s take a look at a specification for testing an eCommerce website.

Feature: Go to the Amazon web site, search for Harry Porter in the book section. Find
 Harry Potter and the Chamber of Secrets (book2) and then add it to the cart.
   
   Background: Establish the test settings for the test
     Given the following settings:
       | url             | http://www.amazon.com | 
       | platform        | Vista                 |
       | browser         | firefox               |
       | browser version | 23                    |
   Scenario: Verify that we have an empty cart on the amazon home page
     Given I am ON the "HomePage"
     Then I should EXPECT
       | cartCount       | 0                     |
   Scenario: Search for Harry Porter from the books category
     Given I am ON "HomePage"
     And I click "allButton"
     And I select
       | selectCategory  | Books                  |
     And I enter
       | searchField     | Harry Potter           | 
     And I click "go"
     And I wait 5 seconds
     Then I should be ON the "SearchResultsPage"
   Scenario: From the Search Results Page, verify the first result Special
             Edition Harry Potter Paperback Box Set 
     Given I am ON "SearchResultsPage"
     Then I should EXPECT
       | firstResult           | Special Edition Harry Potter Paperback Box Set |
       | chamberOfSecretsBook2 | visible
       |
   Scenario: Click on the Chamber Of Secrets Book2 to reveal the details
        Given I am ON "SearchResultsPage" 
        And I CLICK "chamberOfSecretsBook2"
        Then I should be ON the "BookDetailPage"
   Scenario: Verify the details
        Given I am ON the "BookDetailPage" 
        Then I should EXPECT
        | kindlePrice          | $7.99                    |
        | hardcoverPrice       | $13.94                   |
        | paperbackPrice       | $8.56                    |
    Scenario: Add the book to my cart
      Given I am ON the "BookDetailPage"
      And I click "hardcover"
      And I click "addToCart"
      Then I should be ON the "PreCheckoutPage" 
    Scenario: On the pre-checkout page, verify the details
      Given I am ON the "PreCheckoutPage"
      Then I should EXPECT
        | orderSubtotal            | 13.94       |
        | cartCount                | 1           |
    Scenario: proceed to checkout
      Given I am ON the "PreCheckoutPage"
         And I click "proceedToCheckout"
         Then I should be ON the "SignInPage" 
         The test script should be extremely easy to read and understand, right? 

The test case language shown above is Gherkin. Gherkin is a domain specific language (DSL) that allows us to describe how an application should work in plain, understandable English. When developing test scripts, the scripter should not need to understand or describe the implementation. Most of the above specification is free-text; however Gherkin has a few key words: Feature, Scenario, Given, When, And, and Then. Jazz Automation has extended this list of key words to include: Expect, On, Click, Hover, Wait, Refresh, Enter, Select, Forward, Backward, Visible, and Invisible.

There’s more to Gherkin than will be discussed here; however what’s important to know is that it uses a line-based format must remain intact, so that Jazz Automation will know how to execute the steps for each feature.

With Jazz Automation, there is no need to write any code, be that Java, Ruby, etc. to execute the above test. With Jazz Automation, each website page must be defined as a Page. Both JSON and XML are supported for Page configuration. Each page contains high level attributes that allow Jazz Automation to verify that a page has been loaded properly. Also, each page should have one or more elements defined. Take a closer look at the above specification and you will notice that several words are surrounded by double quotes. The objects within the double quotes are significant to Jazz Automation. These represent Page objects or elements on a page. These elements are defined in a one or more page files for each project.

Let’s take a look at the test folder structure.

First we notice the /features directory. This where all test specifications should reside. The file is of type .feature. Jazz Automation can execute one or more features at a time. The features and order in which they are executed is defined in the jazz.properties file. Next, we notice there are several ‘page files’ located in the /pages directory. This is where a Test Configurator will define each page and page elements that will be accessed during a test. The test specifications can only refer to pages defined here and to page elements defined within a page file. We will cover custom locations for page files later. Lastly, we notice the mandatory jazz.properties file. Let’s skip over it for now.

On to Page Configuration

Test Configurators are the key to properly configuring a Jazz Automation test. Jazz Automation supports both JSON and XML for page configuration.

Test configurators need to have experience with some web technologies such as jQuery. Test configurators need to create a page for each website page to be accessed.

A page has a few required attributes. Each page must have a pageName and at least one domElement. The pageName is heavily used in features and it should be meaningful so that testers know exactly website page it refers to. DomElements are the web page elements used to validate the software. Typically, they refer to label values, drop-down values, images, buttons, data displayed (pricing, etc.). The name of these elements should be meaningful so that the test writers understand each component’s purpose.

Let’s take a look at a JSON page configuration.

{
    "pageName":"HomePage",
    "title": "Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more",
    "urlExtension": "www.amazon.com",
      "keyDomElementName": "searchField",
    "domElements" :
    {
        "cartCount" :
        {
            "jquery": "$('#nav-cart-count')"
        },
        "allButton" :
        {
            "jquery": "$('#nav-search-in')"
        },
        "selectCategory" :
        {
            "jquery": "$('#searchDropdownBox')"
        },
        "searchField" :
        {
            "jquery": "$('#twotabsearchtextbox')"
        },
        "go" :
        {
            "jquery": "$('.nav-submit-input')"
        }
    }
}

Let’s define the key components of a page configuration file.

That’s it!

Once the remaining pages are defined, we can run the scenario through completion. Test configurators are required to know one of the following: jQuery, XPath, or CSS. We use these to ‘select’ the web elements needed for testing. JQuery is the preferred selector mechanism for Jazz Automation, and most QA analysts can pick up jQuery in a few days, especially with all the online tutorials and modern-day web browser developer tools available.

General Configuration

The final item for discussion is the required jazz.properties file. This configuration file contains a few general configuration items such as the project name (used in reporting), a flag indicating if the tests should be run remotely or on the local system, action and page pace, and the features to be executed. There are additional optional configuration properties that have been omitted.

#project name
projectName: Amazon

# turn on remote support. if useRemote true, you must specify the following remoteWebDriverUrl property.
useRemote: false

# remote web driver url - usually your sauceLabs url
remoteWebDriverUrl: http://jazzautomation:****@ondemand.saucelabs.com:80/wd/hub

# you can customize you folder name of pages. By default it is pages
#pagesDirectoryName: pages

# page pacing between two pages loading
pagePace: 10
#action pacing between two actions on the same page. The default value is "1 second". 
actionPace: 1

# features: can be a list of features, separated by comma
features:searchForHarryPotter

Reporting

For each test, Jazz Automation produces a report that details the test run. If the tests are successful, a simple report is generated with a success rate of 100%. Upon any failure, a screen shot will be taken where the error occurred and inserted into the report. Additionally, the report is generated from a data file produced during the test run. This data file can be used for historical tracking, charting, and trending.

Let’s take a look at a failed test report:

(Click on the image to enlarge it)

Conclusion

This concludes our tour of writing automated tests using Jazz Automation. For the curious, there’s plenty more to learn. For example, Jazz Automation can also be used remotely with technology offered by companies such as Sauce Labs so that your organization does not have to maintain every possible browser and operating system combination required for full scale testing. One can also dig into how Jazz Automation works with websites that do not use jQuery. Hint: it’s dynamically inserted.

Jazz Automation completes the continuous integration loop by also testing the user interface and providing immediate feedback. This is more critical than ever as the industry moves to feature rich HTML5 and JavaScript user interfaces. Organizations need to know immediately that the application is working as expected from top to bottom. Further, the use of Gherkin as our DSL will allow key stakeholders to articulate their understanding of how the application should work and communicated effectively with technical resources, while leaving the development team free to build the product.

Jazz Automation should not be confused as a replacement for conventional unit and integration testing frameworks such as JUnit or TestNG. Unit and integration testing is still critical to successful software implementation; Jazz Automation takes it a step further and now allows automated tests to be written against the user interface. Additionally, Jazz Automation tests at the DOM level, so you can be sure that moving items around within a web page will not result in test errors.

Automated user interface testing is not a priority for every organization, but where it is, never let front-end or regression testing slow you down again.

About the Author

Dedrick Boyd is a seasoned IT professional with extensive experience across the full software development and product life cycles. He has technical skills and management experience with small and large development teams in a variety of industries including retail, pharmaceutical, media and entertainment, healthcare, and financial services. Dedrick is currently employed at Statement, where he is a technology innovator as well as a relationship manager. He holds a bachelors in Management Information Systems from Florida State University. Dedrick’s hobbies include world travel, reading, cooking, american football, and spending time with his family.