What should the next generation of functional testing tools offer? Should they serve as readable documentation of requirements? Should they come with an advanced test editor that supports test completion for user-interface elements as well as test code refactoring and analysis? Or an editor that supports multiple representations, like wireframes and state machines? Visualize the test results? Suggest and generate tests based on herustics?
The Agile Alliance is holding a workshop to envison the next-generation of functional testing tools, from October 11th to 12th, 2007 in Portland, Oregon.
The workshop background describes the context for this initiative by describing how far functional testing has come and how far it has yet to go:
The good news is that tool support for automated functional tests has grown significantly in recent years. There is a large variety of commercial and open source testing tools/frameworks available that support Agile development practices. The FIT framework was a significant boost to the state of the art of automated functional testing, both in terms of the syntax of the specification (tables), the detailed test execution feedback (cell by cell), and the development/execution environment (desktop tools rather than development or specialized tools).
However, we believe that it's time for another significant boost to the state of the art.
We are lacking integrated development environments that facilitate things like: refactoring test elements, command completion, incremental syntax validation (based on the domain specific test language), keyboard navigation into the supporting framework code, debugging, etc.
We need more expressive test specification languages, possibly integrating executable: text, tables, shapes, and colors together into a single test.
We need specification languages that can describe user interaction in a readable and maintainable fashion.
We need to be able to view/navigate the tests in multiple different ways in order to see how the pieces of the puzzle contribute to the bigger picture of the domain/feature: organize tests based on their domain context; search for tests based on user-defined keywords (cross cutting concerns).
and things that we haven't even thought of.
This event is the first step of the Agile Alliance's Functional Testing Program, headed up by Jeannitta Andrea with Ron Jeffries and Elisabeth Hendrickson. Jeanitta's been writing about next-generation functional testing tools for some time now, most recently in Envisioning the Next-Generation of Functional Testing Tools
So what do you need and want from your functional testing tool? What might the participants say at the workshop? What tools are you using now, and where do they work, where do they break down?
We'll stay in touch with this program as it progresses, and keep you informed here at InfoQ. You can use the Customers and Requirements tag to see more on this topic.