It is always a herculean task to write automated regression tests for a legacy application. The questions vary from where to start, how much to automate and deciding on the best strategy for automation.
Responding to a thread started by Mark Levison on the Agile software testing group, Hubert Matthews suggested that the approach that he recommends is a risk based approach.
You can never test everything so you have to choose where you want to spend your time and money. Testing for me is primarily about information and risk and not about blanket coverage or completeness.
Hubert mentioned questions which help evaluate the areas to focus on. Some of his suggested questions include,
- Where is the client currently experiencing the poorest quality?
- What are the key areas of functionality (e.g. what makes them money)?
- Would the client prefer more features to better quality?
- What are the biggest risks to their systems?
- If the client could improve one thing what would it be?
- Would exploratory manual testing find more important bugs?
Rakesh Patel suggested a similar approach when he mentioned,
- Only automate through the GUI the most important paths through your app. A lot of apps have some common path that is used in 80% of cases. If this stops working, your business is in trouble!!
- If you can bypass the GUI and go straight to the backend to test business functionality, do that. That means GUI-specific integration tests are about making sure the data from the front-end get to the back end.
Mark Fink suggested that before starting on automation for a legacy project, he likes to get a big picture of the project to identify the specific areas which need attention. He suggested a list of tools which are helpful in getting to the big picture. Likewise, Nat suggested that the key is to build end to end tests for the area that you want to focus on. He recommended that for a legacy system, if there are manual tests, then usually those are scripted well enough to become low hanging fruit for automation.
Ralph Bohnet and Gerard Meszaros talked about test driven porting and one of their conclusions was that for any legacy application, the most critical business scenarios should have automated regression tests if the porting has to be successful.
Lisa Crispin agreed that there is a little chance to automate the entire legacy application in one go. What worked for Lisa was a combination of factors. Some of which were,
- Ask the customers to prioritize the critical areas of the app
- Write manual regression test scripts for each area
- Budget time in each sprint to write GUI smoke tests for those areas
- Execute the test suite at least once a day with the CI framework
- All new development should have adequate tests
According to Lisa, with this approach, they could adequately automate their legacy application within a period of 8 months.
Thus, it might be a far call and a time consuming affair to get the entire legacy application covered with automated regression tests. The suggested approach is to build enough coverage for the business critical functionality and gradually build a harness around the system.
Related post on InfoQ : Testing Techniques for Applications With Zero Tests
