Scrum is an iterative and incremental framework that embodies an agile approach to software development. It has become popular among software organizations for software application development and testing.
Testing in the Scrum team is as crucial as development. Features need to be tested in specified, short timeframes within each sprint for testers to help in removing bugs as early as possible.
While agile testing has many advantages over traditional ways of testing, among its shortcomings is that it sometimes puts excessive pressure on the quality-assurance (QA) team at the end of a sprint, which can result in sprint spillover.
Another problem that testers face is a lack of comprehensive documentation. A major pitfall in agile projects is a lack of emphasis on design and documentation, due to which many requirements remain vague or ambiguous. Too much detailed documentation is said to hinder the important work but I think this problem can be overcome by maintaining for every story its proper details, documentation, and every possible scenario in an agile project-management tool.
QA teams cannot plan more than a couple of weeks ahead. In agile projects, testers are required to test code in the same iteration in which it is developed, and they are expected to provide fast feedback on the code and application. Most of the time, however, working code is available only towards the end of the sprint, when code is often frozen before a demo or
showcase. As a result, testing teams usually lack enough time for verification, so that testing for a particular sprint is deferred until the next iteration when work will be thrown over the wall to the test team.
The job of a tester in Scrum is not just to test and to log bugs in defect-tracking tool. It includes multiple tasks like test management and analysis along with testing responsibilities. There are also responsibilities from client handling to bug tracking to incorporating the frequent changes the client keeps suggesting.
The true agile QA is also often responsible for non-unit-test tools, test environments, and test data. People in this role will find themselves weighing conflicting choices. The choices resemble those in non-agile projects, but the short timescales of an agile project make the problems particularly acute. The responsibility for test management is often delegated to one or two members of an agile team, rather than taken on by the team as a whole.
Although working in agile keeps you on your toes, distributed responsibilities and better time management makes your work easier as well as efficient.
Estimations also challenge agile testers. Accurate test estimation considers important factors like size of the project, types of testing required, testing tasks, and past experience, but sometimes even the best estimate falls short as a lack of time at the end of sprint prevents QA from performing end-to-end testing. Any delay in the prior development risks impacting QA timelines, sometimes preventing QA from executing a test case for the whole iteration and leaving it to struggle to finish. While estimating, QA has a duty to remind the team of testing tasks to be performed so that team members do not under-commit to the task. Estimates should include time for both manual and automatic tasks that the team may need to write or modify for a story.
Another snag in agile testing is testing without the customers, who may think that their involvement is only appreciated when the product has been finished. This leads to problems with acceptance tests as well as acceptance criteria. We generally don’t get feedback regarding what has to be build next during a showcase. Building a trusting relationship can mitigate this risk.
During my last project, I saw clients propose huge changes in the core functionalities of the application. Changes like these can impact other features of the application and result in code changes and double the testing effort. Late feedback from a client can delay production. Having business people dedicated to daily customer interaction would solve this gap in client response time.
One of agile’s major advantages is testing as early as possible. As a project matures, agile testing becomes more important. Each feature should be fully tested as it is developed, rather than at the end of development.
After a few successful iterations in the early phase of project, stories and work increase and the project requires more team members. With an increase in the number of developers, the number of testers should also grow in order to maintain a constant tester/developer ratio (usually one tester for every two developers).
Now, assume the above situation repeats every sprint (about every two to four weeks). Agile testing needs to validate one or more new software modules, from the customer’s perspective, during each cycle. It also needs consider how and when to handle regression before the eventual release. Testing stops being a phase; rather, it blends with development, and continuous testing becomes the mantra and the only way to ensure continuous progress and eventual success.
Agile testing checks each new functionality as a sprint progresses. Typically, testing reserves a small window toward the end of a sprint for a short regression test before moving to the next sprint. Often, agile teams implement a BVT (build verification testing) routine in which they perform a standard set of verification steps, cutting across the application, to ensure application stability and function. If possible, this routine could be automated and integrated as part of the CI server to make the release even more stringent.
On projects running over several sprints, it’s standard practice to have a code-hardening or release sprint to ensure the overall functionality of the application, mostly from an integration point of view. Ideally, this should not stretch past 30 to 45 days, assuming enough care was taken during the individual sprints to cover defects. This can be accomplished by setting manual and automated testing fields for each story and bug. It should be responsibility of QAs to mark as manual the field for any story/bug that is not automated. As a result, we get filtered list of regression tests that we can execute manually whenever new build is deployed. On the automatic front, we should have good suite of automated tests that run as part of the CI team for every check-in made by developers.
Sprint after sprint, we add new features or change existing ones. We also must ensure that what we’ve built previously continues to work. An automated testing framework helps the team to quickly test and uncover bugs. It not only acts as a safety net against regressions caused by new development, but more importantly frees up a lot of precious developer and tester time - allowing them to focus on what they do best.
But the time-box limitations of a Scrum sprint raise challenges to simultaneously writing automated test cases and manually testing. To overcome this, our team added one more stipulation to a story’s definition of done: a story is not ready for testing until its happy path is not automated. Usually, it is a good practice to have one developer and one QA tester collaborate to write a happy path.
Sometimes, it may not be possible to test non-functional aspects, such as system performance, within a sprint. Separate stories and estimates should be created for each and every test of non-functional aspect. Additionally, these tests should be automated and added to the regression suite to ensure that the system continues to work after any defect fixes. Rigorous integration testing may not be required if the system was continuously integrated and tested by leveraging automation.
Although the task of testing does not differ in principle among waterfall, iterative, and agile practice, the agile mindset and its testing practices provide new, effective means to achieve the desired results. The agility lies in the agile practices rather than in the overarching process itself.
In a nutshell, a good agile tester needs to be talented at multitasking and able to cope up with speed of development/release. For a tester, being creative is more important than being critical. A testing professional should strive to learn and innovate, and must have a comprehensive vision of the client’s expectations. Finally, an agile tester must have multiple task skills like manual testing, functional testing, and performance testing as well as soft skills like leadership and communication.
About the Author
Priyanka Hasija is a QA Consultant at Thoughtworks having worked in the IT industry for the past 5 years. During this time she has developed a sound understanding of Agile principles and has had the opportunity to practice them successfully on several Agile projects. Priyanka has gained good experience performing manual test as well as working with automated testing tools such as Cucumber , Web-driver, JMeter. She has also been speaker at various internal and external events.