Burn-Down or Burn-Out? How to Beat the Red-Sprint Agile Anti-Pattern

Summary

There are ways to obtain sustainable pace beyond scrum that can help stem the increasing number of failing scrum projects. Because executing sprints as small projects often does not lead to the desired results, it is more effective to apply a backlog-item-oriented workflow and to treat sprints as iterations.

One of the key and often much underestimated benefits of working in agile teams, whether working on products or projects, is the idea of sustainable pace. Sustainable pace makes sure that the team retains its cool even under time pressure, which is common in software development. Those of you who have been part of agile teams will have noticed that achieving sustainable pace is not always easy. Either project management is chasing unrealistic estimates or is trying to prevent overruns or management expects ever-increasing productivity to meet a shorter time-to-market for their products.

In traditional, waterfall-oriented projects, big trouble usually remains undiscovered until late in the execution, especially when the project phases have deadlines that were set in concrete at the start of the project. With such deadlines, it is usually expected that a certain milestone be met and the product delivered at the end of each stage – whether requirements, functional design, or technical design – needs to be complete and perfectly detailed. Since each stage is traditionally only executed once, missing requirements are often seen as failures and result in change requests that are only implemented after the original project’s delivery because plans are usually so strict that there’s no room for change.

Due to striving for completeness and perfection, earlier stages in such traditional projects often take longer than expected. And when they do, there is of course less time than originally planned for building and testing the required functionality. I have seen projects where developers were working nights and even weekends to get the code out on time. Essential activities such as unit testing are too often skipped to make the deadlines, and when deadlines are really tight, managers add new developers to the team, not realizing that this only results in (temporarily) lower productivity as the new developers familiarise themselves with the team and the way of working (a principle eloquently formulated in Brooks’s law).

Worse, testing quite often totally jams as the project’s testing phases shorten from a couple of months to merely a few weeks just to make the deadline. The worst case I witnessed was a project for which the company installed field beds so that developers and testers did not have to commute, but could literally stay at their workspace to get the project done. Developers and testers worked for 14 hours per day, and ate and slept in the workplace. So much for sustainable pace in waterfall projects.

Estimating sprints

When I first read Kent Beck’s seminal work on extreme programming, his principle of a 40-hour work week initially surprised me. Although this principle is often misinterpreted to mean that everybody should work for 40 hours a week, many organizations don’t even have a 40-hour work week, opting instead for a week of 36 or even 32 hours. Beck actually meant that it should not be necessary for anybody to work more than full time. This is what nowadays is referred to as sustainable pace in agile projects. It is a pace at which you can continue to work for a long period of time, whilst working evenings and weekends will simply wear you out.

By making sustainable pace an essential value for agile projects, teams are attempting to move away from the horrible experiences of working late and on weekends. For instance, scrum has introduced the mechanism of sprints. Each sprint has a well-defined start and end that are set to allow this sustainable pace. At the start of a sprint, the team members define how much effort the next backlog items (usually represented in user stories) will require, and only plan to do as much work as they can handle. When estimating and planning well, the team should be able to deliver what it has discussed during the next two or four-week sprint. And so the team commits to deliver exactly this amount of work during that sprint.  

What most teams do to achieve a proper estimate is to break the backlog items down into smaller items (usually called tasks) that individuals from the team can handle. Next, the team estimates in hours the amount of effort required to do the task. The team adds up these hours and matches them to the work hours available during the upcoming sprint. And so, the team commits to deliver the tasks, and hence the stories. It sounds like a decent and easy-to-implement procedure, right?

Failing in small projects

Using this approach, every sprint becomes a small two or four-week project with a well-defined scope, a clear beginning, and a fixed end date (most sprints have a fixed period). However, this approach is not as easy as it looks. In real life, many teams struggle to deliver these small projects. I have witnessed many teams who were not able to deliver what they had agreed to at the start of their sprint.

Recently, I reviewed a typical scrum project that was delivering a product in 15 four-week sprints. I interviewed the three teams at the end of their 12th sprint. The teams were badly motivated and complained about the heavy workload. A number of people had left the project because of these complaints. As most scrum teams monitor sprint progress in burn-down charts, the first things I asked the teams for were their charts.

Most of the 12 burn-down charts showed the same pattern. At the start of each sprint, the teams had estimated the amount of work that they thought they could finish, as can be seen from the starting peaks on the charts. From that point forward, the burn-downs progressed slowly and bumpily down towards the end of the time box. However, in none of the 12 charts that I investigated was the allotted work finished by the time the sprint ended. All had work remaining, as the example below clearly shows.

Even more interestingly, a number of the burn-down charts showed that work was added to the sprint backlog almost immediately after the first stories were estimated, as if the teams realized that their original estimate was way off and that they could handle even more work during the sprint. From the perspective that considers each sprint a small project by itself, as scrum normally interprets a sprint, you might say that the teams had 12 small, failing projects. None of these projects delivered the anticipated functionality on time. No wonder that the teams were unmotivated.

Red sprints

At another company that also used scrum for product development, I witnessed the same pattern. Here, teams failed to meet deadlines during a number of consecutive two-week sprints. Again, at the end of each sprint, the teams still had work left. One of these teams even had a name for sprints that do not end right. They called them “red sprints” and, you could guess, they referred to “green sprints” if they made the self-imposed deadline. When I first talked with these teams, they had had six red sprints in a row. Go figure. The team was lagging behind more and more, and had been working overtime for four of these six sprints.

The product manager at this company produced an interesting chart. As the teams worked towards a new version of their products, he had created a release burn-down chart. And as the teams continuously had red sprints, they more and more lagged behind the velocity they needed to be able to release the new versions with all the functionality they had agreed to at the start of the release. Eventually, the product manager calculated that to meet the deadline, work velocity in the last three scheduled two-week sprints needed to be three times the velocity at which the teams had been able to work during the first nine. Optimistically, he referred to the number of points the teams needed to deliver as the “expected average” when in reality it was an average to pray for. Based on his calculation, the product manager pushed his teams to work harder, to create fewer bugs, and to work overtime. Mission impossible, I’d say.

As you can imagine, motivation in his teams had dropped to zero. To me, the red-sprint anti-pattern sounds much more like burn-out than burn-down.

No sustainable pace

An interesting question arising from this agile anti-pattern is where does it come from? And more, how can you avoid red sprints? There are a number of reasons why such red sprints occur:

•  The golden triangle revisited. First of all, it all starts with considering sprints to be small projects, with a fixed scope, a fixed team, and a fixed deadline. Cementing all three endpoints of this software-development golden triangle causes the same problems as in traditional, linear projects, only on a smaller, repetitive scale. Instead of having a deadline per phase as in traditional projects, teams now face a series of self-imposed deadlines, each of which are hard to make. Even though I realize that working in sprints puts some desired pressure on teams, the deadlines the small projects seem to impose potentially cause too much stress on the people in teams.
•  Estimation is still hard. At the start of each sprint, teams spend a reasonable amount of time breaking down the assumed mandatory user stories into tasks and trying to accurately estimate the time needed for each of these tasks. Teams need to be thorough and precise in these estimates as they commit to deliver the amount of work agreed upon in these estimates. Incorrectly estimating the tasks means that it will be hard to predict whether or not the team can get the work done.
•  Estimation in hours is threatening. Most teams, traditional and agile, estimate the amount of work at the lowest level of granularity, in this case tasks, by the number of hours it will take to complete the tasks. In my opinion, estimation in hours is dangerous, as it always threatens those who create the estimate. How do you know it will take you eight hours to implement a particular web page? What if during the sprint someone else picks up this task who is either more experienced in building web pages or, worse, has far less experience in it? And what if you can’t implement the page in eight hours, but need 12 hours? What will your project manager have to say about that? The bottom line is that you have to feel quite confident with your estimates. In practice, estimation in hours is a tough cookie.
•  Stop extensive grooming. Because of the above, and similar to traditional projects, teams take a lot of time to break down or groom backlog items – user stories – into tasks and to estimate the effort each task will take, just to feel confident. Moreover, breaking down backlog items into tasks is usually quite complex as, especially with user stories, there is no standardized way to split them up. User stories are a fairly unstructured way to describe backlog items. Basically, the breakdown is different for every story. I have seen teams spend a full day on “backlog grooming”, “story time”, or “backlog-refinement meetings” for a two-week sprint cycle. That is over 10% of their total working time.
•  The unexpected. And it gets worse. What happens if something unexpected occurs? Think of changing requirements, technical issues, or something as ordinary as a server that cannot be reached or a developer who stays home with the flu. The effect these have is similar to that in waterfall projects, albeit smaller. The next tasks get delayed or even postponed, resulting in missing the sprint’s deadlines. There’s another red sprint for you.
•  Parkinson’s law. Even if a team is able to estimate accurately, does the time needed to implement the individual tasks exactly add up to the team’s available man-hours for the sprint? In most cases, it will not, resulting in slack. Usually, some slack is good, but always remember that Parkinson’s law will kick in: work at hand will fill the time available. This will happens in sprints, too, if the amount of time necessary to complete the selected backlog items is just a tad less than the time available. In sprints, this results in a slower pace than estimated (or the expected average from the product manager in the example above) because more time is spent than is required to implement the sprint’s tasks. If this pattern occurs, it usually becomes visible when developers spend time beautifying their code.
•  Cutting down on quality. Last but not least is the other way around: what happens if the team is squeezed? The team still needs to finalize some tasks but does not have enough time remaining to complete them. This is usually leads to reduced quality as teams often decide to skip some of the work necessary to implement their tasks – writing unit tests, for instance.

So this is where red-sprint anti-patterns come from. And for sure your teams will not work at a reasonable sustainable pace, either.

From sprints to iterations

So, back to the drawing board: could you have avoided this agile anti-pattern? Please allow me to rephrase the question: do teams have an alternative if the standard sprint procedure fails? I strongly believe they do, after having worked with many teams that moved away from the traditional sprint model I described above.

As it is hard to plan the exact amount of work, even for a short two or four-week period, and to complete it all before the period elapses, why not move to a more backlog-item-oriented approach instead of a task-oriented approach? The realization of individual backlog items becomes more important in this sort of approach than successfully finishing one small project after another. Green sprints or red sprints become much less relevant, as it is less important to complete your backlog items within the scope of a single sprint. Work on a backlog item simply continues into the next sprint if it isn’t done yet, or the item even returns to the backlog if it’s unclear that it can even be completed. How would this work better?

I’ll break it down into bullet points for you:

• No breakdown. When you decide that is less important to know exactly whether a bunch of backlog items will exactly fit your sprint, there is less need to break down backlog items into tasks – or there is even no need to do this at all. You just work on implementing backlog items.
• Follow the life cycle. This is even better. In my experience, backlog items that are similar and equal in granularity can follow a similar life cycle when being realized. In software development, this is nothing new; we do similar work on most of our backlog items anyway. Think of the usual activities such as analysis, design, test design, build, developer test, test, and accept – at least in my projects, this is common sense.
• Visualize the life cycle. Tracking a backlog item’s life cycle is much easier than having to create a collection of individual tasks such as “Build X“, “Write unit tests for X“, or “Test X“. It saves a lot of work and, moreover, allows you to visualize your progress more clearly than you could on a traditional scrum board with only To Do, In Progress, and Done categories. Most scrum boards I see in projects have a big spread of sticky notes in the In Progress column, but usually nobody on the project can precisely tell me what “in progress” actually means for each of the tasks.
  Once you’ve identified a life cycle for your backlog items, you can also track their progress through the different life-cycle activities – a typical kanban procedure. There’s now only a single sticky note for each backlog item, and this note progresses through the different steps in the life cycle from left to right. As an example, my teams often visualize the backlog-item life cycle using tools such as SpeedBird9 (see screenshot), Trello, or Mingle.

(Click on the image to enlarge it)

•  Optimize the life cycle. It gets even better. Once you’ve moved to visualizing the life cycle, you will be able to more quickly identify bottlenecks. These will simply be the columns that have the most sticky notes: the activity in which the notes accumulate takes more time than other activities. You can now tweak and tune your life cycle to remove the bottleneck.
  Here’s a straightforward example of such life-cycle optimization. During one of my projects, the team was developing backlog items at a good pace but sticky notes began to pile up at the Test activity. The simple reason was that the tester’s wife had just given birth, so our tester had been away for a couple of days. When he got back, he had a nasty surprise of a dozen notes to pick up. Of course, our developers immediately stopped producing even more code and instead spent time helping to test our backlog items. Remember, a chain is always as weak as it weakest link. The activity in the life cycle that takes up the most effort sets the pace for delivering backlog items.
•  Hours don’t matter. When you stop breaking up your backlog items into tasks, there is no further need to estimate tasks in hours. A point scale that expresses the size or complexity of your backlog items will do. Knowing the average number of points that a team realizes during a sprint is enough to plan the next sprint. You simply choose the number of points worth of backlog items to match the average number of points your team has so far completed per the relevant time and you are on your way again. You can plan the next sprint in half an hour instead of half a day.
•  No mini-deadlines. I guess this last bullet is the main point. If you are able to implement these guidelines, you can move away from the continuous pressure of having to run the 100-metre dash every sprint. Although sprints give you a nice cadence and allow you to reflect on your work on a regular basis, you don’t have to panic when you deliver “only” 10 out of the 12 backlog items you hoped to produce.

Over the past decade, I’ve seen so many projects struggle with the self-inflicted small-project sprint model that it’s hard to imagine that this model will hold. I recommend moving away from this traditional sprint model because it is hard to estimate exactly how much work you can do, even over a two or four-week period. And it’s not worth the effort of breaking down items in even smaller items. I’ve successfully implemented the less restrictive model of iterations many times as a follow-up to sprints, without breaking down backlog items but simply sticking to the backlog-items point scale and realizing that you can’t fit work to iterations to deliver something shippable. There are too many projects with red sprints and disillusioned teams. Even Usain Bolt can’t run 12 consecutive 100-metre sprints at the same pace.

About the Author

Sander Hoogendoorn is the author of the best-selling book This Is Agile and is an appreciated dad, programmer, software architect, speaker, and writer. As principal technology officer and global agile thought-leader at Capgemini, Sander innovates and coaches software development at Capgemini and its many international clients. He has published more than 250 articles in international magazines and is an inspiring (keynote) speaker at many international conferences.Twitter: @aahoogendoorn.