The Scrum world has changed a lot in the last few years. Nowhere is that more true than estimating; Scrum folks have struggled with the practice since the framework's origin. Other folks also struggle with it, but this article focuses on Scrum. Many don't see the value in estimating. The 2020 Scrum Guide replaced the word estimate with size or sizing.  Some people often forget that estimates are, after all, just estimates.

Community opinion on the whole area of sizing and forecasting is fraught. My intent is not to add fuel to the fire but to add some thoughts to further the discussion. In my view, the Scrum world needs an article like this. It is a theoretical piece assuming the reader has a reasonable amount of  Scrum knowledge, informed by practice. I hope you find it helpful. 

If your forecasts are routinely correct, you're a freak of nature. Forecasting is rarely perfect due to the following:

• Waiting time due to dependencies is a huge factor in how long work takes and is affected by many unpredictable events.
• Even in straightforward work environments, people overestimate how efficiently their day will go.
• Often, people doing complex work in the pursuit of speed leave work behind them that is untidy and potentially embarrassing (accidental complication). Even worse, people compound the untidiness of the work in follow-up work, which takes longer than intended whether the untidiness is removed. Some call it "broken window syndrome."
• Complex work involves many unknown variables. 
• Lack of focus, e.g., "have you got 10 seconds" or "throwing in the kitchen sink" due to prolonged release cycles or over-planning (too much work) around the flow bottleneck.
• Changing priorities, and the lack of updated sizing based on the changed priorities, e.g., after a problematic forecasting session where it's clear that the odds are against us, another priority is thrown into the mix.

It's often assumed that forecasts are requests to tell with some confidence, "when will it be done?" Often there is another question behind the question, such as:

• How can I transfer worry to someone else?
• What progress is being made?
• What risks remain?
• When will we get some return on this investment?
• What trade-offs can we tolerate regarding which work can discover/deliver the potential value, e.g., the 80:20 rule?
• What trade-offs can we tolerate in terms of reducing some or all of effectiveness, efficiency, and predictability, e.g., running some experiments?
• What progress trade-offs can we tolerate in terms of required "dead work" to avoid execution bias, such as laboratory setup?
• How much investment will go into acquiring skills, e.g., education or apprenticeship?

A meta-question of "what does winning the game mean?" is well worth considering. Is the team being given a game it can win? And if the team can win, what are the odds? Probabilistic forecasts can help, e.g., Monte-Carlo simulations.

Despite the hazards, sometimes teams still feel the need to forecast because people fear that stakeholders will make up arbitrarily fixed undoable dates in a vacuum. Sometimes teams want to attain a ballpark date range to get ahead of stakeholder expectations. Interestingly, most of us can accept a weather forecast that gets updated regularly based on the latest information, even if we know it's still inaccurate. 

Monte Carlo simulations

Troy Magennis does an excellent job of explaining Monte Carlo simulations; simply put; they allow people to model a future based on data and assumptions. Probabilistic forecasts can tell you how much work you can get done, not whether the work at hand fits that forecast. Although probabilistic forecasts are an illusion for complex work, one can redo them regularly as new information comes to light, making them somewhat more reliable over time. 

• Monte Carlo When forecasts play out scenarios using 500 to 1 million random number generations for throughput (delivery rate of valuable work items per time period, let's say per Sprint) within min/max limits per time period (real data or 90% confident guesstimates) combined with random number generations for the number of items within a range; each forecast returns a range of dates with associated probabilities. For example, in the results shown below, after 10k simulations, 50-75 items have an 85th percentile probability of delivery on 22nd November or sooner.

Screenshots of TWiG and ActionableAgile

• Monte Carlo probabilistic How Many forecasts use random number generations for throughput within limits per time period to forecast how many items would be done for given dates; the output is a range of dates with a number of items done by that date plus a probability percentage. For example, below is an 85th percentile number of items that would be delivered by date or sooner.

Screenshots of TWiG and ActionableAgile

In either case, using the wrong limits results in a low-quality forecast. For example, let's say we used a seven-sided die of 1, 2, 3, 4, 5, 6, 7 when throughput has only ever and is only likely to be 3, 4, 5, 6, or 7. We've allowed the possibility of 1 or 2 when that should likely be outside the limits. That said, 1, 2, 8, or 9 are still possible, just not probable. Prateek Singh goes into a lot more detail about how random the number generation would be using various models. Also, if throughput is erratic or mostly zero per time period, the quality of a Monte Carlo simulation reduces considerably. Forecasting, at its essence, is about risk management. It answers the question - How much risk is contained in our current plans? Lower quality forecasts also mean inadequate risk management.

Communities are not aligned on this approach. One project is only executed once. While probabilities may help inform decisions, the problem is that they don't make the decision any easier. Estimation is often used as a proxy for a decision (do we do this project or not?). In that regard, does probabilistic forecasting offer any answer, or does it only offer more questions? They are important questions, but the reasons for using estimates differ from probabilistic forecasting. I have seen many probabilistic forecasts based on guesstimates and a lack of history, yet they were not far off in the end. If you'd like to dig more into that, check out when I spoke with Troy Magennis about it.

Let's examine in more detail the options for sizing and forecasting to help inform the lesser evils in a Scrum context.

Sizing Techniques

When using Scrum, the Scrum Team self-manages, and the Developers carry out sizing because they are the people closest to the work.

The most popular sizing techniques are either based on data or educated guesses. When dealing with complexity, know that these techniques are almost always inaccurate. 

I find that most people are poor at estimating. Still, people perceive value in the conversation because assumptions can surface while we come to a common understanding of the work and its challenges. Understanding 'the work item' and 'Estimating the work item' are separate activities that should not be conflated. My view is that estimates for individual product backlog items don't provide a solid foundation for forecasting the time it will take to complete the work.

The most popular sizing techniques using estimation for forecasting are as follows:

• Relative estimation 
  • Time reference - Comparing current work items to the time it took to complete historical reference items.
  • Assigning numeric values - Examples include using story points based on the Fibonacci sequence and often carried out collaboratively with playing cards (planning poker). 
  • T-shirt sizing - Assigning s, s/m, m, m/l, xl, xxl, xxxl, xxxxl to Product Backlog Items instead of numeric value, sometimes simplified by leaving out every second size without losing scale S=1, M=3, L=8, XXL=20 (rounded up), XXXXL=60,    rounded up), ?=100 (rounded up)
  • Wall estimation - Assigning numeric values by collaboratively placing and moving cards on a wall, also referred to as magic estimation or silent estimation.
• Flow metrics or counting items
  • For a goal or time period - Identify/count (as best you can) a number or range of Product Backlog items to get to Done.
• Right-sizing - Identify small enough items for intake — often using a flow-based approach — it's less about whether an item is small or medium. The practice involves figuring out if the team can complete a Product Backlog Item according to the Definition of Done comfortably within one Sprint.
• #NoEstimates - one interpretation being …
  • Strive for an even distribution of "ballpark" item sizes throughout a backlog.
  • Count running tested stories or running tested features to demonstrate progress in output terms.
  • Focus on simplifying the what for the why - a focus on desired outcomes.
  • Right-sizing - Identify small enough items for intake.
  • Slicing into 24-hour timeboxing of items encourages the creation of experiments that validate assumptions/hypotheses towards a goal, discover to deliver.
  • Use rolling-wave forecasts to communicate uncertainty.

Some would argue that counting Product Backlog Items without throughput data is also an estimation, and I agree with that view. Counting based on the throughput of done items is forecasting.

Of the above sizing approaches, only wall estimation considers relative potential value directly. It uses value points for estimating value (not effort). Value is not absolute; it is relative. Customer value gets realized after a release to the customer, and it might be a lot higher/lower than we expect. 

All of the above sizing options are devalued by:

• Not having caveats associated with the start date, e.g., nine weeks from the date we start.
• Not recognizing the amount of work in progress and the progress (or not) of that work.
• The severity of impediments.
• Not ordering items higher up the Product Backlog according to delivery risk.
• A sub-optimal approach to handling dependencies.
• Confusing outputs with outcomes; a customer/end-user outcome is a change in customer/end-user behavior.
• Not engaging in discovery activities when the risk of not harvesting potential value is high, compounded by assuming that every item moves from discovery to delivery.
• Delusions of accuracy and pursuing more accuracy.

Suboptimal trends include

• Size per skill - typically caused by focus on resource efficiency over flow efficiency
• Size inflation - typically caused by pressure for more "velocity.” No prizes for guessing which size a team might use for a borderline case, out of higher or lower; they're likely to pick higher. In extreme cases, I refer to this as (sizing) bingo.
• Not taking quality seriously, e.g., lack of conformity with the definition of done - typically caused by pressure for more "velocity." Make no mistake - quality via the Definition of Done commitment is a big deal.
• Not taking the customer seriously, e.g., delivering outputs without measuring if they made a difference - typically caused by an excess of "thinking inside the building" or distance from the customer.
• Size normalization across teams is typically caused by the assumption that people are replaceable machine parts. In my experience, given many situations with the same product backlog, the same reference items for comparison, and the same scale, teams have never come up with the same sizes. 
• Counting complete but fake product backlog items, items that don't deliver value, as throughput - typically caused by the use of work breakdown structures that lose sight of value
• Not focusing, not finishing. Focus is a big deal in Scrum primarily via the Product Goal in the Product Backlog, the Sprint Goal in the Sprint Backlog, and the Scrum values. And how about focusing on finishing what we started, even if it means splitting or canceling work? 
• Delusions of predictability for work that is uncomparable with work from the past 
• Lack of discovery to find the items we maybe should not build, some say 70%. If we run low-cost experiments, we might fall upon better ideas. For example, insofar as I can tell when Bose discovered noise-canceling headphones, they were looking for something else, higher sound fidelity. Lack of discovery is typically caused by leaders committing to work on behalf of teams; the view becomes "there is no point in finding better ways, we have a deadline.” 

This article is not designed to help people sustain such trends but to shed light on better ways. If you want help getting better at estimation with story points, you're looking at the wrong article. 

The main sizing options this article draws attention to are estimation, right-sizing, and #NoEstimates. Neither of those options is perfect. I've included some other options to consider, but regardless of the approach, the level of exactness is low. 

• About estimation:
  • It comes in different flavors.
  • If you estimate, the best thing that can happen is the estimates are correct. How long things take has little to do with the processing time - see an article by Troy Magennis on this topic. 
  • Estimates are prone to the "flaw of averages" (Sam Savage). Is 50:50 an excellent way to set expectations? 
  • The average of independent blind assessments can be near enough to the truth (Dave Snowden on Xagility podcast). Estimates are rarely blind, though. 
  • If you don't estimate at all, you don't waste time; hopefully, you will discover/deliver outcomes sooner.
     
• About right-sizing:
  • How much time could you save caring more about whether the team can complete an item within the Sprint and less about making that item infinitely smaller?
  • Think of the reduced cognitive load on the Product Owner resulting from fewer PBIs.
  • Counting the number of (valuable right-sized) PBIs delivered to Done per Sprint is valuable for Sprint Planning and forecasting goals.
  • If throughput is sporadic or irregular, we have more significant problems than forecasting; we have a "plumbing problem."
  • Using average throughput also pursues the "flaw of averages." Monte Carlo probabilistic forecasting is preferable.
     
• About #NoEstimates, one interpretation is:
  • Counting the number of (valuable right-sized) PBIs delivered to Done per Sprint is valuable for Sprint Planning and forecasting goals.
  • If throughput is sporadic or irregular, we have more significant problems than forecasting; we have a "plumbing problem."
  • "Rolling Wave Forecast” based on throughput with variance limits is preferable.

Whether one uses estimation, right-sizing, or #NoEstimates, sometimes it's difficult to break down Product Backlog Items (PBIs), so they're still valuable (and not just subtasks).  There is a variety of guidance on how to split Product Backlog items.  Example mapping is a powerful approach; the Scrum Team can pick just one example for this Sprint and get feedback. When evidence (we can harvest value) is lacking even if the potential value is high, it’s sensible to experiment to validate assumptions through hypotheses; #NoEstimates and Scrum with UX support this approach. Splitting can result in discovery or discover-to-deliver. For example, customer interviews and UX research might be triggered by splitting.

Potential upsides of some sizing approaches

Potential downsides of some sizing approaches

Other considerations for sizing approaches

Forecasting

When forecasting effort, the most popular methods tend to be based on:

• Past performance of the number/range of Product Backlog Items (PBIs) to Done, based on averages or a range – past performance does not predict the future, but it is acceptable for a short period such as a Sprint.

Screenshots of TWiG and ActionableAgile - This team is delivering stuff, but they deliver nothing to Done for many days.

• Past performance of the number/range of numerical values or PBIs to Done
  • Past performance does not predict the future, but it is usually acceptable for a short period, such as a Sprint.
  • Beware of using arithmetic on numeric values, as complexity rises exponentially with Product Backlog Item size, and putting an "elephant" in a Sprint is problematic.
  • Beware of the number of PBIs already in progress.
  • Sometimes based on a burnup/burndown chart using "exact"/relative sizing of remaining work over time.

Burnup/burndown charts are based on averages and often misinterpreted.       

• Probabilistic forecast with a percentage probability that a number (or range) of items with varying probabilities for a range of dates; can be used for short-, medium- and long-term forecasts. 
  • Consider using a parametric scale or random sampling to model based on previous work. For example, you might think: this investment feels like two other similar past investments, but we'll need to add 15 items. 
  • Consider also a number or range of items for pulling everything together as work doesn't always knit tightly and easily in the end – but isn't that risk well-managed through iterative, incremental, and continuous delivery anyway?

Screenshots of TWiG and ActionableAgile

• Probabilistic forecast for how many PBIs can be completed by a specific time - with a percentage probability, a date, and a number (or range of numbers) of items by that date; it can be used for short-, medium- and long-term forecasts, but it's more useful for output orientation than goal orientation. 

Screenshots of TWiG and ActionableAgile

• A rolling wave forecast demonstrates the art of the possible and uncertainty at the same time

• A statement something like, "We're using an empirical approach operating one Sprint at a time; the Sprint Goal is not even a guarantee; the real answer is we don't know, but let's start and learn quickly." You might not even use Now?, Next ??, Later ???.

The most offensive part of forecasting is that it often does not consider how many PBIs we're working on simultaneously. Alas, all too often I have also seen teams of teams bringing in more items than their usual capacity.

As mentioned, teams can use different approaches to size the effort to deliver a Sprint/Product Goal. The forecast will almost certainly be wrong, so they should convey uncertainty, reminding stakeholders that they're using an empirical approach, moving one Sprint at a time, and refreshing forecasts frequently. The Product Backlog is (hopefully) a living and breathing emerging artifact.

Some would say it's better not to provide a forecast at all. They would say, "Discover and deliver capabilities—review outcomes with the customers and end-users. Learn what can be learned. Act on what we have discovered. Don't manage expectations."

Takeaways

1. Avoid story points. If you must use story points, use them as a temporary crutch. The story points approach is a popular complementary approach for teams, but it is not part of Scrum. I am not a fan. When people use story points, I usually observe destructive behaviors. I see story point inflation, story point bingo, story point normalization, and story point velocity used as a proxy for performance measurement. For example, senior leaders look for more velocity leading team members to split items that are not Done to claim some kind of story point velocity. Teams are not wholly in control of how much work they can deliver. So, if a team is under pressure for more velocity, the chances of a medium item being defined as large or extra-large increase. 

2. Avoid average throughput. The throughput approach is better, and while it's not perfect, it's a more accurate basis for forecasting when used with probabilistic forecasting and counting valuable items within throughput. Throughput lacks credibility without value or right-sizing. Though not part of Scrum and only optional in Kanban Guide, for non-software, consider separate throughput for separate product backlog item types, e.g., packaging vs. communications. Using averages to forecast time/points/throughput is not a great way to manage one's career. "Boss, we have a 50/50 probability of on-time delivery; is that ok?"

3. Consider historical reference items. Historical reference items include "waiting time" and depend on remembering how long that took but don't consider accidental complication. Accidental complication is the untidiness of the work we leave behind and its knock-on effect on the quality of estimates; it explains why reference items still don't work well for time estimation. 

4. Try throughput of valuable items as a basis for probabilistic forecasting. For non-software, consider separate throughput for separate product backlog item types, e.g., packaging vs. communications. It's still "smoke and mirrors" for complex work, so you'd best add the caveat that you'll provide a better forecast next week and the week after that once you learn more. Beware of probabilistic forecasting based on fake throughput (throughput of fake items - items lacking value), sporadic throughput, and particularly when most time periods selected have zero throughput—probabilistic forecasts based on erratic throughput lower the quality of forecasts considerably.

5. Try #NoEstimates - rolling wave forecasts communicate uncertainty well. To avoid breaking Scrum or Kanban(for example, using the Kanban Guide for Scrum Teams), ensure that all (product backlog) items are valuable.

6. When asked when the team will have something done, try responding by saying you don't know. Explain that you're using an evidence-based empirical approach and operate one Sprint/week/month at a time. Many items require discovery/spikes; you don't know what feedback you'll get, you don't know what you'll learn, and you don't know what you'll have to respond to. Beware that in many cultures, vacuums tend to get filled, and if one does not provide a forecast, others eagerly await to offer one, probably with an undoable deadline.

7. Favor leading by example with agility. Understand the problem space. Discover and deliver capabilities—review outcomes with the customers and end-users. Learn what can be learned. Act on what we have discovered—set expectations of uncertainty and bets, not dates. Don't feed the desire to know something we don't know. As a well-known agilist said to me, forecasting makes us feel we are doing something useful; that feeling or unmet need is what needs to be addressed.

We get lost during sizing and forecasting, and we often forget value. Measure with people affected if the work made a difference and tweak.

Acknowledgments

Many thanks to members of the Design, UX, Product Management, #NoEstimates, Scrum, and Kanban communities for either inspiring ideas for this article, or reviewing this article in its entirety or in part. Reviews of the article added perspectives that were acted upon; even so, reviewers might disagree with the article. I want to offer special thanks to Glaudia Califano of Red Tangerine and Christian Neverdal for reviewing in detail a much earlier iteration of this article, I am in their debt. I am in debt to many other un-named reviewers who also put in a lot of effort. I especially want to thank Arpad Piskolti for inspiring me with his uncertainty-embracing catchphrase “and I’ll give you a better forecast next time (week/Sprint/month).”

Appendix  

Flow metrics

Flow metrics are not just about throughput. Following the Kanban Guide for Scrum Teams, one can also look at WIP, Cycle Time, and Work Item Age; in that context, looking at throughput alone would be myopic. Even in a Scrum context, focusing on throughput but ignoring impeded/forgotten product backlog items is a fool’s errand.

The Kanban Guide for Scrum Teams introduces four metrics to Scrum, namely:

• Work in Progress (WIP): The number of work items started but not finished. The team can use the WIP metric to provide transparency about their progress towards reducing their WIP and improving their flow. Note that there is a difference between the WIP metric and the policies a Scrum Team uses to limit WIP. 
• Cycle Time: in a Kanban Guide for Scrum Team context, the elapsed calendar time between when a work item starts and when a work item finishes (rounded up). 
• Work Item Age: The elapsed calendar time between when a work item started and the current time; this applies only to items still in progress. 
• Throughput: The number of product backlog items finished per unit of time.

See How can Scrum with Kanban help people solve complex problems? 

Some techniques to inform/challenge forecasts

• Scenario planning
• Threatcasting 
• Future Backwards 
• Ritual Dissent 
• Cynefin
• Reality Tree

Guidance on Sizing - other approaches

Upsides of other sizing approaches

Potential downsides of other sizing approaches

Other considerations of other sizing approaches