A web-based tool for calculating project estimates using a Monte Carlo simulation was recently made publicly available. The results of a Monte Carlo simulation can help manage expectations with stakeholders. However, such tools were previously available only within purchased ALM software or required manually setting up and running a spreadsheet. The new tool was developed by agile technology leader Rich Stewart. He hopes that agile teams will use it to facilitate conversations between Scrum Masters and product owners, focusing on realistic outcomes and not overly optimistic projections.
Software delivery teams often struggle with providing reliable estimates for future work. Some agile practitioners have discussed the benefits of using Monte Carlo simulations as one option to provide more realistic predictions. Historical performance data, in this case story points or user stories accepted during past sprints, is used as input for the simulation. The algorithm then uses repeated random sampling to predict likely future sprint outcomes, again measured in story points or user stories.
While primarily intended for Scrum teams that work in sprints, the web app is flexible enough to adapt to other agile methodologies, such as Kanban, by choosing an arbitrary timebox for grouping velocity measurements. Multiple sprints are combined into a program increment (PI), often referred to as a release.
Using the web interface, historical data can be either entered in a grid or uploaded from an Excel file. The Monte Carlo analysis creates a range of possible outcomes, along with a corresponding probability, and displays the results as a histogram. The tabular results can be exported to Excel for communicating with project stakeholders.
The tool is available at https://sevawisegames.com/games/tools/monte-carlo, and has a detailed help page.
InfoQ spoke to Stewart to better understand his rationale for developing the tool.
InfoQ: Why do we need a tool to help with project estimation? Do you think this is becoming more important now than it was in the past?
Rich Stewart: Much has been written in recent years about the importance of agile teams delivering value for their customers. The mantra is often "outcomes over output" and product owners in many organizations are rightfully starting to focus on ways to measure outcomes. While this shift represents a natural maturation of agile delivery in organizations, the importance of predictable delivery of features (i.e. output) remains an important goal for agile delivery teams.
InfoQ: Does this run counter to the #NoEstimates movement, or do you see some benefit for that philosophy as well?
Stewart: #NoEstimates doesn't preclude one from planning based on story count, which has been shown by many, including myself, to often be as accurate as estimates based on story points. The rationale is easy to understand. If a product owner writes user stories in a consistent manner, the law of large numbers indicates that any grouping of 10-20 user stories written by the product owner should amount to roughly the same total story points. In this event, story count should be equally effective as story points for estimating.
Furthermore, while estimating may be difficult early on in the product development lifecycle, as additional, similar functionality gets built, it gets easier to estimate stories. At that point in the development lifecycle, an estimation technique such as Monte Carlo may prove to be very useful.
InfoQ: Are there any caveats people should be aware of when using the tool?
Stewart: To improve the usefulness of this tool, historical data should only be used that reflects the current maturity and/or size of the team. Additionally, if there is an outlier historical data point, such as an anomalously small Velocity value due to end-of-year holidays, remove it from the set of historical data to be simulated.
InfoQ: What was the technology stack you used to create the application?
Stewart: It developed the tool as a single page application (SPA) utilizing Angular and several JavaScript libraries, including ag-grid, chart.js, and js-xlsx.
Editor's Note: Links to the Monte Carlo Estimation tool were updated on 2020-09-23