A superior level of performing well in spite of significant uncertainty will be achieved only when a decision making process is established that verbalizes the uncertain potential results and lead the decision makers to contemplate decisions that would achieve high gains most of the time, but also take into account that in some cases limited damage will occur. - (Schragenheim 2015)
Introduction
When working on projects, it is most of the time necessary to forecast the project delivery time up front in order to secure funding, obtain approval, or learn what skills are missing and need to be hired. Delivering on time and within budget is a competitive advantage.
A probabilistic high-level plan forecasts the initial budget as well as the range of the time frame for a project. We don’t plan in detail what is not absolutely necessary to plan (Bakardzhiev 2014). The short-term details, like the scheduling, are based on immediate needs and capabilities – and we create these schedules upon the execution of the high-level plan.
Little's Law helps us take an "outside view" on the project we are forecasting, based on knowledge about actual system performance on a reference class of comparable projects. The method can help any team that uses user stories for planning and tracking project execution, no matter the development process used.
This article also covers how to manage the inherent uncertainty associated with planning and executing a project by protecting project delivery date with a project buffer.
We’ll see some sample forecasts but first let’s recall what Little’s Law is.
Little’s law
Little’s law for development systems is:
is the average throughput of the development process per unit time, 
is the average number of work items in progress during the development process per the same unit of time, and 
is the average time a work item spends in development as WIP (i.e. its lead time) in the same unit of time (Little 2008).
Anderson’s formula is derived from Little’s Law to give us the relationship between the average lead time 
per work item, average number of work items in progress 
for the development system, and the finite time period T over which the project system will deliver the collection of N work items that we call a project:
Little's Law deals with averages. It can help us calculate the average waiting time of an item in the system or the average lead time for a work item. In product development, we break the project delivery into a batch (or batches) of work items. Using Anderson’s formula, we can forecast how much time it will take for the batch to be processed by the development system.
Forecasting project staffing
Imagine we have a major project with 2,200 user stories. The business needs the project delivered in 10 months. How many people do we need?
We have N=2200 user stories and T=10 months, which is 40 weeks. We also have historical data that average lead time 
for the development organization is 0.4 weeks. Anderson’s formula gives:
is the number of developers multiplied by the allowed number of work items per developer, and then plus the size of all the queues. If the development organization has the policy of one work item per person, we will require 22 persons. That assumes the development queue has finite size of zero work items. If the development queue has finite size of two user stories, we will need 20 persons (20 developers * 1 user story per developer + queue of 2 = 22).
Forecasting delivery date
The business wants to know when we will be able to deliver. There are no options for adding more developers to the development organization. We know the average work in process is 22 user stories. Historical data tells us that average lead time 
for the development organization is 0.4 weeks per user story. Using the Anderson’s formula, we have:
Analyzing all the stories in a project in order to get the total number of work items requires significant time and it can easily happen that a great part of this effort will be pure waste.
Should we list all the user stories up front?
Not necessarily, if we use a technique such as randomized branch sampling (RBS). RBS allows us to estimate the total number of user stories without prior identification, analysis, and sizing of every single user story. Sampling saves a lot of time, which we could use to manage actual risks. (You can read more about RBS in “Probabilistic Project Sizing Using Randomized Branch Sampling (RBS)”.)
What is a project?
Usually, project planning models projects as a network of activities and calculates the time needed to deliver the project by estimating the effort required to accomplish each activity contained in the work breakdown structure. We challenge the project-management paradigm and suggest that for planning purposes it is better to model projects as a flow of work items through a system. Hence the definition – a project is a batch of work items, each representing independent customer value that must be delivered by a due date. The batch contains all the work items that we need to complete to deliver a new product (or a version of an existing product) with specified capabilities. In order to prepare the batch, we need to break the product scope into work items, each one representing independent customer value. Even for a quality-related requirement such as “the system should scale horizontally”, we need to have a work item. It is important that each of the work items can be delivered in any order, like the user stories created in following the INVEST mnemonic. We don’t try to estimate the size of the work items. There are only two sizes: small enough and too big. The two sizes are context specific. They have no correlation to the effort needed to finish them. Items that are too big should be split and not allowed to enter the backlog (Bakardzhiev 2014).
Can we use Little’s Law for forecasting?
Little’s Law applies to any system and, most importantly, it applies to systems within systems. So, in a bank, the queue in front of the tellers might be one subsystem and each of the tellers is another subsystem, and Little’s Law could be applied to each as well as to the whole thing.
The conditions necessary for Little’s Law to hold are:
- At a minimum, we must have conservation of flow. Thus, the average throughput or departure rate equals the average input or arrival rate. Furthermore, we need to assume that all work items that enter the system will eventually be completed and will exit the system; there are no work items that get lost or never depart from the system.
- Little's Law holds exactly between any two times at which the system is empty. For a project, it means that we start with a batch of work items and when we deliver, there are no items left in the development system.
For a project, both conditions are true, so Little’s Law holds.
Little’s Law is about measuring and looking back to analyze what has previously happened. If we can measure any two of the statistics of our development system – the average lead time, average work in progress, or the average throughput – then using Little’s Law we can calculate the third. If the data is for a particular reference class of projects (Bakardzhiev 2014) then we have a suitable frame of reference for estimating a new project.
If we assume that in the near future our system will have the same characteristics as the reference, we can safely forecast that the future performance will continue to reflect the recent past.
Embracing uncertainty when planning a project
Looking at Anderson’s formula, we see that all of its parameters are averages without variance. The number of work items N is also uncertain, due to things such as dark matter and failure load as we shall see a bit later. Uncertainty exists. People are not blind to it and they add a lot of safety time into their plans in order to protect project delivery date. That safety is called padding, margin, etc.
Managing a project buffer
We sum up all the safety time added to the project duration and call it a project buffer. We borrow this concept, called buffer management, from the Theory of Constraints (TOC). We visualize and monitor the buffer during the project to ensure that project due date is protected. Hidden buffers are always fully consumed and wasted. Buffer management helps us identify potential problems much earlier than we ordinarily would with traditional project-management techniques. It provides focus and helps management be proactive. Building explicit buffering into the project plan would also help communicate the degree of anticipated risk in the work and demonstrate that we are mitigating it from day one.
The promised delivery date of a project is the sum of the initially forecast duration plus the project buffer. Think of the buffer as overdraft protection for a bank account that holds time instead of money. As the project proceeds, a work item that takes longer than planned consumes some of the project buffer. If a work item takes less time than planned, we add its time surplus to the project buffer.
Sizing a project buffer
The size of the project buffer must be based on the uncertainty involved in the project. The uncertainty is based on the fact that project delivery time is not a single number but a probability distribution, as in the following histogram.
The green line is the median (50th percentile) delivery date and indicates that we would deliver the project in 78 or fewer days half of the time. The red line, the 95th percentile, indicates that 95% of the time it would take at least 98 days to deliver the product.
Going from 50% to 95% probability of success means adding 25% to the project delivery time. If we offer our clients the 95th percentile estimate, the 20-day difference between the median and the 95th percentile will be our project buffer, PB, which we can calculate:
We can modify this to:
N is the number of work items to be delivered by the project, 
is the median (50th percentile) takt time for the project, 
is the planned (say 95th percentile) takt time for the project, 
is the median throughput for the project, and 
is the planned (say 95th percentile) throughput for the project
A project buffer should be sized aggressively because an oversized buffer could lead to uncompetitive proposals and consequently to loss of business.
A project buffer should also account for delivery of an uncertain number of work items and for the first and third legs of the Z curve (Bakardzhiev 2014).
An uncertain number of work items
We get the final number of work items by accounting for dark matter and the failure load. We do this by padding the planned (forecasted using RBS) number of work items 
to be delivered by the project:
DM is the average percentage of dark matter we set based on historical observation. If we don't have a history, we use 100%-200% expansion (i.e. 1.0 to 2.0) as the quotient. FL is the average percentage of failure load.
Dark matter
The number of work items will grow through natural expansion due to emergent requirements that we only identify when we bring our results to the customer – and then requirements change and scope expands. In Kanban we call this "dark matter" (Anderson 2011). Dark-matter expansion occurs for at least two reasons: instability of the problem domain or immaturity of the delivery teams that are managing the emergent and high-change-risk requirements. That is why we add work items to the project scope in order to compensate for unknown work or dark matter. We can derive how many items we must add to account for dark matter from historical project data or we can expertly estimate it. I usually add at least 30% more items for this. With novice teams on a new product (especially for a highly innovative new product with which we have no prior knowledge or experience), I might add as much as 100%.
Failure load
We also add work items to compensate for the expected failure load in terms of defects, rework, and technical debt. Failure load tracks how many work items the development system must process due to poor quality (Anderson 2011). That includes production defects (bugs) in software and new features requested by the users because of a poor usability or a failure to properly anticipate user needs. Defects represent opportunity cost and affect the lead time and throughput of the development system.
The delivery rate TH
Throughput (delivery rate of the development process per unit time) follows a “Z-curve pattern” (Anderson 2003 and Bakardzhiev 2014):
On the X axis, we have the project time (in this graph, the axes’ units are percent but it may also be days). On the Y axis, we have the number of work items delivered. The Z curve can be divided in three parts or, we can say, it has three legs. There is empirical evidence that for the first 20% of the time, the delivery rate will be slow. Then for 60% of the time, we’ll go faster; it’s the “hyper-productivity” period. And during the final 20%, we’ll go slowly. Of course, numbers may vary depending on the context but the basic principle about the three sections is correct. That means we need more capacity than can be inferred from the average delivery rate 
We account for that by increasing the project buffer PB by a certain coefficient, usually 25%-50%.
Managing uncertainty during project execution
Monitoring project-buffer usage
The two essential measurements of project performance in buffer management are the percentage of work completed and the amount of project buffer consumed. Upon delivering a work item, we compare the consumption rate of the buffer to the project completion rate. This relationship is the signal to management for appropriate action. Project review meetings focus on whether the completion of the project is on pace for completion without consuming the project buffer. Project-buffer usage determines whether or not the delivery date is affected. Buffer usage should be monitored as a percentage used. We are visualizing the cumulative buffer usage just like we would visualize bank-account usage. On the diagram below, each point represents the delivery of one or more work items. The X axis indicates project completion 
as a percentage of the total scope of N work items. The Y axis is percentage of the project buffer consumed 
at the moment the work item is delivered.
Note that:
is the project start date.
is the date of delivery of the work item at time t.
is the number of completed work items at time i.
is the planned (say 95 percentile) Takt Time for the project
is the median takt time for the project.
is the percent delivered out of the total work items to be delivered at time t.
is the percent of the project buffer consumed at time t.
is the actual lead time at time t.
is the planned lead time at time t.
As long as there is some predetermined proportion of buffer remaining, the project progress is assumed to go well. If the project consumes a certain amount of buffer, say 80%, we take it as a warning. If the buffer consumption rate rises so high that it looks like it will consume the whole buffer before project completion, we must take corrective action.
Focus on the median lead time
When using kanban systems, the focus of the project manager shifts from monitoring all tasks to monitoring average lead time 
because if 
is limited then throughput depends only on lead time.
Human beings do not implicitly understand the average lead time – we have to calculate it. What people can understand is the median – in the sense of “every other one”.
Every team member will understand the message that every other ticket should take two days or less. The odd ones can take longer than that but if they take longer than twice the median (Zheglov 2013) then alert your manager. We don’t want the tail of the lead-time distribution to get really long, which would mean a drop in our delivery rate and that we are not hitting the target!
On a daily basis we can create a report that shows:
- what percentage of the work items have lead time equal or less and
- what percentage of the work items have lead time longer than the median lead time we used when planned the project.
If we have 50% in both columns that means we are as fast as planned. If the first column has more than 50% it means we run faster than planned. If second column has more than 50% it means we are slower than planned.
Lead time is calculated not only for the work items that are done but for all work items in process. That makes lead time (LT) a leading indicator.
Example:
- Planned Median LT=5 days.
- Work item A is Done. Its LT=5 days.
- Work item B is in Development. From the moment it was started till the moment we created the report it has been in process for 7 days.
- Work item C is in Test. From the moment it was started till the moment we created the report it has been in process for 3 days.
Visually the report should look like that:
|
LT less or equal than planned median LT |
LT greater than planned median LT |
|
(2 work items out of 3 ) 67% |
(1 work item out of 3) 33% |
Conclusions
Probabilistic forecasting informed by historical data saves a lot of time, which project managers could use to do actual risk management.
Little’s Law can help any team that uses user stories for planning and tracking project execution no matter what development process it uses. We can use a project buffer to manage the inherent uncertainty associated with planning and executing a fixed-bid project and protect its delivery date.
About the Author
Dimitar Bakardzhiev is the managing director of Taller Technologies Bulgaria and an expert in driving successful and cost-effective technology development. As a LKU Accredited Kanban Trainer (AKT), Dimitar puts lean principles to work every day when managing complex software projects. Dimitar has been one of the evangelists of Kanban in Bulgaria and has published David Anderson’s Kanban book as well as books by Goldratt and Deming in the local language.
References
- Anderson, D.J. 2003. Agile Management for Software Engineering. Prentice Hall.
- Anderson, D.J. 2011. “Predictability and Measurement with Kanban” (presentation). Lean Kanban Central Europe 2011.
- Bakardzhiev, Dimitar. 2014. “#NoEstimates Project Planning Using Monte Carlo Simulation”. InfoQ.
- Bakardzhiev, Dimitar. 2015. “Probabilistic Project Sizing Using Randomized Branch Sampling (RBS)”. InfoQ.
- Little, John D. C. 1961. “A Proof for the Queuing Formula: L = λW”. Operations Research vol. 9, issue 3, pp. 383-387.
- Little, John D. C. 2011. “Little's Law As Viewed on Its 50th Anniversary”. Operations Research vol. 59, issue 3, pp. 536-549.
- Little, John D. C. and Stephen C. Graves. 2008. “Little’s Law” Chapter 5 in Building Intuition, ed. by D. Chhajed and T. J. Lowe, Springer Science+Business Media, pp. 81-100.
- Savage, Sam L. 2012. The Flaw of Averages. Wiley.
- Schragenheim, Eli. 2015. “The current TOC achievements in handling uncertainty”. Eli Schragenheim.
- Zheglov, Alexei. 2013. “On the Practically Useful Properties of the Weibull Distribution”. Connected Knowledge.