Waste Not, Want Not: A Simplified Value Stream Map for Uncovering Waste

Introduction

This article describes how to create a simplified value stream map that can be used to uncover wasteful practices in a software engineering organization. It will briefly discuss Lean principles and the definition of waste. From there it will describe how to construct the analysis and interpret the results.

The Lean Philosophy of Waste

Waste is central to the tenets of Lean Manufacturing philosophy and as a result, it has a special definition. In short, waste is defined as anything that does not directly contribute to getting a product into customers’ hands. That is, any process or task is considered waste if it cannot be shown to result directly in a product moving closer to delivery to customers. Another way to think about it is this: would the customer be willing to pay for the process or task? If not, then it is probably waste.  

Lean Manufacturing places a heavy focus on reducing or eliminating waste in its various forms and in so doing it improves the efficiency and profitability of manufacturing processes.

But what does this mean for software engineering? We don’t have factories and assembly lines churning out widgets that will be used to create a physical product. All of our products are electronic, a series of zeroes and ones that we stitch together to form digital goods. Actually, and perhaps not surprisingly, the lean principles of manufacturing apply quite well to our field. This article will cover just two of them, Inventory and Waiting.

The Problem of Inventory

In manufacturing, inventory is defined as unfinished material that is required to create products or completed products that are awaiting delivery to customers. The problem is that excessive inventory represents waste because money has been spent to purchase or create the inventory but it is not yet being used. That means it is an outlay of capital that is not providing a return on investment. Additionally, more money is required to store, move, or otherwise manage the inventory, thereby compounding the problem. This is wasteful.

The most obvious form of inventory in software engineering is code that has been completed but not yet delivered to customers. This code may be waiting for formal testing, waiting on some kind of approval process, waiting on other code, or waiting to be deployed into the Production environment. Whatever the reason, if the code has been written but is not in customers’ hands then it inevitably must be classified as inventory. 

User stories can also be considered a form of inventory, albeit a less-visible one than undelivered software. If a story is entered into a backlog then it is probably valuable enough to a business that someone is tasked with spending money to create it. Once that happens the story represents unfinished inventory that is waiting to be assembled into a final product.

The Problem of Waiting

Lean principles dictate that the time spent waiting is waste. By definition, anything that disrupts the steady, one-piece flow of goods from inception to delivery is wasteful. The most significant problem caused by waiting is an opportunity cost. Every moment that delays the delivery of products to customers is a cumulative source of forgone revenue. Add in the effects of compounding on that revenue and the costs escalate even quicker. 

In the software world, we often see long periods of waiting. For example, a story may sit in a backlog for a long time before work commences on it. Also, completed code may have to wait for formal testing. Additionally, tested code may have to wait for approvals before it can be deployed to Production. There may also be narrow windows for deploying the code that must be scheduled far in advance. Finally, the longer code sits before being tested or delivered then the more stale it becomes in our minds’ working memory. That means it will require additional time, and therefore cost, to re-familiarize ourselves with it when we begin to test or deploy it. All of these delays are wasteful.

These are just some of the costs associated with waiting. There are certainly others that are less obvious and as costly. 

Making Waste Visible

There are a number of ways to display waste in a system. The most common approach is probably the use of value stream maps. These are maps that show the journey of a product from raw material to finished goods delivered to customers. They are very helpful in understanding the flow of goods and pinpointing wasteful delays. 

These don’t always seem relevant to software engineering because the images of factories, trucks, and forklifts don’t apply. Even the versions developed specifically for software sometimes seem to lack the qualities of being simple and definitive. What if we just want to know one thing: for any given process, how much time is spent waiting versus working? This would give us a simplified view of waste for any process and would be helpful in making it more efficient. 

The details for constructing this are straightforward. Let’s define working as time spent actively creating a product, time for which customers would gladly pay. Let’s define waiting as time spent waiting on something, time for which customers would not want to pay. We use duration (not effort) for both and we maintain consistent time units between them. We assign a positive value to time units when actively working on a product and a negative value when waiting. Add them all up and we can then compare the relative time spent waiting versus working. A perfectly efficient process would show a positive sum without any negative entries. A wasteful process would show a negative sum with many negative entries. Table 1 shows a generic form of this approach. 

Table 1. A generic form.

Notice how “Work” entries are positive time values and “Wait” values are negative. Although the time units here are generic, it is important to note that they are consistent across all events because we can’t compare, for example, days and hours. The final sum is slightly positive, indicating a generally efficient process but one that can be clearly improved by reducing both the quantity and magnitude of waiting. 

Let’s now move to more realistic examples.

A Tale of Two Companies

Let’s compare two hypothetical software companies as they create a user story and turn it into working software. Although hypothetical, the companies’ practices are based on real-world experience over the years. The first one, which we call Wasteful, Inc., is a model of bureaucratic inefficiency. There are long periods of waiting punctuated by short moments of working. The second company, which we call Efficient, Inc., is exactly the opposite. A model of efficient processes that quickly move a story from idea to working software in use by customers. Let’s see how they compare using our simple analysis. 

Table 2 shows how Wasteful, Inc. performs its user story task. 

Table 2. Wasteful, Inc. user story delivery.

Figure 1 Shows this in graphical form.

[Click on the image to enlarge it]

Figure 1. Wasteful, Inc, user story delivery in graphical form.

Notice how the wasteful approach uses an inefficient start-stop-start method of moving work along a timeline. This is often a result of waiting for fixed schedule windows to open, waiting on the availability of overbooked teams, and waiting on decision-making approvals. Several things stand out right away. The story waits 10 days before it is picked up by the developers which adds considerable delay. (This could actually be an optimistic number given that over-allocated teams might require months before they can begin.) The story then has to wait several days before a formal QA team has a schedule opening and begins testing it. Finally, once the code is certified by QA and ready for deployment, it must sit in a queue and wait to be delivered. In bureaucratic organizations, it is not unusual to find code that must wait for a deployment date before it can be delivered to customers. It is also common that deployments require multiple levels of approval with each level necessitating meetings, sign-offs, documentation, and other forms of waste. The steady accumulation of waste is visible in Figure 1 as a line that trends ever more negatively. The final score is 2.3 days of working versus 24 days of waiting, a ten-fold difference. Another metric to consider is the time spent working as a percentage of the total elapsed time. In this case, Table 2 shows that only 8.7% of the total duration of 26.3 days is spent in actual work. Over 90% of the time is spent waiting. 

We can also consider another view of this where we look at a cumulative percent of time spent working versus the total amount of elapsed time. Table 3 shows this for Wasteful, Inc.

[Click on the image to enlarge it]

Table 3. Wasteful, Inc. user story delivery percent working.

The work and wait times are the same as we saw in Table 2 but now we add columns that give us the percent of the total duration spent working. Notice how Wasteful, Inc. quickly drops to less than 20 percent of the total duration actually spent working. Figure 2 shows this in graphical form.

[Click on the image to enlarge it]

Figure 2. Wasteful, Inc. user story delivery in graphical form with percent working.

Figure 2 shows how the percent of time spent working at Wasteful, Inc. represents only a small fraction of the total amount of time required to deliver a user story. Clearly, Wasteful, Inc. is not an Agile organization.

Let’s now look at Efficient, Inc. where the code moves along in a steady, one-piece flow from start to finish. Table 4 and Figure 3 show the results. 

Table 4. Efficient, Inc user story delivery

Figure 3. Efficient, Inc. user story delivery in graphical form.

Notice how there are few breaks in the flow where the process must wait for the next step. Once a story is created, it moves quickly through the workflow all the way to delivery to customers. Some things stand out and give hints to how this is accomplished. First, there is little wait time to review the story with the developers. That indicates that stories are created when they are needed and reviewed the very next day. There is no loss of working-memory context and the story owner can easily retrieve the details. Next, work immediately begins on it because the story is “the most important thing right now” and time isn’t wasted reviewing it for later work. Formal QA testing is part of the development process because QA testers are integrated with the developers and this means there is no waiting for handoffs to another team. The developers are also working with testers to do Test-Driven Development, which improves quality and reduces the roundtrips caused by a defect repair-deploy-test cycle. Code is tested as it is developed, which is an efficient approach. Finally, there is little time wasted waiting on deployment schedules and approvals. The team has an automated build-test-deploy process using continuous integration and continuous delivery. Once code is checked in, it goes through a battery of automated tests that verify its quality and it is then automatically deployed to the customer. At most, there may be a brief delay while a product owner does a final review and then pushes a button to send the code to Production. This efficiency is shown in Figure 3 as a line that trends in a positive direction over time. The final score is 2.3 days of working and one day of waiting. Table 4 shows that 69.7% of the total duration of 3.3 days is spent in actual work. Only about 30% of the time is spent waiting. 

Let’s now look at the percent of time spent working at Efficient, Inc. Table 5 and Figure 4 show the results. 

[Click on the image to enlarge it]

Table 5. Efficient, Inc, user story delivery percent working.

[Click on the image to enlarge it]

Figure 4 Efficient, Inc. user story delivery in graphical form with percent working.

These displays show how Efficient, Inc. spends a significant amount of its time working when it delivers a user story. The numbers would look even better if the story had spent only a few hours in the backlog instead of an entire day. These results are a dramatic improvement over Wasteful, Inc. Clearly, Efficient, Inc. is a highly Agile organization. 

Other Uses

This analysis is by no means restricted to user stories. It can be used for any process or task. For example, how much time is spent working versus waiting when hiring a new employee? There is the time spent discussing a new hire, crafting the job posting, reviewing applications, interviewing candidates, making an offer, and so on. How does that compare to the waiting time between each of those tasks? Or what about the time required to adopt a new technology or process? From the moment the decision is considered until it is fully realized, how much of that time is spent waiting? In short, the only limits to this approach are one’s imagination. 

What’s in a Name?

It’s not obvious what to call this approach. Roughly speaking, it's a value stream map but a much-simplified version that omits all the loops and branches and instead simply lays things out in a linear timeline so that it's easy to visualize the time spent working versus the time spent waiting. 

After performing this analysis and seeing how much time is often spent waiting it's tempting to recast the original title of Samuel Beckett’s famous play Waiting for Godot into the more technologically-appropriate Waiting for Production. So that’s one possible name, a Waiting for Production Analysis.

In a sense, it probably should be called a Simple Value Stream Map but there is the old admonition Waste Not, Want Not. So that's what this is being called, a Waste Not, Want Not Map but call it whatever works. After all, what's in a name? Waste by any other name would smell as unpleasant. The purpose is to make visible what is often hidden and in so doing expose the amount of waste that goes undetected. Whatever we choose to call it isn’t relevant.

Caveat

As with any tool, there is a danger of misuse. Even though a process might display a considerable amount of waiting, there may be a perfectly justifiable business reason for it. For example, let’s take our analysis of user stories. A user story may not yet be fully fleshed when it is entered into a backlog, the market need it describes may change, the business need may change, or the understanding of the need might change. Or perhaps the story is entered as a placeholder for possible review at a future date. All of these mean that an idle user story does not necessarily represent waste and considering it as such is an improper use of the analysis. 

In short, blindly attempting to minimize waiting without regard to unintended consequences may wind up doing more harm than good. Using some discretion is well-advised here.

Final Thoughts

There are certainly additional items that can be added to this approach. For example, what would it look like to add weighting factors to wait and work events that reflect the number of people involved with each? A wait event carries much more weight if five people are waiting instead of just one. Or what if some parts of the software system must adhere to strict regulatory guidelines or schedules and other parts do not? Those with regulatory oversight would require a heavier weighting because defects and missed deadlines would have significant impacts. Alternatively, what if we were to calculate the actual financial cost of working versus waiting and produce an analysis that shows the physical cost of waste? That would likely be more readily grasped by financial stakeholders than looking at things via units of time. There are any number of ways in which to analyze waste.

In summary, this is a simple approach to analyzing value streams. All it requires are very basic skills. And therein lies its attractiveness: it’s simple, easy, and cheap. It is easy to create a timeline that shows where the bottlenecks lie in a given process. Efforts can then be directed toward those bottlenecks to improve efficiency.

The author would like to acknowledge Marc Kase (@marckase) for his help in reviewing this article. 

About the Author

J. Meadows is a technologist with decades of experience in software development, management, and numerical analysis. This is the author’s first article for InfoQ.