The software industry is not good in estimation. Vasco Duarte suggests that we should focus on impact and value instead of output to make projects successful. In the book NoEstimates: How to Measure Project Progress Without Estimating he explores how NoEstimates can help to manage projects with a focus on value and predictability, report progress quickly and often, and adapt plans constantly based on existing data.
InfoQ interviewed Vasco Duarte about how #NoEstimates started; what the use of estimation techniques tries to solve; what makes estimation so difficult; how you can plan projects and track progress using #NoEstimates; examples of how #NoEstimates has helped to focus on the value and where people can find more information about #NoEstimates.
InfoQ: Can you tell us where and how #NoEstimates started?
Duarte: Around 2011-2012 I was preparing a presentation at OOP2012 entitled “Story Points considered harmful”. As part of that work I started tweeting under the hashtag #estwaste, and at the same time Neil Killick and Woody Zuill started to tweet about similar topics. The earliest recorded tweet with the #NoEstimates was by Woody Zuill, and after that the conversation on Twitter took off. I had several Skype discussions about the topic, Woody Zuill talked to 100+ people on skype about the topic. It was clear that people were interested in the topic. The original hashtag that I was tweeting under #EstWaste (for Estimation is Waste) never really took off, but #NoEstimates, a simple hashtag was able to start a much needed conversation in our industry.
From the still very active conversation on Twitter (3 years later!) we can see that we are addressing a real problem in the software industry. By now there are many people tweeting under the hashtag #NoEstimates, and many conference talks on the topic. There’s a community gathering around this very important topic for the software industry.
InfoQ: In your opinion why do projects use estimates, what’s the problem that they try to solve with them?
Duarte: There are many problems addressed by the use of the tool we call estimation, and that’s what it is: a tool, nothing else. Your question is very important, because once we identify the questions, or problems we are trying to tackle we can then make an informed assessment if the tool (estimation) is the appropriate tool for us.
I have designed and by now hosted multiple times a workshop called “#NoEstimates: focus on what really matters”, and I ask this very question from the participants. The answers range from the obvious: “to know when the project will be delivered”, to the equally predictable: “to calculate the ROI (return on investment) for my project”. The problem with these answers is what is not said, the assumption behind the use of estimates as tool to answer these questions. The assumption is that estimates can be made reliable on a large scale (across the whole IT industry), however that is not supported by the evidence we see. The much quoted CHAOS report continues to report, year after year how often projects are late and over budget. As a defense many will say: “projects should not be scored on being on time, on budget, there are other, more important variables”. I agree. On-time, on-budget record is a poor predictor of project success. In one case, a project that was 200% late had the impact of totally changing the company where it was executed, it helped them to reach new markets, create a new business model and is now generating around 50% of the revenues for that company. Was the schedule important? Perhaps, but not as important as the “value” that company was trying to deliver. In the end value trumps on-time, on-budget considerations (to a certain point), and that’s one of the key aspects I continuously emphasize in my #NoEstimates talks and workshops: focus on value, make decisions based on value, not estimates. Estimates are a poor predictor of project success.
InfoQ: What is it that makes estimation so difficult? What are the reasons people estimate badly?
Duarte: I can’t say what makes estimation difficult. But most of us know that it is. We suffer it every day at work. The stats from multiple surveys also backup that assertion that estimates are difficult. Here is a small sample:
- “Of the large systems that are completed, about 66% experience schedule delays & cost overrun”, “Project Management Tools and Software Failures and Successes” by Capers Jones, Crosstalk, the Journal of Defense Software Engineering
- In 2011, the average delay for projects surveyed in the CHAOS report was 63%. This number is in line with a private survey I did in 2003 in a software development company: of the 17 projects surveyed the average delay was 62% and the largest project delay was 200%.
- Scott Ambler runs a popular survey about project success in the software industry. His data from 2013 shows that 53% of traditional projects were failed (no solution delivered) or challenged (project did meet success criteria).
And we can find similar results in other surveys, from multiple sources. All these examples prove one thing: we, as an industry are not good at estimation. Sure, some people will claim they are super-forecasters, but the term already implies that the majority of the people are not. We need to create an ethical and responsible approach to software development, and estimates are not a good tool because they fail to deliver on the promise (know the duration/cost within acceptable range), and additionally promote many different dysfunctions in the organization. In the book I explain some of those dysfunctions such as: estimate bargaining, political games, etc.
InfoQ: Does #NoEstimates mean that you should never ever do any estimates? Are estimates evil?
Duarte: Yes and no. #NoEstimates is a principled approach to solve a particular problem in our industry: to understand and be able to act on the cost and duration of a project one should use alternatives to estimates. When you apply that principle in a particular context you may still take estimates as a “stepping-stone” to the next stage of #NoEstimates. I explain this in the book with the description of what I called “the 7 step journey to #NoEstimates”.
In my own experience estimates are rarely needed, and even when needed they just point to another dysfunction that we should be tackling. In this context, eliminating estimates is indeed the goal. But just like Toyota has devised a multiple-level approach to waste (unnecessary waste, and necessary waste), so must we for estimates. Sometimes we must accept estimates, but endeavor to find ways to eliminate that need for estimates. This is possible, and I have helped many teams and organizations totally eliminate the need for estimates. In the later chapters of the book, we follow Carmen – our hero in the #NoEstimates story – as she discovers some of the ways in which you can eliminate estimation waste, and still be able to answer those critical business questions that customers will ask, such as: “will we be able to release this Feature on time?”
InfoQ: Why do you emphasize selecting the most important work over prioritization based on estimates?
Duarte: This is a very important question that might warrant a book on its own. Simply put, prioritization of a list of items pre-supposes that all items in the list will be needed. This is very often not the case. As we know many of the items of functionality we end up delivering to our customers are rarely or never used. Yet, these unnecessary items take time and effort to develop, and in some cases add considerably to the risks in our project. Prioritization assumes all items in a backlog are needed. I don’t make that assumption. Therefore I help my customers to understand their business model, and then deliberately explore options on how to leverage or amplify the success of that business model. To do this, one must focus on selecting high-impact experiments (Features or Stories), and not mindlessly deliver of a long list of detailed requirements. This single-minded focus on “impact” (or outcome) rather than output (how many stories can we deliver), is what allows us to discover the true value in our projects. This is why I prefer to talk about selection of the highest impact items and delivering those quickly, instead of what is the priority of item #34 vs #35 in a long list of requirements.
InfoQ: How do you ”plan” projects using #NoEstimates?
Duarte: This is a great question, because it clearly exposes one of the most important fallacies in the #NoEstimates debate. Critics of #NoEstimates will very often say that you can’t plan unless you estimate. This is completely false, of course. Estimation is a tool to “size” a plan that has to be constructed before we have those estimates. For example, we can easily decide which stories to take into a sprint without sizing those stories. Many of the teams that I work with will take a number of stories into the sprint based on the goal for that sprint, without even sizing those stories. When they start executing those stories, they constantly re-evaluate their situation compared to the sprint goal, and will remove or add stories to the sprint to help them achieve the goal for that sprint. A goal could be something like: improve performance of X to 500 transactions per second. Such a goal does not stipulate the “how”, only the “why”. The stories we take into the sprint are subject to that goal, not the other way around.
The Sprint example above illustrates how you can separate estimation from planning at the sprint level. The same can be done at the project level. Many of the clients I work with have a business to keep running and have business goals they must achieve. Instead of committing to a specific set of Features or Stories to achieve those goals, they do continuous planning. We define the goals for the project at the start, and define how much we will invest in that project (a budget, not an estimate), and then we continuously manage scope to deliver the goals we have within the constraints (may be time, money or both) we must adhere to.
What this approach to planning fosters is a relentless focus on value and continuous value delivery that I explain further below.
InfoQ: How can you track progress using #NoEstimates?
Duarte: In the book I go into a lot more detail on how we can track and make progress transparent to our stakeholders. Simply put, I count the number of items delivered, and compare that to the number of items in the backlog. A very common assumption that people make is that you need to know the complete list of items to be able to track progress this way. In practice however, it is very rare that we know “all items” until late in a project. Scope creep (or as I call it: value discovery) is a natural process of any software development project. As we develop a project, we discover more value, and it is only natural that we want to deliver the newly discovered items as well.
So, I never assume that scope is fixed, or the backlog is complete. Rather I project (forecast) the rate of delivery (delivered items divided by the time it took to deliver) into the future and assess what is the likely number of items we can deliver in the time we have available. Inevitably, at some point the number of items remaining in the backlog will be larger than the items we can deliver based on our rate of delivery. At this point is when the most crucial conversations in our projects start: which items should we deliver? Which should we remove from the backlog? This process I call: continuous scope management, and in the book I call this the most important tool for software project management. Many will think that this is a difficult conversation, fraught with difficult expectation management. I agree, but these type of conversations are the ones that help us focus on value, and continuous delivery of value. Simply put, I use this approach to foster a conversation about value delivery, instead of measuring output compared to an inevitably fragile plan. Following a plan (result of our estimates) is a fragile approach to software project management. If we are to be agile in our approach to software development we must build anti-fragile approaches to project management, and #NoEstimates is one of those approaches.
InfoQ: One of your claims is that #NoEstimates helps teams and organizations to focus on what is valuable, but how does that work in practice?
Duarte: In the previous question I referred to the only tool that can really help us deliver a software project on time: continuous scope management. But it is important to explore why this continuous scope management is necessary. One of the key reasons is that we will know better what we need to deliver when we start developing the software. As we develop, demonstrate and get feedback new ideas about the right functionality, i.e. valuable functionality, emerge. This value that we discover is often called “scope creep”, as in the scope creeps up on us and when finally notice it we already have way too many things to deliver. I don’t like the term “scope creep” because is hides the true nature of that phenomenon: we discover value as we deliver what we had originally planned. Following the original plans, which were estimated based on a defined set of functionality, leads us to treat this value discovery as scope creep and ignore its value. And there’s a good reason for it: accepting all of these changes would make the project miss all its deadlines. This leads to one of the dysfunctions in estimate-driven software development which is to prioritize the plan, and the schedule over the new value discovered.
I take a different approach. I help teams work within a specified time and investment box, but at the same time constantly adjust the scope to deliver more value. For this we use the forecasting approach I describe in the NoEstimates book, so that we can safely change the scope of our projects to incrementally deliver more value, instead of trying to follow the original plan. The side-effect of this forecasting approach is that it brings continuous scope management to the early phase of the project, we end up managing scope from day one, thanks to the forecasting technique. The alternative would be to continuously re-estimate everything we come up with, and every little change in the plan. This is clearly wasteful, especially when the alternative I propose (forecasting based on data) can give us accuracy levels near 100% as I describe in some of my talks available on YouTube.
InfoQ: Where can people go if they want to learn more about #NoEstimates?
Duarte: There are many sources if you want to get started reading more about #NoEstimates, Woody Zuill has collected a list of blogs from multiple authors in his website. I’ve also collected a “getting started” collection of articles from multiple authors here. And for those that are interested in an in depth look at how to apply the ideas in #NoEstimates I wrote the NoEstimates book which is available digitally at the OikosofySeries website and will be available on Amazon soon. To know more about the book, including getting the first chapter for free, you can go to NoEstimatesBook.com and register. After registering you will get a link to download the first chapter of the book with the Table of Contents.
When writing the book, I realized that I could not possibly write an entertaining, informative book and cover all the aspects of #NoEstimates that I wanted to cover, so I partnered up with Tomas Rybing and Evan Leybourn who wrote two companion mini e-books to the NoEstimates book. Tomas wrote about capacity planning with #NoEstimates, and Evan wrote about contracts and how they define our relationship with clients in ways that either support or make difficult the use of practices like #NoEstimates.
But there is a lot more, I interview 9 people on video about #NoEstimates. The book includes interviews with 4 NoEstimates practitioners. One I'd like to highlight is the interview with Clinton Keith. He tells us a story that is completely mind-blowing: how he turns a previously-lasting 18 month project into less than 2 weeks of work! I was completely surprised by that story, and believe it has critical insights for those interested in applying #NoEstimates.
In these 9 video interviews there’s also 2 interviews with CEO’s that apply #NoEstimates ideas in their own companies.
- Sven Ditz of Seitgeist: who is applying NoEstimates in his own company. In his video he tells us a mind-blowing story of how he can completely obliterated the need for a bidding process and wins deals without ever providing a bid.
- Diego Cenzano of Biko2, a veteran IT entrepreneur has a beautiful story of how he is able to help his customers focus on value (beyond expectation) with his approach (which requires no estimates.
NoEstimatesBook is a work that hopes to start a long needed discussion in our industry: how can we continue to improve how we work, and base our decisions in methods and tools that work better in the software domain. Let’s not forget that estimation is not a technique developed for software projects. It is about time we recognize that, and accept that this tool may not apply in our software development domain.
About the Book Author
Vasco Duarte wants to transform product development organizations into product business organizations. He does that by focusing the work of the product development teams on the end-to-end life-cycle of their products. Vasco is currently a Managing Partner at Oikosofy. Product Manager, Scrum Master, Project Manager, Director, Agile Coach are only some of the roles that he has taken in software development organizations. Vasco has worked in the software industry since 1997, and has been an Agile practitioner since 2004. He has worked in small, medium and large software organizations as an Agile Coach or leader in agile adoption at those organizations. He was one of the leaders and catalysts of Agile methods and Agile culture adoption at Avira, Nokia and F-Secure.