Customize Your Agile Approach: Start with Results You Want

I’ve met a number of people who want to use one and only one agile framework. I have nothing against frameworks, except when people think they can blindly follow a given framework and get the results they want. Too often, your project or your team needs to consider its options to use agile approaches to get the results you need. In that case, why not start defining the business results and outcomes you want to measure?

This is the second in a series of articles that will help you think about how you might want to customize your agile approach for your context. This article is about the data teams might collect and use—working product and other measures—that you might want to share with your managers and stakeholders.

In the first article Customize Your Agile Approach: Select Your Agile Approach That Fits Your Context, I wrote about how you might think about an agile approach that could work for you. One way to customize your agile approach is to consider what data you want to show the rest of the world.

Almost every project needs to show some form of progress so the project team and the management can make decisions about the project. Is the project progressing? Is the team stuck? Is the team at a stopping point because something else is more important?

Many managers and project managers are accustomed to using Gantt charts to see progress. The project manager (hopefully with the team) decide on their milestones for the various project phases. As they achieve those milestones, the team could show what they finished in the Gantt.

My experience with waterfall and other serial life cycle projects is that the project was on schedule until it was late. The project team finished their phases on time or close to on time. Then, when the testers received the project, the team finally realized they finished features. Too often, the testers discovered big problems. Then, the project became late due to rework.

Agile approaches allow us to use empirical data—real data about the work—to help us see actual progress. The data tells us where we are in the project: ahead, behind, or possibly even off course. And, because we can see the product as we build it, we know whether we can stop right here or continue with the project.

Agile approaches can change project measurements. First, let’s discuss velocity because so many teams use velocity as a progress measurement.

Velocity is a Team’s Measure of Capacity

Many teams measure their velocity in terms of story points. They think velocity is a measure of progress. It’s not. Velocity is a team’s current measure of capacity.

Some teams use velocity well as a way to see what they can finish in the next short time period. However, other teams have trouble with velocity in these ways:

• Velocity is a relative team measure so you can’t compare velocities between teams. In fact, if you, as a team, have been improving, you can’t compare your current velocity with your older velocities. Your team velocity will change over time as you change how you work as a team.
• Velocity is personal to a team. If you change the people on the team, expect the team’s velocity to change.
• Velocity changes when people are on vacation or sick.
• It’s too easy to make velocity a target instead of an assessment of capacity.

Teams might find it helpful to measure their velocity when they are trying to understand their capacity. Some teams like to know they can complete about 30 story points in an iteration. They might even realize those 30 points are about five small stories or one large story and two small stories in a given time. Those teams are measuring their capacity with story points.

Some teams try to predict a quarter’s or a project’s backlog. They know their current velocity (call it 30 points). They estimate the entire backlog as it stands now and realize they have about 180 points. They can make an educated, reasonable guess of about six iterations to finish what’s in the current backlog with the current team members. (They might explain, “A minimum of six iterations, knowing what we know now. We’ll keep updating our estimate as we continue.”)

Velocity becomes an issue when people want to compare teams’ velocities or when velocity becomes a target. If you’ve ever seen anyone say, “You can double your velocity!” you’ve seen velocity as a target. I’ve seen managers and teams both think they can create targets this way.

Targets are helpful for archery. They are not helpful when people use them as a measure of capacity.

We use the word velocity as a measure of progress over time. We might even think about our speed on the highway. When we measure speed, our distance measure doesn’t change over time. A mile is always a mile; a kilometer is always a kilometer. The distance is a static measure. And, we know how to convert between the two measures (miles and kilometers).

Story points can change over time. The story points a team can deliver change when the stories change. If a team was accustomed to delivering 30 story points in an iteration for one area of the product, they might not even know how many story points they can deliver in the same time for another area. If the product owner changes how he or she writes stories, the story points might change. And, if the people on the team change, the team might deliver a different number of story points.

That means that we don’t have a static measure of distance when we use story points. We have a relative estimation given certain conditions. When those conditions change (the area of the code and tests, the team makeup, the quality of the existing code and tests), our relative estimation can change.

That’s why velocity is a relative estimate of capacity for now, and not forever.

Estimating with story points might help your team understand their capacity. However, if you measure points, you will get points. You might not get features.

Measure Finished Stories, Not Story Points

Jim is an agile project manager in an organization that wanted more features faster. He had a mandate from his managers: start releasing features faster! Jim’s management wanted to see a more reliable flow of features out of his team so they could recognize revenue faster.

Jim’s team measured their story points. They had a little trouble because they had that hockey stick profile of finishing the points later in the iteration.

Figure 1© Johanna Rothman, from Create Your Successful Agile Project

Jim realized that they had stories in progress at the end of the iteration, so he asked the team to add a story measurement to their graph. Adding finished features (stories) to the graph showed them what was happening:

Figure 2© Johanna Rothman, from Create Your Successful Agile Project

The team thought they had two-point stories. If so, they would have completed five stories in the iteration. Instead, they only finished two stories, which meant the stories were roughly five points each. When they saw this chart, and the fact that they finished more story points than stories, they realized their stories were too large.

In addition, they hadn’t realized they finished stories so late. They finished the first story on Day 9 and the second story on Day 10. They also had several stories in progress that they didn’t complete.

For Jim’s team, measuring what was valuable to the customers (features) and not points helped them realize why their managers wanted more throughput. Now they needed to understand what prevented them from delivering more finished stories.

Measure What You Want to See More and Less of

Jim wondered about all those story points that weren’t part of finished stories. He knew about cumulative flow (more on that later), but he wanted to understand the organizational system that prevented the team from releasing more stories faster. He decided to look at cycle time and lead time.

Normally, cycle time is the time when the team takes an item from their Ready column until they mark it as Done. Jim was sure that this team had dependences on other people or teams inside the organization. Jim asked the team to explain what happened. He built this chart from right to left as they described what happened.

Jim asked, “How long does it take for an item to go from “Done” to “Deployed?” The entire team agreed that it took no more than a day. Jim labeled this as T5 and wrote “Maximum of 1 day.”

Then, he asked, “How do you get to done?”

Everyone started to talk. He waited a minute and then said, “Can someone tell me the flow of the work, so I can understand it?”

Susan, a senior developer said, “I’ll start. We take an item off the Ready column and put it in our “In Progress” column. We work on it until we think we’re done. But, and this is a big but, we’re not allowed to do any UI. We have to wait for the UI team to assign someone to us.”

People around the room nodded. Susan continued. “Then, we have to request UAT (User Acceptance Testing) and wait until they take the feature.”

Jim asked, “Every time?”

Susan nodded. “Every single time.”

Jim created the chart below and asked, “Is this right?”

Everyone agreed it was.

All of these interdependencies took lots of time. Jim asked the team to explain the last few weeks of work. The team had completed five stories in that time, so he asked them to explain the time spent in each part of the flow.

Jim’s managers wanted the team to release features faster. However, the time that the team spent waiting for UI and UAT meant that the team’s velocity and finished story points might not mean much.

This team had no control over the request and response from the UI team. They didn’t know how many times they might have to cycle between their work and the UI work. They couldn’t predict any of that time.

In addition, the UAT time was at least two days to request the UAT and then whatever time UAT took. If UAT found problems, the work might return to the team and possibly to the UI folks. Except, the original UI person might not be available, so the team might have to work with a new UI person.

The team’s cycle time wasn’t just the In-Team Time—the cycle time was the sum of all those times. The team was surprised at how long things took. Those five stories had an average cycle time of 6.6 days, more time than anyone expected.

Armed with this data, Jim advocated for and received their “own” UI person on the team. That changed their cycle time to an average of 2.5 days/feature. The team didn’t have to wait for the UI people to assign someone to the team. They were able to change how they worked in the UI. They tended to swarm on UI stories to start the UI as soon as they started on a story.

Now they could show management that their 2.5 days/feature was still dwarfed by the overall cycle and lead time because UAT and Deployment were not part of the team.

Jim’s team prompted the organizational discussion of how to integrate UAT and Deployment into the team. However, no one is discussing their estimates anymore. Everyone is focused up a level, for how the organization tests and releases new features.

Measure Project Progress

While the organization discusses what to do about the overall problem of testing and releasing new features, the PO had been adding stories to the overall roadmap for the project.

Jim understood the PO’s need to add more stories. However, it was going to take them longer to finish the project if they finished everything the PO added. He needed a way to show management that even though the team was progressing, the added features would change their release date.

He chose two charts: a project feature burnup and the product backlog burnup chart.

Figure 3 © Johanna Rothman, from Create Your Successful Agile Project

The Features Complete chart shows the burnup of the completed features, the burnup of the total number of features, and the burndown of remaining features.

Jim and the PO counted the features. They did not account for complexity or estimates—they counted the total number of features.

Some of the feature growth was because the PO understood the features and realized he had compound features that he could decompose into several stories. Jim and the PO counted each story as a feature.

Part of the growth was because the PO added more features. It was easier to see that in the Product Backlog Burnup chart:

Figure 4 © Johanna Rothman, from Create Your Successful Agile Project

The team measured their progress every iteration. Feature Sets can grow as long as the project continues and the PO decides to add features to the product backlog.

Each of these Feature Sets continues to grow. Because the PO and the team can see which feature sets are growing when, they can ask questions:

• Is this part of the MVP?
• How little of this can we release now and know that we have done what we need to do for our customers?
• What else do you anticipate we need to finish this project?

Your team might have different questions, but Jim’s team asked those questions.

Jim’s team wanted to verify that the customers needed these features for this project. Sometimes, feature sets shrink because the story or feature no longer has value to the customers or is too low enough in rank to keep in this project.

Change the Conversation to Empirical Measurements

Jim changed the conversation from prediction of capacity (velocity) to empirical measurements. He was able to change the conversation inside the organization about what the team could do to what the team had finished, and the impediments they discovered.

All by changing their measurements.

About the Author

Johanna Rothman, known as the “Pragmatic Manager,” provides frank advice for your tough problems. She helps leaders and teams see problems and resolve risks and manage their product development. Rothman is the author of more than ten books, including: Create Your Successful Agile Project: Collaborate, Measure, Estimate, Deliver, Agile and Lean Program Management: Scaling Collaboration Across the Organization, Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, 2nd ed., See more of Rothman’s writing at jrothman.com.