AgileEVM: Measuring Cost Efficiency Across the Product Lifecycle

AgileEVM is an adaptation of the traditional project management practice of measuring actual value of integrated cost, schedule and scope against a baseline plan using Earned Value Management (EVM) metrics. The methods of measurement have been adapted to easily fit within the Scrum project management framework. AgileEVM can be used both in conjunction with traditional EVM metrics across the entire product life cycle to measure cost efficiencies. This is an important addition to an Agile program management structure - giving important early warnings of trends across the entire product life cycle.

Because the AgileEVM method derives the same metrics as traditional earned value management techniques, the two can be combined in a mixed methodology program to give results at the program level. For example, the earned value derived from an Agile project can be combined with the earned value derived from a traditional plan driven project within a program or portfolio, resulting in the total earned value for both. By providing the cost efficiency information across all types of projects in a program, AgileEVM helps to spot potential risk areas earlier and gives metrics analysis information to include in decision making.

Using AgileEVM at Info Tech, Inc.

AgileEVM was initially developed in response to a need for specific performance information on a fast moving project using the Scrum framework. The team involved had decided to move to one week sprints for this release. The final release plan was fairly fluid, with the Product Owner identifying new features every week that were to be included in the release. Adapting the traditional earned value management metrics to this Scrum project allowed the ScrumMaster to accurately predict the impact of these requirement changes, despite a fluctuating velocity, on budget and schedule for the release. The ScrumMaster was able to then share this information with the Product Owner and the team, giving them information to help make additional decisions.

Two years after implementing Scrum and AgileEVM on a single project, Tom Blackburn, program manager at Info Tech, Inc, a market leader in transportation construction management software, now utilizes the earned value data in a dashboard to communicate important budget and schedule information on projects in his portfolio.

Company product leaders use the results of this information in conjunction with the burndown data as input to product investment decisions. In addition, the earned value metrics are posted in the co-located team rooms together with the burndown and Agile metrics for every sprint. The result is that teams can effectively share data with related product teams resulting in improved communication across the product life cycle.

"We realize that one of the key benefits of using AgileEVM is using the data in conjunction with the burndown; it gives us advance warning of a trend. The more information we have: the better decisions we can make" said Tom Blackburn, "I track the value of the CPI (Cost Performance Index) and SPI (Schedule Performance Index) over time as a means to allow my teams greater insight into the project."

The AgileEVM method provides easy cost tracking using information that is already accessible by using this methodology. AgileEVM links actual work completed with budget and schedule in a clear and concise manner. This helps support release planning because trends can be seen over time, resulting in deeper understanding around how each team is performing.

Expanding AgileEVM to the Program Management Level

Earned Value Management (EVM) metrics on large projects or programs have been used as a way to 'roll up' project metrics to the program level. This method directly translates to an Agile program as well. Whether the program uses Agile methods in all phases of the product life cycle,( i.e. development, maintenance and support) or has an mix of traditional and Agile methodologies, EVM metrics can be measured across projects and integrated up into a single set of metrics reflecting progress against a plan at the program and/or release level.

As program manager responsible for Info Tech, Inc. owned products, Blackburn has expanded his use of the AgileEVM techniques from one initial pilot development project to include projects up to the program level for two separate products. In the two years since the technique was first adapted, he has expanded his use of the metrics across Appia® and Bid Express® programs; two software applications used in the transportation construction industry. These programs include projects representing the full product life cycle for each product, from new development to maintenance, support and marketing projects using a mix of traditional and Agile project management methods. He now has a single set of simple metrics for each program for measuring the cost efficiencies of all projects related to a product.

Background on Earned Value Management

Earned Value Management (EVM) is a widely recognized technique of project management. It is the subject of in-depth study by the Project Management Institute's (PMI) College of Performance Management and is now included as a standard in the "Guide to the Project Management Body of Knowledge" published periodically by the Project Management Institute. EVM integrates the areas of technical performance, schedule and actual cost to provide metrics for work actually accomplished. By comparing the earned value (EV) with the planned value (PV) the actual progress on the project is compared against the expected progress. Using this information, metrics assessing cost efficiency and schedule efficiency are calculated. Metrics forecasting the expected cost to complete a project and total expected cost of a project based on past project performance are derived.

Key Earned Value Management Terms Defined

(note: The term value in EVM does not refer to business value. The term refers to value as expressed by the project or program budget.)

Earned Value Metrics

These key metrics are the same measurements in AgileEVM as in traditional EVM. Only the method for obtaining the measurements has been adapted. Agile accentuates the use of rich collaboration and the frequent delivery of fully-operational tested systems, as well as fostering the human component of software development. Agile software development focuses on delivering business value early in the project life cycle that are based on timely customer and market demands. AgileEVM was specifically development to measure and forecast progress on Scrum projects. While the technique is valid on other Agile methods, AgileEVM works particularly well with the Scrum elements of an estimated, prioritized backlog and measuring at Sprint boundaries.

AgileEVM - Adapted for Scrum

In AgileEVM, story points are used as the measure of work planned and work performed; providing the basis for all of the traditional earned value calculations. The other adaptations were to change (more context) how expected percent complete and actual percent complete are calculated. In AgileEVM expected percent complete is derived by dividing the number of complete sprints by the total number of planned sprints. Actual percent complete is derived by dividing the number of story points completed by the total number of story points planned for a release. It is important to note that the story points used in AgileEVM at product backlog level, as estimates of relative size not at the sprint task level.

In order to calculate the initial baseline in AgileEVM five data points are required.

1. The number of planned sprints in the release
2. The length of each sprint in calendar days
3. The number of story points planned for the release
4. The budget planned for the release
5. Start date of the project

Each sprint boundary was established as the 'measuring point' to re-baseline any changes (i.e. work added or removed from the plan) and re-evaluate the earned value results. To calculate the AgileEVM metrics, four data points are required:

1. Number of sprints completed: This measures the expected percent complete when divided by the total number of sprints planned.
2. Net Story points added (this can be a negative number if story points are removed from the release.) This reflects the net change in work planned for the release, in effect rebaselining the release. Actual Cost to date At each boundary we are measuring results to date, therefore we use the cumulative actual cost as of that sprint boundary.
3. Cumulative story points completed This measures the total amount of work completed for the release as of that sprint boundary.

The research paper presented at the Agile 2007 conference provides proof of the validity of the AgileEVM approach. The paper asserts that AgileEVM:

• Is lightweight. AgileEVM leverages already existing work metrics, adding little to no work to the current processes.
• Adds value to Agile teams and stakeholders by providing data for decision making.
• Is cost effective to implement.
• Is at least as accurate as current methods. To review the mathematical proof of this assertion, please review the IEEE published research paper at http://solutionsiq.com/agile/

AgileEVM Equations

About the Author

Tamara Sulaiman is a Managing Consultant at SolutionsIQ where she is focused on coaching teams and organizations transitioning to Scrum. Tamara brings to this work over 15 years of experience in business management and software development. She is a Certified Scrum Trainer (CST) and Project Management Professional (PMP). Since 2003 Tamara has assisted teams in transitioning to agile methods both as a hands-on ScrumMaster and as an Agile Coach. As a Managing Consultant, ScrumMaster, and Project Manager, she is focused on providing value to key stakeholders and customers through effective delivery of software.

Tamara has published author of articles on AgileEVM and is co-author of the original research paper "AgileEVM - Earned Value Management." She is co-originator of the AgileEVM materials and processes that integrate the traditional project management practice of Earned Value Management with the Scrum framework. Tamara is currently focused on Scrum and Agile at the program and portfolio level, and integrating quality and financial metrics in a ‘dashboard’ format for evaluating project and portfolio health.