Agile Introduction: are you a Laggard?

The purpose of this white paper is to portray the worldwide state of agile adoption for our readers. While much has been written about the strengths and weaknesses of the technology, little data has been published to show how widely agile methods are used. This paper corrects that by providing data from our databases for public consumption. As shown in Figure 1, agile methods have become the dominant software development paradigm used throughout the world based on data from 330 organizations. Some of these organizations are offshoots of the 120 firms and government organizations from which we have received data. Figure 2 summarizes which agile methodologies are in use by these organizations. As many said that they were using a hybrid approach, i.e., one that combined agile with traditional concepts, we have included their response and categorized them as either hybrid or hybrid/lean (agile combined with lean).

Figure 1: Worldwide Agile Introduction (No. of Organizations)

Legend

• EA – Early Adopters (31) – introduced agile methods companywide during period from 2004 to 2009
• EM – Early Majority (166) – introduced agile methods companywide during period from 2009 to 2014
• LM – Late Majority (77) – introducing agile methods presently
• Laggards (46) – considering, but not currently using agile methods

We used a technology introduction life cycle model developed by Moore¹ to identify the stage of adoption firms were in. As part of this use, we assumed a technology like agile would take 18 years + 3 to get fully adopted based on studies done by Riddle and Redwine². The four stages of technology adoption are illustrated in Figure 3 and are as follows:

• Early adopters – organizations in this stage took agile in its early stages and put it to work operationally to derive maximum benefits. They took some risk with the methodologies and worked with the vendors to refine them. The primary challenge was stabilizing the methods.
• Early majority – organizations in this stage jumped on the bandwagon as agile started to receive good press and adopted the methodology. It should be noted that the early adopters during this stage fanned the methodology out and put it to use organization-wide. Again, the primary motivation was the benefits. The primary challenge was fan-out.
• Late majority – more conservative organizations now got on the bandwagon with agile. They waited until they thought all was stable and then rushed to take advantage of the benefits. Most used a normal change management process where they first got executive support, then trained, piloted, adapted and rolled out a methodology with vendor help, and finally tuned their products for widespread organizational use.

Figure 2: Worldwide Agile Methodology Usage (No. of Organizations+)

Notes

+ The percentage distribution of organizations by geographical regions is: USA and Canada (50%), Europe (30%), Asia (17%) and Australia (3%)

• Laggards – those organizations who for whatever reason stick with their existing way of doing business. Most are considering moving to agile. However, they might have some good business reason for sticking with their existing ways of doing business. Independent of the reason, they are not currently tapping the benefits of agile adoption.

Figure 3: Stages of Technology Adoption¹

Notes

¹ Geoffrey Moore, Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customers, Harper, 1999.

The chasm represents the time it takes to get the technology adopted by the early majority. This is the difficult time because it is here where effort is needed to get the vast majority of users to stop what they are currently doing and move to a new way of doing business.

Government versus Industry Agile Use

Table 1 looks at government versus industry agile use. As expected, government lags. However, the surprise here is that government in Europe, primarily the United Kingdom, embraced agile methods at least five to eight years before the agencies in the United States made a like commitment. The Table also shows based on the data supplied, industry is past adoption. Government lags behind by between 5 to 7 years. It also shows that over half the organizations in industry are well beyond the early adopter stage of the technology introduction life cycle.

Please note that we did not have enough data to complete the table for all geographical areas. For instance, we included only the United States instead of the Americas because our data from Canada and Brazil was lean. We also stripped out the United Kingdom from our European data to illustrate its agile leadership position in this theatre of operations.

Figure 3: Stages of Technology Adoption by Industry/Government and Geographical Area as a Percentage of Adopters (no. included in parenthesis)

Legend

+ Numbers do not include the UK * Numbers include organizations for which we have data

EA – Early Adopters

EM – Early Majority

LM – Late Majority

Laggards –considering agile

Lessons Learned by Early Majority Adopters

The experiences of those in the early majority paves the way for others who are interested in inserting agile methods in their organizations. Their experiences are valuable because they have gone through the transformation process, fanned out the technology, and put the methods that they finally adopted to work in their development operations. When first adopting, the following success tactics³ are deemed important by those embracing agile methods:

• Define your transformation goals and expectations, i.e., is agile for use in one organization or enterprise-wide, and develop related measures of success.
• Tie your agile transformation effort to your firm’s business goals and tailor your measures of success accordingly.
• Educate your management, i.e., from executives on down; get their support; and then engage them and keep them informed throughout the development.
• Educate your entire organization, including your customers, as to why you are changing.
• Select the agile method that you will use, train your people in its use, and tailor it as you use it on one or more pilot projects; i.e., at least one pilot for each organization that will use it.
• Have the team that pilots the methodology define a set of rules, metrics and processes for its use. Make sure that this extra effort does not weight them down and cause them to fail.
• Develop a train the trainer program for use during fan-out as part of your pilot project. Make sure that you use real project artifacts and experiences as examples in the courseware.
• Refine the methodology as you tailor it for use on the current and next projects.
• As the pilot nears its end, develop a plan of action and milestones aimed at fanning the methodology out organizationally or enterprise-wide. If need be, create a separate team to ready for the transition by creating the infrastructure and prerequisites for success.
• Publish a retrospective when you complete the pilot to ensure you capture lessons learned.

When fanning the technology out organizationally or enterprise-wide, here are a few things you can do to speed up the process

• Start small, build big. The sum of a series of a few small wins often turns into a big one.
• Fan agile savvy and experienced people from your pilot projects out to lead the next projects and train the staff.
• If using Scrum⁴, build up a cadre of Scrum Masters so you can use them effectively.
• Work with the product owners and help them understand their role; i.e., they are there to set priorities and act as the voice of the customer, not act as a project manager.
• Embrace test-first concepts and automate your regression testing; i.e., those tests that you baseline and use to revalidate your builds each iteration, whenever possible.
• Look for risks at your daily standups and then prioritize and manage them based on their consequences.
• Enlist the aid of your quality assurance personnel as you look for opportunities to cut waste. Understand when push gives to shove, quality stands out as the discriminator.
• Use the numbers; i.e., per the metrics and measures that you defined, to manage.
• Build momentum for change through people by spreading the word via networking, newsletters and publicity events that it works, is good for the company, is good for the employee and is fun.

When in development operations with mature agile processes, the following ideas may help:

• Continue to refine and reinvent yourself. Else, the momentum will die and your initiative will end. Use retrospectives as a learning tool. Reinvention can be simple. Embrace the next best thing. For example, if agile, take on lean. They are compatible and complementary.
• Modify your infrastructure and institutionalize your processes. This doesn’t mean setting up a process group and conducting appraisals like under CMMI®⁵. Instead, create a handbook and training. Establish roles for support groups and career paths for new jobs like product owner and Scrum Master. In addition, put quality assurance personnel to work helping to engineer quality into the products rather than trying to inspect it in.
• Finally, as agile is scaled, look at redefining project management roles and responsibilities. Make them the synchronizers and coordinators rather than the overlords. Look at methods like Disciplined Agile Delivery (DAD)⁶ for hints at realizing balance between agile and more traditional methods if this is what you think will work for you.

How Laggards Can Catch Up

Being behind isn’t always a bad thing. By being smart, I believe that you shave the amount of time and effort required to make the transformation to agile. Instead of going through the entire process, you can take the following shortcuts to move to agile in two to three rather than four to five years:

• Acquire rather than build your primary agile capabilities by buying a firm putting its people to work as your primary agile workforce. This gets rid of a year of piloting and around 10 staff-years of effort assuming that you can quickly adapt the methods that they have embraced to your line of business. To move out smartly, key personnel clauses and bonuses may be necessary. These contractual clauses are aimed at retaining the key resource you are after as part of the acquisition, agile skilled and experienced people.
• If you cannot buy a business, hire a qualified firm to help you. This is a good tactic when you are developing products on contract and can pass off the costs to the third party paying the bills. Make sure that the candidates will take your best interests to heart and not treat the engagement as a money stream. Use win-win terms to make the partnership rewarding for all parties. Make them put up something to gain. For example, ask them to provide free training and/or tools as a consideration to show that they are truly committed to partnering. Provide them with referrals and testimonials as part of your commitment to them.
• Avoid consultants. While they can help in certain situations, they often bleed the effort for their own personal gains. Go with an agile consulting firm as an alternative. If they work out, buy or partner with them.
• Focus on the bottom line. Gather the data as part of your transformation to keep the critics at bay. Use the numbers to show advocates and critics alike that agile is the better way. Hard data has always won arguments for me. I am sure that it will work for you as well.

Summary and Conclusions

In this white paper, I have shown that agile is here. The data shows that it is no longer an emerging technology. While government lags, that does not mean that they should standby. In summary, those organizations that are just starting on their agile path are behind the adoption curve. They need to move out aggressively or they will find it difficult to get their job done.

To help, I have also provided lessons learned for those on their journey to disciplined agile use. Both early adopters and those fanning the technology out will find that the suggestions that I have offered will make their lives easier.

Finally, for those considering moving agile, what’s keeping you? You are behind the competition. You are a laggard. You need to catch up. Else, the agile freight train will either leave you behind or run you over. Again, I have made suggestions that you might take to shave the amount of time and effort you will need to expend to catch up. So, do it.

References

1. Geoffrey Moore, Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customers, Harper, 1999.
2. William Riddle, “The magic number eighteen plus or minus three: a study of software technology maturation,” ACM SIGSOFT Software Engineering Notes, Vol. 9, Issue 2, April 1984, pp. 21-37.
3. Donald Reifer, Software Change Management: Case Studies and Practical Advice, Microsoft Press, 2011.
4. Mike Cohn, Succeeding with Agile: Software Development Using Scrum, Addison-Wesley, 2009.
5. B. Chrissis, M. Konrad and S. Shrum, CMMI®: Guidelines for Process Integration and Product Improvement, Addison-Wesley, 2003.
6. See the following web site for description.

About the Author

Donald Reifer is one of the leading figures in the field of software engineering and management with over 40 years of progressive experience in industry, government and academia.  He founded several software firms, built businesses, was a Project and Program Manager in industry, served as a visiting associate in academia and led national software initiatives as a Senior Executive Service Official in government.  He has acted as an advisor in management, measurement and change to Fortune 500 firms and government organizations worldwide.  Currently, as President of Reifer Consultants LLC, he is helping clients put agile methods to work operationally.  Reifer has over 200 publications including 10 books.  His many honors include the AIAA Software Engineering Award, the Department of Defense Medal for Outstanding Public Service, and membership in Who’s Who.  He holds a BSEE from New Jersey Institute of Technology, a MSOR from the University of Southern California (USC), and a Certificate in Business Management (MBA equivalent) from University of California at Los Angeles (UCLA).