It’s iteration 10 and your project is starting to slow down. For the past few iterations the team hasn’t been able to complete quite as many stories as they did in the past. In addition, more bugs have been recently found both in the new stories and also in regressions. The manager knows that the team members haven’t changed and they’re still working the same hours. The customer is asking “What happened? Is the team still working hard?”
Many Agile teams achieve productivity improvements of 150-500%[1], yet your projects are seeing improvements in the 20-40% range. What happened?
There isn’t one big problem; instead, there is a myriad of small issues. Sometimes these were expedient shortcuts (some changes where the developer didn’t have time to tidy up), sometimes the developers just don’t know the language. Other issues are bits of code that have grown like a bush and need severe pruning. All of these amount to Technical Debt.
What is Technical Debt?
It’s all “those internal things that you choose not to do now, but which will impede future development if left undone” [Ward Cunningham]. On the surface the application looks to be of high quality and in good condition, but these problems are hidden underneath. QA may even tell you that the application has quality and few defects, but there is still debt. If this debt isn’t managed and reduced, the cost of writing/maintaining the code will eventually outweigh its value to customers.
Technical Debt is like a credit card that charges a high interest rate, just leaving the team with an outstanding balance cost. In this case, the costs are represented by time and effort needed to work around the problems. The longer the team takes to pay off the debt, the more interest is accumulated (in the form of additional workarounds) and the higher the costs for the business.
In addition, it has a real financial cost: The time developers spend dealing with the technical debt and the resulting problems takes away from the time they can spend doing work that’s valuable to the organization. The hard-to-read code that underlies technical debt also makes it more difficult to find bugs. Again, the time lost trying to understand the code is time lost from doing something more valuable.
Why do we Accumulate Technical Debt?
At first it’s faster to write code without tidying up, without writing unit tests, without doing Test Driven Development, so the team churns out more stories. The problems don’t usually show up quickly and doing the right thing takes more time, especially in the beginning.
Where does it come from?
- Inexperienced Developers – on some projects there are developers writing Java/C#/Ruby who have never had any training or who don’t know what good Object Oriented code is. As a result, they continue to write code that is appropriate for the language they used to i.e. Visual Basic, etc.
- Deadline Pressure – both explicit from management/customer and implicit. “We promised the following stories for the iteration/release”. The team can see that they won’t deliver on the commitment for the release/iteration, so they do the expedient thing. “We have to get it done; we can’t afford taking the time to tidy up. If it’s not a feature/bug, it won’t get done.” Unfortunately this view can get a lot of support from the management, when they’re not aware of the cost.
- Untidy, hard-to-read code. When the existing code in a method or class is difficult to read, the next developer who works on it feels less compelled to write clean code himself. So a small mess grows into a bigger mess every time someone touches it.
- Specialisation – leads to the mindset: it looks like bad code, but it’s not in my area so I can’t/won’t change it. In addition, in the world of specialised code, developers often don’t feel qualified to make the change.
- Over Complication – Developers are often tempted to design solutions to problems in advance of their needs. However, in many cases the program goes in a different direction and the code is written without benefit. Or the code doesn’t really fit the need because it had been written before the problem was well understood. In either case, overdesign takes extra time and produces code that is either unused or doesn’t fit the needs of the project.
- Bad Design – Some solutions are just poorly designed. Building on top of poorly designed code, rather than fixing the problem, just makes the problem worse.
Solving the Problem
There isn’t one single thing that you can do to solve this problem and the solution will take more than a few iterations. Instead you need patience and multi-pronged approach.
Planks in the solution:
- Get the developers some basic training in the language and teach them the principles of Object Oriented, to bring their understanding up to a minimum threshold. To be successful and effective for more than a few weeks, the training needs follow-up and support from people who know the area.
- Tell the developers and managers that the current problems are costing the business money. This is especially important as it will make clear the value of solving the problems. Make it clear that the Management appreciates the problem and is prepared to start paying down the debt. Repeat support for this on a regular basis so that teams trust this statement.
- Provide training that covers Code Smells, Refactoring, Unit Test and Test Driven Development. Use a combination of classroom sessions, web-based material and books.
- Give them time during office hours (2hrs a week is the minimum amount of time needed to achieve the goal), to study and practice their skills. Practice should be on throw-away code so they’re free to try and experiment with things that they otherwise wouldn’t do in their shipping code base. Time to practice and study is perhaps the most important recommendation of all. Without giving people time, the business will never see what is really possible with Agile. One way to organize this is called a Coding Dojo (for details see: TDD Randori Session and TDD Randori Workshop)
- Use Tools (Static Analysis, Unit Test, Continuous Integration, Automated Acceptance Tests) to help the teams find, reduce and measure their debt load. Measurements should be captured at the team level but not shared with the management. Teams need to know that they will not be rewarded/punished as a result of these measurements. If they’re reported to and tracked by the management, developers will soon find ways to game them and they will lose their real value to the business.
- Reward people for having completed their training and demonstrated a minimum level of proficiency in the skills that they’ve acquired. Reward them to acknowledge recognition or give them small gifts to avoid tying them directly to money.
- Create regular biweekly lunch and learn sessions that cover technical material. Use these sessions to stimulate discussions among the developers. Supply lunch, as it will improve attendance. In addition, it’s important for someone from Sr Management to attend from time to time, thus making it clear that there is ongoing support for improving skills.
- Maintain a Technical Debt Backlog – anytime a developer notices some technical debt that they can’t address immediately, get them to write a Technical Debt card. The developers should prioritise the debt and spend 10-15% of every sprint paying it down. On most projects anything less will just prolong the problem.
As it stands, your business has both a problem and an opportunity. The problem: the code bases of your projects are accumulating technical debt and they have already started to slowdown. This problem is costing the business money today as it becomes more and more difficult to make the changes and fix bugs as requested by the customers. The opportunity: improve the developers’ skills; make it clear that management support and the code quality will improve and the bug count will go down. In time, this will increase the capability of the development teams.
[1] RPM Software – by Robin Dymond; Embedded Agile Project by the Numbers With Newbies (2006) - by Nancy Van Schooenderwoert; How Agile Projects Measure Up, and what it means to you (2008) by Michael Mah