Key Takeaways
- Technical debt impacts the whole company, but it especially affects engineers as more tech debt means more bugs, performance issues, downtime, slow delivery, lack of predictability in sprints, and therefore less time to work on new exciting projects.
- Engineers waste seven hours per week on technical debt which is often invisible to stakeholders. Speaking to non-technical stakeholders using business terms is the best way to make technical debt easily seen.
- Mess isn’t always bad, just as technical debt doesn’t always mean there’s a problem. The key is to fix the right amount of technical debt at the right time.
- Companies should aim to proactively and continuously address technical debt. Continuously addressing technical debt gives teams an opportunity to deliver value regularly and empowers engineers to make decisions.
- In the future, technical debt will become less of an engineering problem and more of an important business prerequisite that helps with delivering more value to our customers and the business.
In this article, three experts- Alex Omeyer, Nicolas Carlo and Maarten Dalmijn- discuss some of the key findings of the “State of Technical Debt 2021” report including the impact of technical debt on engineering teams, the pros and cons of dealing with maintenance work continuously, the future of technical debt and what each engineering teams can do to communicate the importance of dealing with technical debt to companies’ leadership.
State of Technical Debt survey
The “State of Technical Debt 2021” report was created by Stepsize, Voice of the Developer SaaS company, after surveying 200+ engineers and engineering leaders. The report shows the impact of technical debt on engineering team morale, velocity, and customer experience. The key findings include:
- 52% of engineers believe that technical debt negatively impacts their team’s morale. Over 60% of engineers think that tech debt causes bugs, outages, and slows down the development process.
- The average engineer spends seven hours per week (roughly one day) dealing with technical debt.
- The majority of engineers (66%) believe the team would ship up to 100% faster if they had a process for technical debt.
- Despite the fact that engineers are convinced about the negative impact of technical debt on the business, 58% of companies still have no process for managing technical debt.
Why does technical debt impact engineering team morale?
Imagine you’re a cook having to prepare many different dishes every day in a messy kitchen full of dirty pots and pans. You’re unable to easily find the cooking utensils you need and there is no space to prepare your ingredients. Unfortunately, there’s no time to clean-up, and hence you’re condemned to work in this terrible environment that slows you down and makes cooking difficult.
Would you enjoy being a cook in such a kitchen? Imagine how frustrating it would be having to prepare a dish under such conditions every hour of your working day with inevitable delays and unhappy customers.
As Dalmijn points out, “That’s the same situation a software developer is in when working on a system with a high amount of technical debt. The mess slows developers down and sometimes even prevents them from doing their actual job. Technical debt can make it difficult to be proud of the work you’re doing, as a lot of time is lost dealing with annoying obstacles. You’re like a football player that has to score goals only with their head and unable to use both their legs to shoot and pass. It’s no wonder that technical debt causes frustration.”
The biggest problem is that unlike a dirty kitchen, technical debt is mostly invisible to our non-technical stakeholders. They can only see the slowing down effect it has, but when they do, it’s often already too late. It’s all about new features, constantly adding new code on already fragile foundations.
Another problem is that too much tech debt causes engineering teams to be in fire-fighting mode. Tech debt impacts the whole company, but for engineers, more tech debt means more bugs, more performance issues, more downtime, slow delivery, lack of predictability in sprints, and therefore less time spent building cool stuff.
Engineers spend seven hours per week on technical debt
“I’m not surprised that it’s the case,” says Carlo. “I believe that the average developer wastes even more than seven hours a week on technical debt. It’s clear that developers spend most of their time reading/debugging/testing existing code.”
A study from Microsoft published in 2017 concluded that 58% of developer time is spent on code comprehension, especially when software is shipped to production and you now have to maintain it. It’s something developers would intuitively recognize themselves: the less you’re familiar with the code, the longer it takes. The more surprises are in your way, the longer it takes. The harder it is to reproduce all scenarios, the longer it takes to get it right, and that’s where good automated tests really shine.
Reading code is harder than writing code. A high amount of technical debt often means you have more code you need to read, and the code is more difficult to understand. Technical debt also means people on your team will mostly understand smaller parts of the system without seeing the big picture.
The time lost fixing this problem is often invisible to non-technical stakeholders, and the productivity gain is invisible until another change is made, which can be many months later. A clean kitchen doesn’t speed you up until you need to cook there. We rarely hear a developer say: “Boy, this change was easy because we spent some more time fixing technical debt last time!” The productivity gain is not obvious, hence, we don’t get extra time allocated to fix technical debt.
Companies could ship up to 100% faster if they had tech debt under control
Controlling technical debt is a prerequisite to delivering value regularly, just like an organized and clean kitchen is a prerequisite to delivering delicious food regularly. That doesn’t mean you shouldn’t have technical debt. You will always have some mess and that’s healthy too. The goal isn’t to have zero mess; the goal is to get rid of the mess that slows you down and prevents you from running a great kitchen.
“Saying that companies could ship twice as fast is the wrong way of looking at it, in my opinion. Nobody asks if it would be beneficial to have a clean kitchen without a mess to serve a restaurant full of busy customers who will complain if the preparation takes too long or their dish arrives cold,” says Dalmijn.
However, mess isn’t always bad. Sometimes mess is also necessary to stay in the flow. When you’re doing something new, e.g. constructing an Ikea bed, you will make a mess while putting it together. If you immediately clean up this mess while you’re busy doing the work, you kill all your flow. It’s about fixing the right amount of technical debt at the right time.
Dealing more effectively with technical debt is key. This means, not extending the same amount of worry to technical debt in areas of your product that are rarely used or changed. It also means that your developers shouldn’t need to justify the fixing of technical debt. Just like cleaning the kitchen is part of cooking, fixing technical debt is part of coding.
“I was surprised to see such high numbers,” says Omeyer. “But I can empirically validate them. Think of it this way: how many times have you estimated a feature would take the sprint, and it ended up taking two, or three? Now imagine this at the scale of an entire company and the ramifications it would have. It’s not hard to believe that companies who truly have a good handle on their technical debt ship twice as fast as those who don’t.”
Carlo believes that some companies could ship twice, thrice, or way faster. “I’ve experimented with different approaches to code quality. Although they wouldn’t be equivalent, I could tell the difference in productivity when I was able to ship a meaningful change to prod within a day, versus having to wait a week. That’s actually a good exercise to auto-evaluate: how long would it take for a newcomer to set up the project, do a change, test it, and get it deployed to production? The answer can scale from a few hours to multiple weeks for projects of similar size.”
The pros and cons of dealing with maintenance work continuously
Dealing with technical debt continuously is crucial for most companies, but especially for those who have achieved market-fit and aim to deliver quality products to their customers. It’s not always easy to achieve, and there are both positive and negative sides to this approach.
Pro’s:
- Agile way of working. Continuous maintenance work is a prerequisite to deliver value regularly. Technical debt that requires a separate project to get back under control means you won’t be able to deliver value regularly.
- Increased developer happiness. Nobody likes working in a mess, and a project to deal with technical debt means there is a visible mess that people had to deal with for quite some time.
- Higher business value through improved time-to-market of new features. Technical debt slows you down, and the true cost isn’t in the developer hours you lose, but in the fact that there is a cost of delay in putting a new feature in the market. The cost of that feature being delayed can be many orders of magnitude larger than the loss of developer productivity. That’s the true cost of technical debt, not all those hours.
- You don’t need to ask permission. If you play boyscout and you clean up the code, you need to change to implement a feature as part of implementing a feature, then you don’t need to ask for a budget for it. It’s part of the job, you’re the expert.
- It’s actually easier to get started. Even if you’re not the tech ldead, you can start on your own by keeping track of little things you want to clean up and address them as you go (e.g. rename some variables, extract a function…)
Con’s:
- It’s harder to address big pieces of technical debt that has accumulated over time (e.g. upgrade the ORM library that is now four major versions behind). However, once the team is experienced enough, they’ll figure out a way to split the work in smaller chunks. This can generally be done continuously too, although it requires having a clear vision on the end goal—which is easier to do with a project for most teams.
- It’s extremely easy to let normal business pressures disrupt continuous maintenance work. And if engineers don’t get rewarded for improving the codebase (e.g. if only new features are celebrated or result in career advancement), they won’t prioritise these properly.
- Continuous work is invisible and won’t bring praise. In my experience, many companies celebrate firefighters instead of fire-preventers. How many times has an employee been praised for working extra-hours to solve some production issue? How many times has an employee been praised for preventing issues from happening in the first place? Continuous work goes unnoticed. The perk is that you don’t need to ask permission and you don’t have to work extra-hard to fix fires. But you won’t get celebrated for that. That’s something I encourage tech leads & CTOs to change: celebrate preventing fires!
The future of technical debt and what can be done today
“I wish more and more developers would be exposed to working effectively with existing (legacy) code. We don’t teach such skills. Just like we don’t really teach “how to read code”. The mantra is ‘try things, experiment and you’ll learn’, which is fun until people rely on these experiments in real-life,” says Carlo.
As the industry matures, we’ll keep raising the quality bar. Writing maintainable software is hard. It’s a challenge for companies when developer turnover is under two years (knowledge loss!) and the more experienced developers stop coding at some point. In the future, we'll see an increased interest in these topics as we get more and more programs to maintain.
These three steps can improve the situation if you have a lot of technical debt:
-
Have the engineering team proactively identify technical debt. Make it visible first, then you can start tackling it.
-
Speak to non-technical stakeholders using business terms. We don’t address technical debt for the sake of it, but because we want to deliver features faster. “We want to reduce our TTM (Time-To-Market)” or “We could reduce our turnover if we invest some time in code quality” among other examples. Bonus points if you can get actual numbers based on facts. Extra bonus points if you tie it back to money: ”In the last three months we spent 42% of our development time on bug fixing, costing us 1M$ USD”.
-
Join these communities to exchange knowledge with peers:
Over the last few years, we’ve seen an explosion of new developer-first tools. At the same time, we’ve seen the rise of the developer, particularly in the power they have to choose their favourite tools.
Companies should aim to proactively and continuously address technical debt. The only way to do this is through refactoring. Agile organisations are constantly learning and evolving. If they don’t invest in continuously improving their codebases, their code will remain static and will become a hefty liability in a matter of days.
Investing in continuous codebase improvement will lead to fewer bugs, less downtime, fewer support requests, more predictable and faster delivery, and happier (read retained) engineers.
“I hope technical debt stops being a developer thing that is invisible to business stakeholders and becomes an important business prerequisite that helps with delivering more value to our customers and the business,” says Dalmijn.
Here’s an example of the invisibility of technical debt. A tech company organized a marketplace to restructure different development teams. Every person could decide which team they wanted to work on. There was one team nobody wanted to work on, and it wasn’t because of the people. The team would be working on a legacy system with technical debt.
The company decided to pay a premium to make it more attractive for developers to work on this team that had a system with a high amount of technical debt. Now people were prepared to join that team. As a result, there now was a clear business case to fix the technical debt. Technical debt was now visible and had a monetary value. Suddenly it became a priority to address it. Nobody really cared about it before, even though it had the same negative impact on the business and morale.
Developers already care; now it’s time to get everyone else on board to care about it too.
About the Authors
Alex Omeyer is a co-founder & CEO at Stepsize, a SaaS collaboration tool for engineering teams to manage technical debt. Omeyer spends most of his time speaking to the best software development teams in the world about how they handle technical debt.
Nicolas Carlo is a web developer who loves building communities and maintainable software. He organizes the "Software Crafters Montréal" meetup, develops VS Code extensions, and shares tips & tricks for working with legacy code in the Understanding Legacy Code blog.
Maarten Dalmijn is the head of product at Rodeo, top writer on product management with Scrum, an ambassador and editor at Serious Scrum, and a writer of an Agile Product Management blog.