Q&A on the Book Managing Technical Debt

The book Managing Technical Debt by Philippe Kruchten, Robert Nord, and Ipek Ozkaya provides principles and practices that help you gain control of technical debt throughout the software development process and life of the software product.

InfoQ readers can download chapter 1 of Managing Technical Debt.

InfoQ interviewed Philippe Kruchten, Robert Nord, and Ipek Ozkaya about how software systems accumulate technical debt, the impact of technical debt on the software industry, different forms of debt, uncovering or recognizing technical debt, measuring technical debt, resolving technical debt with refactoring or other techniques, the cost of ownership of architecture technical debt, and adopting DevOps to reduce technical debt in production.

InfoQ: What made you decide to write this book?

Ipek Ozkaya: We have been working on balancing agile software development with architecture thinking. The concept of technical debt first appeared on our radar during those endeavors. In our conversations with development teams we came to appreciate that while many development teams assume they get into technical debt because they follow or do not follow a particular software development process, or because they picked a particular technology, or because their customers or managers mislead them, they do not realize that all systems will eventually have technical debt. We wanted to arm them with tools and practices to help them avoid unintentional technical debt as well as consciously manage the trade-offs when they do take on intentional technical debt.

Robert Nord: Though technical debt is a long-standing problem in software development, the development community lacks a cohesive approach to it. We decided to write this book to address that gap by codifying the principles and practices for managing technical debt.

Philippe Kruchten: If you google the phrase "technical debt", you get 200 millions hits! 90% of them tend to repeat the same thing again and again, and sometimes point to the few good sources. It is hard to find any valuable guidance in the other 10%. So with the help of our friends, and through a series of workshops and conferences over 10 years, we’ve tried to sort it out.

InfoQ: For whom is this book intended?

Ozkaya: Our goal is to provide software developers practical information to start managing their technical debt throughout the software development process and life of the software product, and also educate the business environment and management of these developers as to the danger of technical debt and what role they play in pushing their teams into taking this debt. So the book is intended for both technical as well as the business side of the house in software development.

InfoQ: How do you define technical debt?

Nord: Technical debt is a term that conceptualizes the tradeoff between the short-term benefits of rapid delivery and the long-term value of developing a software system that is easy to evolve, modify, repair, and sustain.

Our definition was devised in a week-long workshop on managing technical debt that we organized in Dagstuhl, Germany, in 2016, building on earlier definitions by Steve McConnell:

"In software–intensive systems, technical debt consists of design or implementation constructs that are expedient in the short term but that set up a technical context that can make a future change more costly or impossible. Technical debt is a contingent liability whose impact is limited to internal system qualities-primarily, but not only, maintainability and evolvability."

Kruchten: Note that we do not include clear defects affecting immediately external quality from this definition. This had been an issue blurring the boundary of technical debt for a few years.

Ozkaya: It is also important to note that the definition highlights the software that needs to change as a consequence of technical debt. We have seen many get confused in enumerating what causes technical debt without concretely understanding where the debt is and how to address it. This definition is intended to draw the attention back to the immediate potential task at hand.

InfoQ: How can software systems accumulate technical debt?

Nord: Technical debt accumulates in the form of interest. There is the cost of changing the design and retrofitting the dependent parts to repay the debt, what we call accrued interest. And there is also a recurring interest, the cost of the effort incurred because of the technical debt whenever the system must evolve.

Kruchten: It also simply accumulates under constant schedule pressure; borrowing more, increasing the "capital" portion of the debt.

Ozkaya: Development teams often do not realize that when you keep changing and growing software atop technical debt, the consequences they need to account for in the long run get messier. I also would advise readers to look into chapter 10 in the book where we give many examples of what causes technical debt - a classic common cause for example is insufficient and ineffective testing that hinders teams from avoiding unintentional technical debt. Other major causes are rooted in the business, arising from a change in context, associated with the development process, and arising from people and team.

InfoQ: How big are the problems with technical debt? What's the impact on the software industry?

Kruchten: The impact ranges widely across the software development spectrum. Most likely organizations witness falling productivity, and decreased quality. Some growing level of dissatisfaction from the developers. This is more likely to happen on large and long-lived systems. A small short-lived system has not had time to incur and suffer from a lot of technical debt.

Ozkaya: Today software is everywhere, it is no different than water or air. So the pervasive impact of technical debt when not managed has a cascading and snowballing impact. We see this in the form of cancelled high profile projects, significant glitches bringing operations to a halt, and in failing systems with lost lives. Software-enabled systems are complex; assuming they will always be correct and without technical debt, vulnerabilities or bugs is naive. However, not managing the debt is irresponsible software engineering.

InfoQ: What's the difference between potential and actual technical debt?

Kruchten: Technical debt is the sum of all these "not quite right things" you did to your system over the years. Actual technical debt is the small subset that is preventing you today from making progress. We advocate recording technical debt similar to how software development teams record new feature requests, bugs, and other issues in issue trackers as a technical debt item. A technical debt item is associated with the current state of a development artifact: a piece of code, built script, or test. What connects this artifact to technical debt is that it makes further changes to the software system more difficult. In this case you have a potential for technical debt, but it will be actual debt only if you have to evolve your system.

InfoQ: How can we uncover or recognize technical debt?

Nord: For many development projects, technical debt is discovered when symptoms of slowing development or defects point to workarounds or "fix me" comments in the code. It is important not to stop at the symptom, but to trace to the underlying software artifact so the technical debt item can be described and managed just like other software development issues.

Kruchten: Over the last 10 years we’ve seen more and more tools - static analysers - which help point at technical debt. But not all technical debt can be detected automatically. The number one step in recognizing technical debt successfully is to empower the development teams to concretely and openly share technical debt when they see it. Developers are closest to technical debt as they touch the code everyday.

Ozkaya: We also advocate teams to make technical debt conversations as part of their routine review, retrospective and planning procedures. And of course, as we give many examples in the book, the most costly technical debt is the one that accumulates over time with an impact on the systems architecture, therefore having an architecture mindset; conducting design reviews and making architecture design trade-offs as explicit as possible will also help in uncovering existing technical debt as well as recognizing technical debt as teams are taking it on.

InfoQ: Is it possible to measure technical debt? How?

Kruchten: We’ve grown suspicious of approaches that tell you: "You’re $653,299 in technical debt." It is not that easy to measure, and remember that only actual debt matters today, not all potential debt.

Ozkaya: We talk about this in the book from several perspectives; it all depends on what the expectation from measurement is. Of course any software project can baseline their overall productivity, time spent on refactoring/rearchitecting, time spent getting stuck and the like. And all of those maps to multiple measures, all of which at the end of the day are communicated as time and cost. One way teams can measure technical debt if they employ a disciplined practice of recording their technical debt items and creating a registry is by assessing the cost and schedule impact of each.  

InfoQ: When trying to resolve technical debt issues in source code, refactoring comes to mind. How can we apply it effectively?

Kruchten: Refactoring is well-understood and well-supported by tools, nowadays. The key is selecting what to refactor. That’s were a technical debt mindset comes handy; refactor not the easy pieces, but the pieces that are in your way going forward, actual debt actually hurting you.

Ozkaya: Refactoring will come handy when you know how to resolve technical debt. Since existing refactoring know-how will guide developers to cleaner solutions, it will also help in eliminating unintended complexity in the codebase. However, oftentimes technical debt that accumulates over several years needs larger scale solutions and more re-architecting, rather than refactoring.  

InfoQ: What other techniques can be used to deal with technical debt in source code?

Kruchten: Besides static analyzers, consider self-admitted debt, as seen in the comments and issue trackers, and the knowledge of the developers who are very familiar with the codebase.

Nord: Keep in mind that technical debt is not synonymous with bad quality. Static code analyzers assist in identifying problems with low internal quality, but there are other issues in the technical debt landscape like those related to architecture.

InfoQ: You mentioned in the book that architecture technical debt has the highest cost of ownership. Can you elaborate?

Kruchten: Refactoring code is actually easy and becomes easier with the recent advances in tool support. But architecture encompasses design decisions that are not just a simple matter of refactoring. Selection of development platform, programming language; decomposition in subsystems and layers; team organization; acquisition versus development, etc. Some of these choices may have been optimal at that point in time they were taken, but 5 years and 2 million lines of code later, they have become technical debt. And this is extremely expensive to remedy. Most often, companies are left with the only choice to scrap and re-implement the system, which comes at a huge business risk.

Ozkaya: It is also important to realize that it is not easy to decouple small implementation issues that accumulate over time and the structure and behaviour of the system. The small issues over time will also degrade the architecture and will impact its deployment. Technology will change over time, impacting many of the decisions that have been made within the system if not evolved timely, and will result in expensive technical debt. Using architecture reasoning will help teams go a long way in managing their technical debt strategically.

InfoQ: How can we examine the architecture to uncover technical debt?

Kruchten: There has been recently a flurry of tools to assist the examination of architectural issues in software systems. But most of the architectural debt is actually known by the designers and developers; once you describe to them what architectural debt is and give them an example, they usually are quick to identify technical debt in their own system.

Nord: There is a growing number of analysis techniques such as reflective questions, tactics-based checklists, and scenario-based design reviews.

Ozkaya: Qualitative reviews continue to be most effective when it comes to uncovering architectural debt, but the trick is to use these early and often. Also, today the increasing availability of automated tools that can be integrated into DevOps pipelines provide an opportunity. While they will not provide you with a comprehensive architectural reasoning, they will give you an opportunity for example to ensure that unintended calls are not placed to particular elements of your software stack.  

InfoQ: How can adopting DevOps help to reduce technical debt in production?

Kruchten: DevOps plays two roles relative to technical debt. On one hand, the implementation of Devops accelerate the iteration in software development, increasing the pressure, and making more opportunities for incurring technical debt, and more opportunities for reducing it. On the other hand, it exposes some of the technical debt that is not square in the code but lays in the infrastructure; in the deployment process, the scripts it uses, the manual steps, etc. Adopting DevOps in itself will not reduce technical debt; no magic wand there.

About the Book Authors

Ipek Ozkaya is a technical director at the Carnegie Mellon University Software Engineering Institute, where she works to develop engineering practices, techniques, and tools to develop, operate, and evolve software and AI-enabled systems.
Robert Nord is a principal researcher at the Carnegie Mellon University Software Engineering Institute, where he works to develop effective practices for agile at scale, software architecture, and managing technical debt.
Philippe Kruchten is a professor of software engineering at the University of British Columbia, in Vancouver, Canada, with an interest in the software development process and software architecture.