At QCon London, Adam Tornhill presented "A Crystal Ball to Prioritise Technical Debt", and claimed that although the technical debt metaphor has taken the software world with storm, most organizations find it hard to prioritise and repay their technical debt. Key takeaways from the talk included: methods to identify 'hotspots' of code complexity and churn; the benefit of continually evaluating the suitability of the architecture for the current system, for example, by examining 'temporal coupling' of components or microservices; and that the cost of inter-team coordination can often be learned from examining the social metadata associated with code commits within a version control system.
Tornhill, founder and CTO at Empear, began the talk by quoting Martin Fowler with regards to the problems caused by technical debt: "Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development". In relation to this, three propositions were stated: technical debt interest rate is a function of time spent working on the system; that all technical debt is not created equal; and most technical debt is not actually 'technical' - there is a strong link between technical debt and the organisation that produced the code.
Tornhil continued by suggesting that a 'wish list' of questions and information required to effectively priortise technical debt would include:
- Where in the codebase is the highest interest rate?
- Does the architecture support the way the system evolves?
- Is there any productivity bottlenecks for inter-team coordination?
The code with the highest technical debt interest rate may be found using 'hotspot' analysis, which can be used to visually represent areas of the code base that are large, complex, or have high churn rates. Size and complexity of code can be determined with traditional tooling, such as Sonar Qube or Code Climate, and churn rates and other social information can be obtained from the version control system (VCS) in which the codebase resides by using tools such as Tornhill's Code Maat. Visualisation and analysis tools, such as Moose or CodeScene, can be used to explore and understand the findings from hotspot analysis.
With a nod to George Orwell's Animal Farm, Tornhill quipped that "all code is equal, but some code is more equal than others", and demonstrated graphically that over the long term the high churn within the codebases of several popular languages was clearly localised to several areas of the code (with a corresponding long tail). Citing the example file gc.cpp from the open source .NET coreclr project, this localisation of churn can also be seen to be true at a function/method level as much as at the Class/file level. Concluding this section of the talk, it was suggested that these 'hotspots' should be priortised for refactoring in order to reduce the technical debt interest rate, and that corresponding complexity trends of this code be supervised over time.
Next, the question of "Does the architecture support the way the system evolves?" was explored. Each system has a 'tipping point', in which complexity grows to be unmanageable, and code modifications become extremely difficult. Complexity should be regularly supervised, and the tipping point identified and avoided by appropriate redesign and refactoring. Analysing 'Temporal Coupling' is extremely useful for identifying whether two or more components (classes, layers, modules, microservices) are constantly being modified at the same time - implying a architectural dependency that is not obvious from simply analysing the code itself. The 'microservices shotgun surgery pattern' should be identified and avoided, and if additions or changes to the application regularly require the coordinated change across multiple services, then the architectural service boundaries may have been specified incorrectly.
The final part of the talk focused on the social side of developing software, and began by stating that there is often high 'process loss' from the potential productivity of all developers combined working on the codebase. Team coordination, or the 'diffusion of responsibility', can be measured using VCS analysis looking for areas of the code where commits from developers across multiple team overlaps. Tornhill proposes that this is a way to visualise and measure "Conway's Law". The use of feature teams can lead to large areas of the codebase requiring inter-team coordination, which is costly and often results in inconsistently applied standards and styling. It was strongly suggested to "align your architecture and your organisation" in order to reduce the increase of technical debt.
The talk concluded with a strong recommendation to make design, implementation and refactoring decisions influenced by data. Tornhill believes that as the systems that are being developed increase in complexity, this is becoming an essential practice, and accordingly has distilled his thinking into the Pragmantic Programmer's book "Your Code As a Crime Scene".
The slides for Adam Tornhill's QCon London talk, "A Crystal Ball to Prioritize Technical Debt", can be found at the conference website. The public availability of the conference talk recordings can also be found on the QCon London website.
