To write better code, programmers have to apply design fundamentals and read existing code, says Martin Thompson, a Java Champion and high-performance-computing specialist. InfoQ interviewed him after his Engineering You talk at QCon London 2016 about the challenges that the software industry is facing and what programmers can do to deal with those challenges and become better software engineers.
InfoQ: You presented several quotes from the first NATO conference on software engineering in 1968 which are still valid. How is the software industry still struggling?
Martin Thompson: There were a number of themes in the 1968 NATO conference. They realised that the software industry was having a crisis with delivery. They also realised some projects where successful and wanted to know how to extract the goodness and make it applicable on a wider scale. For me, some of the standout points were the realisations that software development is iterative, experimental/learning focused, practitioner lead, and best done in small teams. In many ways, they described TDD and agile decades before these were common practice.
As an industry, we have improved greatly but still have a long way to go. Software development is a very young discipline and we still have a lot to learn. I like how Dijkstra described it as a "radical novelty" and we are setting ourselves up to fail by using metaphors and analogies that are woefully inadequate. Building software is a step change over previous activities. The activities and constrains are radically different to previous activities. Some people have a natural gift for it, some can learn to do it, and the majority struggle. It took us centuries to reach our current capabilities in civil engineering so we should not be surprised by our current struggles with software.
InfoQ: Why do you think that understanding design fundamentals such as coupling and cohesion is important?
Thompson: One of the biggest challenges we have with software development is taming complexity as the size of an application grows. The more coupling we have between objects, components, modules, or systems, the more we experience many consequences. These consequences include but are not limited to difficulty of modification, propagation of failure, inability to scale because of contention, and performance issues due to dependent actions. Low and loose coupling in time, space, and implementation are essential to scale and resilience. Coupling is well illustrated by Connascence whereby a change in one module/component causes a change in another.
I find cohesion even more subtle than coupling. I like to think of cohesion as unity of purpose. When you have concerns pulling a component in different directions then it does not have unity of purpose, which can lead to unwanted behaviour and characteristics. Poor cohesion in software design is often a very good indicator of issues in requirements or team dynamics. A cohesive design is usually easy to follow and discoverability is high as related functions and features are grouped and not lost to each other.
Refining our skills with design fundamentals should be at the core of our daily activities if we want to become better software engineers. Repetition in practice and coaching are the best ways make the skills second nature.
InfoQ: Can you give some examples of how applying decomposition and abstraction can help developers to write better software?
Thompson: I think abstraction is one of the most misunderstood topics in software development. Dijkstra eloquently described abstraction as a means to create "a new semantic level in which one can be absolutely precise". Most developers completely misuse this term and create what they call abstractions to gloss over things they do not understand. Joel Spolsky even coined the law of leaky abstractions which badly tries to justify this state of misunderstanding. We have some great examples of good abstractions like block devices in the Linux kernel or device drivers, but unfortunately most software abstractions are often the result of some form of mental masturbation that results in a Frankenstein monster that makes code more difficult to work with rather than improving precision and aiding understanding. Bad abstractions can be much more costly than duplication.
We need to get better at decomposing business objectives into measurable concrete deliverables and then build software from composable components of high quality that demonstrate low coupling. The commercial companies want us to build within their frameworks so they can lock in customers. These frameworks are the wrong example. They are driven by commercial pressures. Commercial pressures that are often at odds with the delivery of high-quality maintainable software.
If we look to other engineering disciples, we can see the use of tooling to support the process of delivery rather than imposing a process on it. It looks like we are showing signs of coming out of an era where the commercials are focused on person-day rates, per-CPU licences, and lock-in maintenance contracts. Utility computing that we can consume on an as needed basis is now becoming available from the likes of Amazon. It is very interesting that cloud computing is great for supporting continuous integration and delivery models. This changes the dynamics and drives better behaviours. We can also see this with tooling now where the likes of Jetbrains don’t lock you into a contract for their product; they lock you in by providing a great product that increases productivity.
InfoQ: You mentioned rereading code as a way to find flaws or ways to improve it. Can you elaborate?
Thompson: Any creative endeavour benefits from continual revisiting and refinement. Have you ever gone back and read an email, essay, blog, report, etc. that you have written and then thought you could do some part of it better? This is a very natural thing. When we revisit, the context is different and this gives us new insights. At the simplest level, our assumptions about what we intended to write is now not in our short-term memory and we have to truly reread and reason. In other ways, we have have more information and the world has advanced and we have gained knowledge.
I like to think of the code as the place where I capture my current understanding. Besides the fact that we all make mistakes that we can correct when we revisit, we also can record our advanced understanding. Applications are typically a software simulation of a business process. If the software does not capture the current understanding of the business process then a mental mapping must be performed by the developer. This mental mapping is a high tax on any project. This is one of reasons I believe domain-driven design is a critical tool in software development.
Read all code on a regular basis and not just your own code. Reading the code of others is a great way to learn. As the writer Stephen King said, "Reading other peoples books is the best way to make you a better author." The same applies to code and open source is one of the best steps we have taken as an industry. By developing software in the open, we can share understanding, get feedback, and learn from others. Participating in open source is one of the best ways to make yourself a better engineer.
Martin Thompson presented Engineering You at QCon London 2016 where he talks about the characteristics of a good software engineer and explores the individual practices and techniques that can help bring out the engineer in everybody.