At the OOP 2015 conference James Grenning, signatory of the Agile Manifesto, gave two talks Technical Excellence and Test Driven Development for Embedded C. InfoQ interviewed Grenning about why people are not doing technical practices sufficiently or well enough, why he thinks that TDD can be fun, the importance of unit tests, why programmers need to have a good nose for code smells and how they can become better in discovering "bad code”.
InfoQ: Can you elaborate what you mean with "technical excellence"? What makes it so important?
Grenning: For me technical excellence is being an engineer, solving problems. Too often in our industry it seems that people just accept that code will have bugs, and be a mess. Engineers sit around waiting for their big monolithic code bases to build. They have dependency nightmares. They accept these realities. They appear to be victims of their situation rather than the masters of it.
When I go to my internist, he always knows new information about my medications, or latest treatments. He is a professional. He learns new things continually. He tells me that when he was first starting his practice, the half-life for medical knowledge was about 7 years. Now he says things are changing so fast he could become obsolete in under a year.
Technical excellence is about being a professional. It is about learning. It is about continual improvement.
InfoQ: In your opinion what are the causes why are people not doing technical practices like TDD or refactoring sufficiently or well enough?
Grenning: Even though the technical practice of TDD and Refactoring have been around visibly since 1999 (with the publication of Extreme Programming Explained), most people in software development have no idea what they are. I recall being a young engineer, having large demands on my time from my employer and family. There was not a lot of time to devote to learning what the best ideas and techniques are. There are many ideas out there on how to build software. Not all of them so helpful. So its hard to know where to put your energy. As a young engineer in a new field, I thought I knew everything too.
With experience came the humbling realization that I did not know everything. There is a lot to know in this broad and deep field. We each learned how to program in a very unique way. For me, a professor showed some examples of C programming constructs and then gave us an assignment to write an operating system. Go figure it out! And we each did figure out how to program in our own unique way. It seems that no one was taught how to program. If you think you are already a master of your craft, what motivation if there to learn more?
I give a survey to people that are about to attend my training. You can see the questions and answers here: Training preparations replies. You can see the techniques for writing and testing software are all over the map. Most report using the techniques we used at the university in the 70s, and in industry in the 80s and 90s: print statements, break pointing and single stepping the debugger. Congratulations, it is 2015 and you are using state of the art debugging techniques from the 1970s! How about trying 1999 state of the art defect prevention with TDD!?
After seeing TDD, engineers tell me they do not have time to write tests. They do not have time to proactively prevent defects but they do have time to react to those defects. Engineers also tell me that their boss won’t let them write unit tests. It should be equivalent to me not not following my doctor's prescription because I’m in a hurry.
The long feedback loops that most developers live with are part of the problem. Let’s say I provide an estimate of three months to my boss. After two and a half, I go and deliver the bad news that I need another month. This repeats. Finally a subset of the promised functionality is delivered after six months. It only looked like the engineer was really working when the deadlines were close. What does this do to the engineer’s credibility? Not much.
Working in iterations, with slim vertical slices of functionality, delivered every couple weeks, does a lot for a programmers credibility and confidence. Having the test safety net catches unwanted changes in behavior. The programmer gets all that for free once they master TDD as it takes about the same amount of time to get the code working in the first place with TDD as it does the 1970s way, Debug Later Programming (see the physics of test driven development).
InfoQ: In your talk you mentioned that TDD can be fun to do. Can you give some examples?
Grenning: A couple of the other questions I ask people about to take my training are: What do you like most about software development and what don’t you like about software development?
I do this to make sure I have the right people in the room. I find that people tell me they like problem solving, creating something, designing, and the intellectual challenge. Some like the challenge of debugging.
They don’t like chasing bugs too long, they don’t like unrealistic deadlines, changing requirements, documentation, someone else’s bad code.I try to show them that they can do more of what they like (problem solving, creating value) and less of what they don’t like (chasing bugs, useless documents, being late).
Here is what I say about TDD in my book Test-Driven Development for Embedded C:
“TDD is fun! It’s like a game where you navigate a maze of technical decisions that lead to highly robust software while avoiding the quagmire of long debug sessions. With each test there is a renewed sense of accomplishment and clear progress toward the goal. Automated tests record assumptions, capture decisions, and free the mind to focus on the next challenge.”
InfoQ: You stated that unit tests are critical as they lay a foundation for the software. Can you explain this?
Grenning: TDD leads to code that does what the programmer thinks the code is supposed to do. Modules are developed with an executable specification of the module, the test cases. The test cases document very precisely what the code is supposed to do. If the code starts to violate the specification, a test fails. One of the big problems with code is that unwanted side effects are very difficult to anticipate. I make a change to one part of the code, and a seemingly unrelated other part of the code breaks.
If you want code that is doing what you want, you need tests to keep the code under your control. I am no web developer, but I have been learning about it so I can control and evolve my website to support my training and business. My web deployment script was working just fine until I created a new server from scratch. I had a directory named wrong in my original server. As it turns out my script has the wrong name too, and in several places. The conditional logic that checked the directory was exactly backward allowing my website to be deployed. It worked but for the wrong reason! I had offsetting errors.
Once I fixed the deploy script I got to fix the wrong directory name in several other scripts as well. Too often systems work by accident. A solid foundation of code that is doing what the programmer thought the code should do is critical to a reliable system.
InfoQ: Why do you think that programmers need to have a good nose for code smells?
Grenning: The programmer that wants to improve the code they are working on needs three critical skills. A good nose for code, skills to envision an improved design, and the skills to transform the smelly code to the better code.
Simply, if you cannot identify with some precision a problem in the code’s structure, how can you fix it. I recall code reviews in my career were usually just a matter of opinion. “I don’t like that, I would have done this”, totally unsupported. Any programmer can announce “this code stinks”, but that is not good enough. A chef in the kitchen takes whiff of the air and says “throw out the potatoes, they are bad, and get some fresh ones”. Programmers need to be able to identify specific smells to have an idea of how to make the code better.
InfoQ: What can they do to become better in discovering "bad code”?
Grenning: Martin Fowler does a nice job defining them in his book. I add a few more to the list in my book. Also Bob Martin’s SOLID design principles give a specific vocabulary to discuss modularity and coupling issues. Developers need to speak in the language of code smells and principles.
To get better at discovering bad code, developers need to be exposed to clean code. Clean code is code that does not need comments to tell you what the code is doing. Clean code is written in a way to make it easy to come back to the code later. Clean code has tests to explain desired outcomes. Clean code has well thought out names. Clean code uses vowels. Code reviews can help, but they have to focus less on opinion and more on attributes of code, like DRY, SOLID and avoidance of specific code smells. Pair programming is a good way to write better code. You know, two heads are better than one. There are really good resources on this; seek them out.
InfoQ: What can be done to increase the adoption of technical practices to increase technical excellence?
Grenning: I think that the culture of solo work and specialization is in part at fault. If all you see is your own code, you’ll get the idea that it is good and there is nothing new to learn. Reviewing code late in the development cycle means improvement will be the hardest. Companies should encourage people to work together more closely. Pair programming is a great way to spread technical excellence, and it is a two-way street.
If you are reading this, it is likely you already understand the issues. So give this to a friend and see if you can help spread the message.
InfoQ: Any suggestions how people can convince their manager (and their manager's managers) to support learning and continuous improvement?
Grenning: Back at Object Mentor in the early 2000s, we had convinced ourselves that the technical practices of Extreme Programming were really important. We thought, now that we are convinced, we'll tell people about them and they’ll be convinced too. Well, not so fast. We started promoting a solution to a problem that had not yet been identified or accepted. I’ve come to rediscover the most important skill of the engineer is problem solving. It should also be an important skill for the management and executives of a company. So, my advice is, don’t push solutions. Identify problems, and then get support for solving the problems. Look beyond your own experiences for solutions.
About the Interviewee
James Grenning, founder of Wingman Software, trains, coaches and consults worldwide. With more than thirty years of software development experience, both technical and managerial, James brings a wealth of knowledge, skill, and creativity to software development teams and their management. As his professional roots are in embedded software, he is leading the way to introduce Agile development practices to that challenging world. See James' articles for applying Agile to embedded software development. Areas of interest are software process improvement, Object Oriented Design, programming, embedded systems, project management, Extreme Programming, Test Driven Development, test automation and Agile software development. James knows his way around Scrum, with Scrum Master and Product Owner certifications.
