1. Introduction by Floyd Marinescu
I am here with Jim Coplien and Bob Martin at the JAOO Conference and here we have 2 very interesting divergent opinions on what is the value of TDD. So let's open up the floor and let each one have a couple of minutes of say. Let's hear you guys talk about it!
Jim Coplien: Well it may be good, because you did this at your keynote yesterday: you said what you mean by "test driven development". I have adopted a very strong position against what particularly the XP community is calling test driven development. And I have audited this versus a lot of tutorials at about 4 conferences in the past 6 months and they give a very consistent story on what they mean. Here, it was a little bit different yesterday, so maybe for the sake of making this a meaningful conversation you can quickly reiterate so we are on the same page.
3. Bob Introduces his 3 laws of TDD and Jim Responds
B: So I have 3 laws of test driven development. The first one is: a test driven developer does not write a line of production code until he has written a failing unit test, and no production code can be written until there is a failing unit test.
the second law is, you do not write more of a unit test than is sufficient to fail, and not compiling is failing. So you cannot write very much of the unit test before you write production code.
The third law is you cannot write more production code than is sufficient to pass the test. You cannot write a little bit of unit test and then run off and write production code. These three laws lock you into a cycle that is perhaps 30 seconds long, and that means that you are actually writing unit tests and production code concurrently, with the tests perhaps 30 seconds to a minute ahead. That is my definition.
J: Per se, the main concerns I have about TDD are not problematic with respect to what you've just said in isolation, so if it's no more and no less than that we may not have a big disagreement. What my concern is, then, comes out of doing broad work with a lot of clients and a little bit of interactions with other consultants and other scrum masters who have seen these things happening in their project. And we've seen 2 major problems: one is that use of TDD without some kind of architecture or framework into which you're working - which was very strongly Kent's original position: you use TDD to drive your architecture - leads to a procedural bottom-up architecture because the things you are testing are units.
We just had a discussion upstairs about "is TDD the same as unit testing?" Well, no, it's a little more, but unit testing was a great idea in Fortran, when you could build these layers of APIs and the units of organization of the software were the same as the units of testing, but today the units of organization of the software are objects and we're testing procedures and there is a little bit of a mismatch. Now, if you are using the procedures to drive your architecture, I think you are trying to build a 3 dimensional structure from 2 dimensional data, and you end up going awry. And one of the things we see a lot, in a lot of projects, is that projects go south on about their 3rd sprint and they crash and burn because they cannot go any further, because they have cornered themselves architecturally. And you can't refactor your way out of this because the refactoring has to be across class categories, across class hierarchies, and you no longer can have any assurances about having the same functionality.
The other problem we've seen is that this destroys the GUI and this is what Trygve [Reenskaug] and I talk a lot about, because you have this procedural architecture kind-of in a JAVA class wrapper; you no longer are driving the structure according to domain knowledge and things that are in the user's conceptual model of the world, which is where object orientation came from. I mean even Kent, as he's very often said: "you can't hide a bad architecture with a good GUI." The architecture will always shine through to the interface, and I strongly believe that, and that is why I believe we need something in the infrastructure that gives you a picture of what the domain model is out at the interface. Then, if I want to apply Uncle Bob's 3 rules I probably don't have a problem with that, but I want a starting place that captures this other dimension, which is the structural dimension.
And in fact he says this, in "XP explained" or something, his page 131 of his book he says: "Yes, do some up-front architecture, but don't knock yourselves out".
J: I do agree that architecture evolves, I do believe it's informed both by the code that you write and, maybe even earlier, by use cases that come in, that inform you about things that are relating to scope and other relationships; but if you try to do things incrementally, and do them literally incrementally, driven by your interaction with the customer without domain knowledge up-front, you run the risk that you do it completely wrong.
I remember when I was talking with Kent once, about in the early days when he was proposing TDD, and this was in the sense of YAGNI and doing the simplest thing that could possibly work, and he says: "Ok. Let's make a bank account, a savings account." What's a savings account? It's a number and you can add to the number and you can subtract from the number. So what a saving account is, is a calculator. Let's make a calculator, and we can show that you can add to the balance and subtract from the balance. That's the simplest thing that could possibly work, everything else is an evolution of that.
If you do a real banking system, a savings account is not even an object and you are not going to refactor your way to the right architecture from that one. What a savings account is, is a process that does an iteration over an audit trail of database transactions, of deposits and interest gatherings and other shifts of the money. It's not like the savings account is some money sitting on the shelf on a bank somewhere, even though that is the user perspective, and you've just got to know that there are these relatively intricate structures in the foundations of a banking system to support the tax people and the actuaries and all these other folks, that you can't get to in an incremental way. Well, you can, because of course the banking industry has come to this after 40 years. You want to give yourself 40 years? It's not agile.
So you want to capitalize on what you know up-front, and take some hard decisions up front, because that will make the rest of the decisions easier later. Yes, things change; yes, architecture evolves; and I don't think you'd find anyone who will say "put the architecture in concrete". I also do not believe in putting the code in place, that is, the actual member functions, up front. You put the skin, you put the roles, you put the interfaces that document the structure of the domain knowledge. You only fill them out when you get a client who is willing to pay for that code, because otherwise you are violating Lean. So you do things "just in time," but you want to get the structure upfront, otherwise you risk driving yourself into a corner.
But the problem is: that's like saying that words have meaning apart from any definition. And so the fact that I call something a mule, without saying what a mule is, doesn't make it a mule. Like Abraham Lincoln said, "Calling a mule an ass doesn't make it one." And so the thing that gives meaning to stuff is the member functions as semantics. You don't want to go crazy and you don't want to be guessing, and here is where I agree with Kent. He says in the "XP Explained" book: "You don't want to be guessing" and that is true. But I do want to assert what I know and there are some things you just know about the structure of a telecom system, a banking system. You know that you don't build a recovery object. I was on a restructuring project in a large telecom company once where they were redoing a toll switch using object oriented techniques and modern computer science techniques, and I got assigned to work with the guy who was making the recovery object. Well this is ludicrous, recovery isn't an object, but yet his superficial knowledge of the domain let him do that. If you get down to understanding what the member functions of that are then you will see this isn't even an object. So you ask: "How do I know it's not an object? What are its member functions?" "Uh... to recover". "Great. That is a lot of help." Actually I think there are people who have capitalized on this and it's now called SOA. That is the danger.
You want to have something there to give the object meaning.
So, 2 million is, in my experience, pretty small. I am working with hundreds of millions. Before the first executing code... it depends a lot on the individual system, but let's say I were building a simple telecom system. What I would probably do, let's say I am doing it in C++, I would have at least constructors and destructors in place and be able to start to wire up important relationships between the objects and...
11. Would you have tests, testing those wirings?
I would have tests for those wirings. An obvious test is to make sure, when the system comes up and goes down, that the memory is clean, for example. Half an hour.
I think that is a separate disagreement, but maybe we can put this one to rest, this is nice. But the thing I want to make clear for the audience is that, again, I think when I am running into people that are doing things right, that avoid the kind of problems they talked about earlier, it's not TDD out-of-the-book or TDD out-of-the-box. So, people have found a way to move to what Dan North now calls BDD, for example, which I think is really cool (if you ignore the RSpec part and all the stuff which is kind of dragging it back to too low of a level).
So there are a lot of people doing the right thing and my concern is that they are calling this good thing TDD and then people are going to buy books and they are going to look up TDD and they are going to find this old thing which is "architecture only comes from tests," which I have heard four times in tutorials in the past 6 months, and that is just, like you say, horse shit. But now, on to the professionalism issue, how would you know a professional if you saw one?
13. They practice TDD? (smiling)
"Professional," to me, is just someone who makes money for doing a job in that area.
J: I do disagree with that. Now, I think there is something deeper that is important, and let me attack this by example. As an example of something I could do as an alternative: I could wave my hands and say a lot of things about code inspections or pair programming and those are good and probably have more value, but it's kind of an independent discussion. But let me give you something that I think hits the nail on the head even more importantly. Let's look what a unit test is. What a unit test does is: looks at an AP of a procedure and kind of goes and hits the state space of the arguments and maybe hits a half a dozen of them, or a hundred or a few million of (2 to the 32nd power) or whatever, and so you're just doing hit and miss. That is really heuristic, you've got to be really lucky to find bugs doing that.
What I think is more powerful is "design by contract." So you have preconditions, post conditions and invariants. Now, the technology isn't there in most languages. They haven't matured to the point where Eiffel has, where you can statically check these things, but you can build additional infrastructure to do that kind of thing. I think it has all the advantages TDD, there are these advantages (supposed advantages), about: I am going to think hard about the code, I am going to focus on the external view and so forth. And I have found, at least for me, that contracts do that more effectively than tests do. Furthermore, they actually give you broader coverage because you are covering the entire range of the arguments rather than just randomly scattering some values in there.
Now, Bertrand Meyer actually taken this further and he has something called CDD, which is Contract Driven Development, where what he does is: he takes contracts and he kind of feeds random numbers at them and if they don't meet the preconditions you don't run them because you know that test will fail, but he tests if the post conditions hold after you run the test, and if they don't it's a bug. And they have actually done this. They have a tool that automatically runs tests. They have done this on the Eiffel library and they ran it about a week, they found 7 bugs in the 20 year old Eiffel library - that is kind of interesting. But it comes from a part of the code where you are expressing intentionality in a way that has hope of being traced back to something of business importance, and the problem about TDD, as most people practice it down at the class level, is that it's really, really difficult to trace those APIs at a class level sometimes all the way up to business significance.
I think you are creating a dualism that needn't be created, in that there is one thing, which is the code, the code is the design, it's what's delivered, anything else is not Lean. In typical projects that use unit testing, the code mass is about the same as a test mass, and where there is code, there's bugs. You cut your velocity in half. There is well known examples, the most famous example is the ADA compiler where actually, use of test driven development increased the number of bugs in the code, because your code mass increases, because you have more tests. If you are using assertions you have this nice coupling, that is essential coupling, between the semantics of the interface and the code itself. Whereas with the tests the coupling is a lot messier and hard to manage. There was another point you've made I was going to react to...
17. I am surprised that you think the code mass is different.
J: In my experience, it is. If I look at how people actually use this: I like it when I see a JUnit spec that looks like assertions, but a lot of the time it isn't.
That isn't my argument. My argument is: how I am seeing this being used in broad practice - and they are not getting it.
19. Well, OK. And do you see contracts being abused in broad practice?
First of all, they are not being used enough...