1. Jim Coplien for InfoQ: Hi Dave. Would you like to introduce yourself for our audience?
Hey Jim. I am Dave Thomas, pragmatic programmer. I have been writing books, publishing books, consulting, working with teams for the last 15 years or so. Doing work with Andy Hunt has been a blast, we have been having a lot of fun.
2. What's the most fun thing you have done in the past 5 years?
Technically, probably starting a publishing business, because it started off as a "Hey let's form a band" kind of thing, we had no idea what we were actually getting into. If we had, we would never have done it, but you swallow all the problems as they come up and you just keep going and in the end you actually get a book in the bookstore. And the first time we walked into just a neighborhood bookstore and found the books we have published actually on the shelves it was wild, it was really quite something. And now it is getting more and more, we have 25 - 30 titles out, so pretty much every bookstore in the world we go into, we find our titles and that is fun.
Good question. It was an accident, frankly. We did all the type setting and the layout, actually we did all the work, for the "Pragmatic Programmer". So we delivered to Addison-Wesley the camera ready postscript for the book. And that was fun, we enjoyed doing it. We did the same thing for a Ruby book, which also was through Addison-Wesley at the time. And again we enjoyed the process. It was cool getting a book actually out, and then we stopped writing books for a little bit, we went back to doing our regular training, consulting, talking. Right around 2001 the bottom fell out of that market, and a lot of training just evaporated, so we were thinking: what is that's counter-cyclical with training, so if the training budget drops, then where else is the money going to go? And it struck us that "OK, if you are not going to send someone to a conference or have a consultant come in, then maybe you will buy the book instead". It's, like, 30 dollars as opposed to 3000 dollars, it seemed like a good bet. So we took a lot of stuff that we were doing in our training business and we encapsulated it into three volumes which we called "The Starter Kit", and that was on version control, unit testing and automation, because these were the practices we felt would underlie every decent methodology, every team should be doing at least these three things. So we wrote those three books, in fact initially we wrote two books and Mike Clark wrote the automation book, and then we said "OK, now how do we get these things published?" And we clearly could go to Addison-Wesley and ask if they could do this, but at that point we thought we already did all the work to get the book ready, why not take that extra step and get it published ourselves. So we investigated what it would take to do that and it turned out to be not very much. And so we got a printer, and then we got the books printed and then 3 weeks later trucks arrived in my driveway and I got the classic "My garage is full of cartons" situation, and we sold books like that. For the longest time we sold books and pdfs just directly, we shipped them all ourselves and after about 9 months O'Reilly approached us. And O'Reilly turns out to be kind of stratified, they have O'Reilly the book business and then they also have O'Reilly media which handles distribution, and that is independent from the book business. And we signed up with their distribution people and they now distribute our books worldwide to the retail channel. So a Barnes & Nobles or Borders or Amazon gets books via them, from us.
That's really easy. The books we are looking for are books that make developers' lives better. Our fundamental focus is making the developer's life happier and so whatever we do always has to have some practical bent to it. We are not interested in "Here's my pet project API." We want people to say "Here's a good technology to use under these circumstances and it makes your life better in the following way ...". Any topic is fair game, anything people want to come up with that has that kind of profile. What we look for is: we ask people to give us an outline, which can be just a table of contents or even a rough idea of where they want to go, because the reality is at this stage the outline is just a wild guess anyway. Then we ask them for a sample of writing, simply because writing a book is clearly very hard and if people struggle when they are writing, then it is clearly going to be a very very long road for them, and they would probably give up. So we would like to see what their writing looks like. And we also ask them to give us one paragraph why this book should exist, because that then lets them tell us where they think the important areas of this book are, what difference it is going to make to people's lives, and that's where we get to see their passion. And ultimately passion trumps just about everything else in this business, and that's what we are looking for.
5. You are an Agile publisher. What does that mean, to be an Agile publisher? That's a great phrase!
Actually we are an Agile publisher, I think. And if you go back to the underlining ideas behind Agility, then we are looking at feedback, we are looking at communication, we are looking at rapid iterations, and we do all of those. We didn't know anything about how to be a publisher, literally, so we just did it the way we know, and the way we know is software. So we set up a publishing business based around the same way you would do a software project. In this publishing business we have version control systems that our authors use, we do "betas" so that customers can see the books early and comment on them and give us feedback, and actually even change the content of the books as it goes along. So we are doing a whole bunch of things that would normally be associated with an Agile software project, but from the publishing prospective.
I think Agile is good and bad news, but I'm not the best person to talk about it. But let me give you my prospective. I think that Agility has moved from the kind of wild and wacky into conservative mainstream. I think it used to be all those Agile folks and "How do I get you into my organization" kind of thing. I think people have recognized that agility is valid, is here to stay and it's a valid way to build software projects. I think that we have passed that barrier which is really good. The trouble is that once you passed that barrier you then get a bunch of people flooding in that don't really know what it means to be Agile. I think, as a community, we are doing a pretty good job but we could do a better job of trying to make people understand what it really means to be Agile, what the true meaning of it is. It doesn't mean I am doing XP or I am doing Scrum; it's really a question of what I feel when I am doing it, what's the philosophy of it. Because you certainly don't have to do XP or Scrum or one of these standard methodologies to be doing Agile software development. That's the challenge that we are facing now, as we grow as a community and as the adoption rates increase, how do we keep it right and how do we stop it from being diluted?
I don't think it is. I think the question is not "Are you doing Agile?" but "Is your software development working for you? Is your practice work for you?" And if your practice works because you go out and kill a sheep at midnight and look at the entrails, if that works for you reliably (I mean, animal rights not withstanding), do it. Agility is only one way of developing software. And the objective of Agility is to develop good software that works for the customers at reasonable times etc. It doesn't have to be Agility to meet those objectives so don't forget why you are doing it, don't become slave to Agility in the same way you shouldn't become slave to XP or another methodology in Agility. Instead ask yourself "Are we developing the software the way we want to? Do we have the kind of quality we want? Are we doing it within budgets? Are our developers happy? Do they have a home life?" Ask yourself what's important in your project and if you are meeting that then that's good. If you are not meeting that then you have to say "How do I get there?" and that's where you may want to consider Agility as something that works for you. I think, of the methodologies I see right now in the majority of commercial shops, Agility is probably the best fit. But it isn't always. There are certainly many cases where something heavier or more formal works better. The question is not really "Should we be using Agility?" it's just "Is our software working the way we want it to?" Do the analysis and then work it the way you want it to be.
I don't consider myself a Ruby guy, I get nervous when I ask someone what they do and they say "I am a Java programmer". Because that's kind of like asking a builder what they do and they say "Hammers". They are all tools and we should as professionals use the appropriate tools for the job. If someone calls me in for a particular kind of project I may say "OK, I would like to do that in Java, it's the most appropriate tool for this job". I may say "I want to do it in Ruby or Erlang, or C++". Whatever the tool is that fits the job is the tool that I would like to use. The real danger I see right now with programming languages and developers is this idea of monocultures, "We are a Java shop", because you know that's always going to be a compromise, it's never going to be the ideal solution for everything you do. I strongly encourage developers to diversify, diversify the languages, tehniques, frameworks, everything they do. Because by doing that they get experience they are going to apply back to the work they are already doing.
A while back, 4-5 years ago, we started this "language of the year" idea, so people could sign up on a mailing list to learn a language and we spent a year cooperatively learning this language. And we chose Haskell, the functional programming language. And everybody on the list learned it. At different rates, but that doesn't really matter, I mean I was playing along too because I never programmed in it and I became that year pretty good at Haskell, really enjoyed it. I have never written a line of commercial Haskell code since. But it doesn't matter, because the knowledge I gained during that year instantly improved my Java coding and my Ruby coding. I could use techniques and little tricks that you could pick up with Haskell, or even different ways of thinking about things. And now I found myself learning Erlang and all that stuff I learned in Haskell works for me in Erlang.
And that's gorgeous. So, I think for all developers you need to think "Yes I need a common language, a C # or Java or whatever it might be". But I think you also need to diversify out into some of the more Agile languages, functional languages would be great. A real o-o language, like a SmallTalk or something would be very good. And see what you can learn from those, don't necessarily do it with the view "I want to get a job in this language next week", but instead you are a professional, you should be investigating your tools set and you should be trying different techniques and different ways of doing things. And maybe then, more and more it looks back into practices. And that's not just to make you better at your job, it's also to make you safer in your job. Because, at the speed things are changing, in 10 years the job description for Java programmer is going to be "maintenance." And that's just the case, just the way things work in this industry.
Good question. I think the answer is fairly simple: you need to get to the point where you know what you don't know. And once you got there then you are safe. There's a point where you are first starting off where it is confusing and you don't get a thing. A little bit later you say "OK I understand this now". And at that point you are dangerous, because you get out there and what you are really doing is writing Java code in Haskell, or Ruby code in SmallTalk. Then there's that next step you get to, where you realize "Oh, I didn't understand it the first time around, and now I get it". At that point a whole bunch of stuff starts to make sense, the documentation suddenly makes sense. Where it was previously opaque, it is suddenly "OK, I know what you mean by this now". At that point I would say you: haven't got anywhere close to mastery, but at that point you are competent to be able to write in that language, because you know what to do and you know what not to do and you then know how to fix it when you get at that point. So: "I don't know what the library function is called to do a binary search but I am pretty sure there must be one and I will go find it". That's where I would say you stop.
My personal trick for doing this, and I am doing it right now with Erlang, is I wrote a while back a series of blog posts called "Code Kata" which are exercises for developers to do. And that was based on the karate idea of "kata", where you repeat the same thing over and over and over again, not because you want to memorize it, but because you want to internalize it. And by doing that, then when you are actually called to do it, your body will instinctively have the motions. I do the same thing with code, I have a series of 21 or 22 code kata, some of them are coding exercises, some of them are design exercises, and I repeat them often. I have no idea how many times I have done my little binary search kata, not because I want to write a binary search thing but because it is a great way of experimenting new languages on new environments. I am doing that right now with Erlang and just going through the kata, and by the time I get to the end of them I would be in a not quite so dangerous stage. Right now I am writing ugly Erlang code, but in a couple of weeks time when I get to the end I would be pretty comfortable "OK, I don't know it but I know what I don't know" and therefore I would be able to carry on and write it from there.
That's true. And I think actually there are a lot of similarities between music and programming. And one of the interesting things, as programmers, is typically that the symphony gets out there and the first time they see the piece is when they sit in front of the public to play it, in programming terms. Which is ridiculous because a real orchestra would practice and rehearse. They have down times, and we don't do that with our teams. And if we want to get our teams working effectively, we need to give them rehearsal time, time when it doesn't matter if you make a mistake, because that's when you learn, and we don't give people the opportunity to make mistakes on the job.
12. That's a great metaphor. Dave, have you ever been in Academia? Got the professor thing?
No I haven't.
No, I think it is a really important question because I think to some extent the tone of the industry is set by Academia, because most people coming out are expected to have a degree in, typically, Computer Science. I don't know if there is a list of 3 things, it depends to some extent on where the student wants to go. But one thing I would say that you have to be very careful of, if you are an academic, is that you are dealing with a very delicate product in your students, and ultimately when a student gets into the industry it is not how well they can analyze a particular function or the depth of knowledge in this particular architecture, it is their passion that drives them forward. And as an academic I think you have a responsibility not to squash that passion, I think you have to find ways of nurturing it. I think that's important. I don't know what that means in practical terms, apart from, I think you have to try and motivate the things that you do, not necessarily in a practical way, but at least give motivation so people understand the context in which they are doing things.
I think it is really important to be well-rounded, by that I mean both rounded in computer terms, so you need, for example, to not just write Java code for 4 years as part of your degree. I would love to see degrees that start up with, maybe, C and SmallTalk in the first year, or something like that. And then migrate on to a Haskell and OCML or something. Just to get some breathe in there before you get down to the "payroll" kind of languages. I think you have to understand that there is a certain beauty to computing, that you may not see if all you have been exposed to is Java. I think that is important. And I also think, I am not too sure because I personally hated it as a student, but a lot of colleges are focused on computing for the computing students, so you sit there and you have all your courses in software on whatever it might be. Some of the best hires that I have ever done have been music majors, history majors, philosophy majors, who never saw a computer in college, they saw probably the inside of a bar most, and they turned out to be really really good developers.
Because they cared, they were smart, they could synthesize, they could draw parallels, all these kind of things that you have to do when you are working with clients in a real software situations. You could teach some of the programming language but you can't teach someone how to work the parallels between this and that. It's tough, I don't know if that means that computer science courses should try to be more broadly based, or whether as employers you should look for people with minors in nontechnical areas, I don't know, but that would certainly be some advice I would give.
14. Might it mean that computing is not a proper discipline?
I actually quit my PhD because I didn't feel that there were valid PhDs in software. I got into it and I was enjoying myself, I was having a blast, but I thought "I am not really doing research in any kind of meaningful way." I know there are genuine software research projects, but a lot of time, the stuff that passes for research in academia is not research. Yeah, maybe it's not a discipline in that sense, but that doesn't stop it from being a university topic. I think it is still a valid thing, because fundamentally universities are not about turning out people for jobs, I think universities are for giving people who are 18, 19, 20 years old an opportunity to experience passion, in depth. So if you want to do music, you can go to college and you can spend 12 hours a day making music and playing with other people who want to do music. If you want to do history you do the same thing: you go and join a group of people who are equally enthusiastic and hopefully that helps you build the passion for the subject. And so anything which you can identify as a discipline I think it's a valid topic for university and software certainly can ...
Ask the question differently. Think about all the 5 years olds you know, they are full of passion, they are passionate about everything, whether it is Power Rangers or Star Trek, they have passion for everything they do. They do everything with gusto, they enjoy it, they don't mind making mistakes, they laugh when they make mistakes, it's so cool being a 5 year-old. By the time they get to 25 we have stamped that out of them, somehow we've destroyed their passion. So, it's not a question of instilling passion, we can't instill passion into someone, but what you can do is stop killing it in people. How do you do that? If you are an employer, I think you have to be very cognisent of the fact that every time you put someone into a situation where they are out of their depth, for example, you are doing a little bit to kill their passion. Every time you implicitly criticize people for doing their best, you kill their passion. It is very easy to destroy passion. Look at successful employers like Google for example, incredible reputation, how do they do that? First of all the make it special to be a Google employee. To get in you have to pass some hurdle, this means it is a valuable experience just to be there because you are a Google employee. Then they say "We are going to respect you". Every Google employee gets 20 % of the time one day a week to work on their own personal projects - unbelievably valuable, both for them and also for Google. It keeps their passion going. It means that if you got a project that is interesting to you but doesn't line up with your regular day job, they still give you time to do it, so you can keep your passion going forward. From their point of view, a whole bunch of their cool new stuff have come from these pet projects of people. Passion is not something to be instilled, it is something to be nurture and something to be allowed to exist. I think you can see companies doing it, it is possible to do it.
Today? Well, I turned 50 this year... (Jim: Congratulations!) Dave: Well, I don't know if that's "congratulations" or not, but they made me stop and think a little bit about... I didn't have this kind of thing "Oh my God I will be 50!"... but I had this thought about "All right, where is it all going?" And it gave me the opportunity to look around to other people who are also advancing in years. And I see a whole bunch of developers out there, not just developers, but let's focus on developers, who have got maybe 40-45, and they have put all their lives into software, they really love it, they are passionate about software, they spent a whole bunch of time doing this and they get to 40-45 and suddenly at some point they stop and look around and say "That's all there is. I put everything into this and it's good and I have reached a very high level of expertise, now what?" I am beginning to think that there is a bigger picture here that I need to be working on with developers, to help people broaden out just a little bit. So for example I was giving an interview a while back and someone said "What would you do if you had just 3 months free?" And I said I would like to learn to play the piano. And my wife read that interview and for my birthday last year I got piano lessons. It's absolutely glorious. I had no idea just how much I'd enjoy it. I just sit down at the piano, and if I had a frustrated phone call I just go down and worked through something. And that feeling of something totally different is just great. Those kind of things, it's not a hobby because it is deeper than that. With people that have passion for stuff, when they take on something new they will do that with passion as well, whatever it is. Just like James Duncan Davidson (http://duncandavidson.com) took up photography: "It is just a hobby" (quote), and now he is being paid for it, as a professional photographer. He loves it, he is an expert in his field, people come to him for advice, that's the kind of people that I am talking about. These people look for new challenges, and we are not very good at serving these people. We have dummies books and that's about it. So I am thinking there is a way of helping people look for these passions in their lives and making easy for them to take them up.
17. At 50 years old you are a 5 year-old with passion. What do you want to be when you grow up?
Three! Three years old have a bunch of fun. That's a really good question... I don't know, I am still exploring. It is really cool. If you had asked me 5 years ago I never thought about the publishing business. But that has been really, really enjoyable, we did a whole bunch of stuff, we are taking it in different directions, so I am exploring all sorts of interesting things. So I don't know. I look for stuff that pays back by being fun. And when I find out, I will just go and pursue it.
18. How long have you been in the industry now?
I wrote my first commercial program in 1973, I was young, very young.
20. Is the industry still a fun place to be?
Great question. For some people yes, for other people no. There are clearly people who are enjoying themselves, having a blast they are trying new things, doing great work, and for them it's a wonderful place to be. I mean, after all, when you think about (this is not me, this is Fred Brooks):" Software is like poetry: you start with a blank piece of paper and anything you can imagine you can create," and that's a unique place to be as an industry. But it is also really challenging, it is very hard work. So a lot of people don't get that part of it, they don't enjoy that part of it, for them it's a job: they turn up, they do the work, they go home. Which is fine but it's a really really hard way to make a living. Software is tough, because of that challenge with the blank piece of paper, every day you are faced with these kind of decisions you have to make, and every decision you make comes back to haunt you if you get it wrong. So for a lot of people software is a tough way to make a living. I would say that if you are not enjoying it then you are not writing good code. I think you have to be happy to write good code, just because it is so hard to do. Frankly if you are not enjoying it, you are not writing good code, you might ask yourself: "Am I really in the right job? In the right industry?" Because I think everybody has the right to be happy. When you spend 8 hours a day, half of your waking life doing something, I think you deserve to be happy doing it. I think the question is "As an industry are we happy?". Some yes, some not. "As an industry should we be happy?" Absolutely! I would say that the people who are not happy really deserve to ask themselves "Am I really in the right place?" and then do something to make themselves happy. Everybody has the ability to find something that does make them happy. They have a right to do that. And I think now for the first time in a long long time we actually live in a world where that is possible.