Walking around and looking at all the stuff going around here, I am amazed on how many people are involved are interested enough to come to conferences like this. But in some ways I am a little bit dismayed. As you said I've been involved in this for fifteen or sixteen years, and tried to stay pretty current with what's going on in the world, and I see an awful lot of sessions where I just read the title of them and I am little bit concerned that I don't quite understand how or what these people are talking about actually fits in what I believe Agile to be. The other day Ron Jeffries and I did a talk about what we got right and what we got wrong in Agile. And actually it was about what we got right and what you got wrong.
But there is enough of wrong to go around, and we believe that one of the things that is right about Agile is the sweet spot. There is a sweet spot of software development where you take a fairly small number of people bring in someone who understands what the problem is from the business side and have them produce software every few days, small number of weeks perhaps, and pay attention at what they are doing and get better at it. And a lot of what I am seeing in conferences like this are sessions about how to make do instead of how do you get into the sweet spot. And that bothers me.
So, in some places I am really happy that this hasn't grown as large as it was beyond a couple dozen of people sitting around and talking about it and now we have fifteen hundred people sitting around and talking about it and it turns out every year we only get fifteen or thirty percent repeat business of these companies so most of the time people on these conferences are folks coming for the first time so that means a lot.
I think there are some various key things, if you go back and look at what the Agile manifesto says, if you look at what the practices of extreme programming are, if you look at what Scrum starts with and expects you to add on to, it is all about getting a small group of people together who can fit more or less into one room along with somebody who understands the business problems and let them come up with the solution aided by the best technical practices we know, things like test driven development, behavior driven development, acceptance test driven development, all those sorts of things and then paying attention.
I believe the biggest thing we see, the biggest dysfunction we see is not actually having what the Scrum world calls the Product Owner or what the XP world calls a Customer, that is that business person, somebody who works for the profit center, who actually knows why we are doing this project, who understands the money that's to be made or lost if we do this project well or don't do it well, who is involved intimately with the project. The other day I was interviewing someone and they were talking about that you could get by with a product owner who is there maybe twenty five percent of the time. And my fear is that if we tell people that you can get by with a product owner that is there twenty five percent of the time they'll have a product owner who is there twelve percent of the time and that is not Agile. I think that is the biggest problem the lack of the product owner, there are other things but that's the biggest one.
I think the best place to start is continuous integration. The notion that every time someone changes the code, we interject that into all the rest of the code base, we recompile everything and run every test we can possibly think of in an automated fashion. Once you start doing that, then the need for better testing falls out the need for better refactoring, drops out the need for simple design, drops out from that. So if you had to start one place, I would start with continuous integration. And once you do that and pay attention to what happens to you, you will start driving the things you need to do next.
Because change is hard. I quoted something from the Princess Bride the other day in our talk, that was the man in black said that "life is pain", and that anyone who says otherwise is selling you something. And change is pain, change is hard, and anybody who says otherwise is probably selling you something. So anything we do that involves change is going to require some amount of pain and I think if making a transition to Agile isn't hurting some then you are not actually doing it. At least that's the way that it's been in every place that actually did it.
We see that a lot of times, we see teams who have done fairly well come back and some of that is that things are difficult to maintain sometimes, and others is that they were in an organization where anyone doing something different is eventually eaten; I think that's what happened to our first project we were different than anybody else in the organization was doing and it didn't matter how successful we were, we weren't doing what everybody else was doing and so the tall grass gets mowed.
I think because an organization or culture fights back, transparency is not really valued in most large corporations, they may say it is but their behavior indicates otherwise at almost every level. And lots of large organizations tend to run on blame that you are much better off in the last months of your project stopping work and figuring out who you can blame the failure on as opposed to actually get the project done. And I have actually seen instances this where it happened, where a project was two weeks away from being completed and they stopped to come up with the power point presentation to fix blame on someone else because they didn't complete the project on time, instead of actually completing the project on time which is what they could have done otherwise. So you see all kinds of odd dysfunctions.
Well, it's often very difficult down at the grass roots level to actually have any real change to the world around you and that's sad. If you have an organization that really wants to change and willing to do things there's things that can be done. One of the best things that you can do is help educate those above you in the food chain, in the hierarchy of the company about what to expect. Most organizations today operate in a PMI sort of approach expect certain sorts of artifacts to flow out of projects, those artifacts don't naturally flow out a project, but information that is sometimes much more informative does flow out. And so going to the folks a level or two above you and telling them what to expect out of your project, telling them how to help you, when those things don't look like what you've told them they will do, is a very powerful technique and I've seen that work time after time.
It turns out that most managers actually want to help the projects running under them. And if you just tell them when you need help and give them an idea what they actually can do to help you, it demonstrates that if they give you help things get better a few times, they will come to understand how this new Agile world works, and will come around to it. The other thing which is sad but unfortunately true is in the training that Ron and I do, we have seen more than perhaps is worth saying here loud in public that we often see a large percentage of the members of the teams we train leave the company they are at and go often form a small company where they can actually do the stuff we taught them because they were unable to do it in the company they were at. That may not help my sales so you might want to edit that bit out, but that is sad but true.
I think that teams are beginning to understand the need for technical practice. We got involved with the Scrum alliance two or three years ago now because they had discovered or realized something that we were making a living of for several years before this, that Scrum teams got to the end of their sprint and they weren't actually getting done. They were not meeting the definition of done because they did not have good technical practices. And so Ron and I work with a lot of Scrum teams teaching them how to do basically XP technical practices. And now the Scrum alliance has baked that in, and we actually were involved very closely in the creation of what's called the certified Scrum Developer program. And we've been training a fair number of folks primarily in Perl teams on the basics of technical practice. So that teams practicing Scrum are getting the full benefit.
But even absent that, it's amazing what difference it can make just going to something as simple as Scrum where you can actually try to get something done every few weeks and the changes that makes, particularly if you couple that with a reasonable amount of retrospective at the end; paying attention to what went well and what went poorly and doing more of the stuff that went well and less of the stuff that went poorly then teams get better pretty fast. And that's a really good thing.
I think there is a lot to be learnt from Lean and Kanban I believe there is place where the right way to start is looking at Kanban. Kanban has the attribute that in order to get started with it does not require you to make wholesale changes to how the company works, it is simply go out and observe what happens where value is added, where things stop in the process and notice that and begin thinking about how to do something about it. Contrast that to what we say we have to do to actually do Scrum, which is basically rip your chart apart, create this cross functional team with the person from the business side, put them all together, do all these things before you can start the first iteration.
If you can do that it's great, but lot of organizations can't do that and therefor starting with Kanban allows you to see the big problems in a company which are quite often not related to software development, you see organizations where it takes six months from the time an idea gets realized to a project getting created, takes three months to write the software and then another six months to get the software out into the public hands, you have a year and three months go by and you have fifteen months of which three months were software development. If you took no time at all to do the software development you will still take a year to get something out the door, then the software development is not the problem, and Kanban shows you those things.
12. Do then organizations need to adapt in order to be able to use Agile practices better?
I believe they do. If it was the same, if you didn't have to adapt it would be the same, therefor you'd have no change. I spent the first half of my career working in very large corporations who sometimes were incredibly successful and followed by periods of incredible unsuccessfulness and very cyclic, and they would come along every year or two with some grand new idea. And we would go to classes on this grand new idea and we would put up lovely posters about this grand new idea and we would have new stationary notepads printed up with things about this grand new idea on it, and never once would anybody say here is what you are going to do differently tomorrow at your desk. And the thing about Agile and XP and Scrum is we tell you that tomorrow you are going to start doing your work actually differently. And that's change and the other thing is window dressing.
I think it is getting people throughout every level of the company that is involved in the change to understand why the change wants to be done and how the change needs to be done, to educate them on the new processes and allow people to figure out what actually works in their environment. Our friend, Alistair Cockburn - whom you may be speaking to this week or not, I don't know - used to travel around in his previous carrier looking at teams around the world to see which ones were effective and which ones weren't ineffective, which ones had a process that was successful or not. He believed that a process was successful if the project was or appeared to be successful, and the members of the team would do the next project using the same process.
And he discovered that teams that were successful by large didn't follow any process that was actually written down. They followed their own process, one that they created themselves, that met their needs. So even within something like Scrum or XP or any of the other real Agile methods, you want to have room (and I believe they have room in them) to allow the team to figure out how they can actually work in the little box that we've given them; and that I think is the key to success overall.
Well, I would recommend hiring some really good consultants.Obviously you want to do that. As I said to you earlier, the big change that you have to make in doing XP or Scrum is finding the business person who likes to work with your team, not just a couple of hours a week, not just phone calls but is willing and able to come in and spend a fairly large amount of their work day working with the team. Over time you may discover that you may wean off a little bit but maybe not as much as you might think; so the first big change to make is coming up with the customer or the product owner who can help you actually get the project on the way and will guide the project. Scrum and XP both believe that that person is the one who will be responsible for the success or failure of the project, everybody else is following their lead in doing the things they see need to be done, but it is their job to steer the project to the best possible outcome.
That is a difficult thing. And I don't believe there is a clean simple answer to that. What you have to do there is almost entirely dependent upon your company, what is important to the company and sometimes what is unimportant to the company. Sometimes you find that person because you have a project that must be successful, must be completed by some date otherwise something really bad will happen, that sometimes makes a really good project for your first project. Other times you find an organization that has a need but the outcome is not really important to the company. But you find somebody who is passionate about new ideas and you find the person who is passionate and the fact that the project is not important may not matter because you go off and you learn things. And you do that in a way which is less pressure packed then if the project is ultimately responsible for the life or death of the company.
Finding the right person and finding the right project is going to differ from company to company, organization to organization and week to week within a company, I believe. So it's a matter of being able to navigate through the company and find someone who has a passion about wanting to get something done as opposed to doing what you have always done before.
16. That sounds like a really big challenge in a lot of companies.
It's the biggest challenge there is. The biggest challenge about doing real Agile is finding the people on the business side who are really willing to devote the time and the effort to actually make it work. As I said, that is the biggest dysfunction when we find is that you have somebody who is just another business analyst who has now got this job of being the product owner and they don't really know what the answer is and they're turning around and make decisions that are overruled repeatedly because they really don't know the right answer.
17. What then can that person on the business side do or become to be more effective in that role?
To be more effective in the role of product owner? There is training, which I think can be important depending on where you are starting from, there are some really good folks out there who teach some good training, I don't happen to teach that training, so I am in some way biased against it, but I believe there is really good training and really good folks who do it. The best thing you can do is learn from your team, the whole idea is to shorten feedback loops and not be afraid to be wrong but always be certain about what it is you want and make it look like - you know when the cat jumps off the table and misses the floor somehow and does that thing where it jumps up and says "I meant to do that", the product owner should always look like they meant to do that. When I talked the other day that we reflected on the idea that Kent Beck had a while back which was that the product owner role, the customer owner role in XP was the biggest mistake in XP. But he didn't know what to do about it.
If he would have talked to Mary Poppendieck and Tom
18. What do you think that product team might look like?
I think that, depending where it is in the lifecycle of transition, but you can imagine bringing together folks who have skills in different areas that all need to come together to build something, both technical and business, and put them in close proximity, put them in an environment where you have tight feedback loops, and put them in an environment where it is ok to stumble a little bit and let them figure out how they can move forward and share all the knowledge around. The big technique that comes out of XP to do that is pairing. If you have a person who is good at building GUIs and somebody who would like to be good at building GUIs or needs to be better at building GUIs, you have them sit down and work together and after a while the person with the great GUI knowledge will impart a fair amount of that to the other person.
If you pair on customer things, on product owner stuff, take technical people to the meetings with the product managers so they start to learn the things that the product owner is learning. So you put them together and you live a little bit of the other person's day everyday, after a while you will pick up enough of it where you can start adding to the conversation. We've never seen a situation where we had a new person on a team that was working in this sort of style that they were not able to contribute new and interesting ideas very quickly in the processes and thought of things these wizened old guys on the project didn't, never occurred to us because we knew you couldn't do things that way so the young kid out of college doesn't know that and often proves that we are wrong. So that is a pretty cool thing to do and I believe that can happen on the product side as well as on the technical side.
I hope it is. To the extend that I have any influence on what happens in the real world I will do what I can to make things moving in that direction, but I think the natural evolution is going to be to pair the teams to come together to be more homogeneous as opposed to more specialized, and that has to include product owner in the same way it includes quality assurance and data base administration and all the other specialized skills the team needs to have.
There's lots of things you'd like to see happen, lots of things I would like to see happen. Some of it is reversing trends that are counterproductive. We have had over the past 10 years a splitting away from what happened at Snowbird. If you think about the years before the Agile manifesto, you had several processes; several thought leaders out there who had different ideas about how you should do software but realized they were pretty similar. And they came together at Snowbird and hammered out those similarities and put together the Agile manifesto and lived together in peace, love and harmony for a short period of time. So that feeling of peace, love and understanding sort of started fracturing after 9/11, I believe, when jobs become a little more difficult to find, we started living in zero-sum game. So people kind of went their own way. And I believe now that we are starting to come back together and realize that the ideas most of us have are really the same ideas.
And as you mentioned earlier, we have some new ideas coming in, Kanban and Lean are coming into the mix, and instead of holding up our hands and saying "This is what XP is, we can't hear those things, this is what Scrum is, we can't hear other ideas", we should pull them in and make sure that whatever we're working on, whatever process we are trying to use has the benefit of all the good thinking we have out there in the world. So maybe iterations are a good way to start, but maybe what we want to go to is a more continuous flow process, because we see the benefits of doing that and not be worried that the Gee-Wiz doesn't say anything about that in the Scrum guide. Actually the new Scrum guide does say something about continuous flow, which I think is wonderful, so we want to pull all those ideas in.
I think that is the big thing I'd like to see, recognition that all these ideas are out there and all of them are valid in certain places and we should try them enough to know when they work best and not be afraid to use them just because we didn't think of them. In our talk yesterday, one of the things that we pointed out that we did wrong, that Ron and I did wrong, was that in Extreme Programming Installed we broke down how you do iteration planning. And it's been picked up, Mike Cohn's work on how you do planning, is basically retelling that process, and it's wrong, that's not how you ought to do it. What you should do is take your stories down to a small size that you believe was actually can be done and commit to doing some level of those every iteration and count how many you get done and that's it. Don't worry about velocity calculation, don't worry how many story points a particular task has, don't do any of those things, just build the thing that the product owner wants in small little slices and count how many you get done and predict based on that slope of your line how many you will have when the money runs out.
A second grader can do that as opposed to, I have trouble figuring out how to do the other thing and I wrote the chapter on how you do it.
21. Is there anything else that you wish you had known ten or 15 years ago that you learnt since then?
There must be a multitude of things, it's hard to know what I knew then and what I know now, because I always tend to believe that I know everybody, and everybody knows me and I've thought of tried everything there is out there right now. When you do this for as long as I have it's hard to remember when you knew things and when you didn't.
What do they really need to stop doing? There are so many things that they need to stop doing. As I said, stop doing that silly planning process is a big one. I think the biggest one is stop believing that being distributed across the face of the earth is just as effective as being together. I like to say that you shouldn't have your team members farther apart than we are from the International Space Station, which is about 125 miles away. Going to South-East Asia to save an extra two dollars actually doesn't save very much. The world is getting to be a pretty complex place and sometimes you find that the team you need to work on your problem is 1000 miles away from you. What you need to do is figure out is "How do I get my problem moved over there" as opposed to believing I'm going to be able to work effectively in the style that XP and Scrum and Agile calls for over a telephone line. So, get in the same room, discover how that works for you before you try anything other than that and realize that that thing you're feeling in your gut is pain and telling you to stop doing those things.