My name is Steve Adolph, I live in Vancouver, British Columbia, and I’ve been in this business developing software longer that I care to remember, and I like to sometime put in my bio, I could remember Fortran-G and OS-MVT, so most of my work is been involved with building Control Systems for Railway Signaling and Telephone Switches and I got the Agile bug approximately 15 years ago, and thought of it as just “yes, of course, why wouldn’t you develop software this way?”
3. Electrical Computer Engineering, will you tell us a little bit about your PhD topic?
My PHD topic was, I was investigating how people manage the processes of Software Development and there is numerous theories about how people should work, we’ve got methodologies that say this is the way people should be doing Software Development and something you notice is not a lot of people actually use published methodologies, and so the big question I had was, well why? And a good way to answer that might be well, why don’t we go and ask people how they actually manage the process of Software Development and what I generated out of that data was a theory called Reconciling Perspectives of how people actually manage the processes of Software Development. And the bottom line that comes out of that theory is people need to talk with each other, it’s a pretty shocking conclusion and seems to be blindingly obvious and yet it seems to be one of the major impediments to our industry being able to make progress, if people won’t simply talk with each other or talk with each other sufficiently in time.
4. So why is that, is this dependence, we are technologists, we email as the way we communicate?
That is one of the ways, that is a number of reasons that people do not felt to talk to one or another, each one of us has our own point of view of the world, I’m an engineer and I see things from an engineering point of view, is this one stereotype and business people see the world from a business point of view, and the problem is you are looking at a problem through those different lenses of your perspective and so when you hear something, you filter it through that perspective and unless you actually go talk to the other person, you may not get data that challenges that perspective and you promptly go out and proceed in the wrong direction. And this is not just fundamental to software, this is fundamental to many, many humans endeavors, it’s been studied for years and years, all I’ve been able to do is just say this is talking to the people in the Software Development World, this is a serious problem that people are not on the same page and there are numerous impediments to getting people on the same page.
5. So what are those impediments and how do we overcome them?
A lot of those impediments come from, for example I relate to an example that was really tragic example, called “The Man Gulch disaster”, where something like 19 smoke jumpers lost their lives in a major forest fire and what it came down to is from their perspective they could not believe the fire was moving as fast as it was. So their model of how a fire should behave did not explain what they were seeing and there is saying that goes with that, “believing is seeing”, you see what you believe, and unless you have something that explicitly challenges that you may not say: “What I’m hearing from you is incorrect, maybe I should challenge you, maybe I should reach out and challenge your view of the world”. So this first of all, do people even see that there is an issue, that was one problem.
Second problem was there is sort of a stereotype in our industry that people, developers are a little bit introvert. That is fine, introvert does not mean shy but unfortunately a lot of software developers are shy, they will not reach out and talk, they will not recognize that there is a problem and there it seems to be a cultural element in engineering particularly that if you have to ask a question that somehow that diminishes you. I like to say that one of the most wounding questions that an engineer or an wounding insult an engineer can give to another: “You don’t know that?” So there is a reluctance sometimes for people to reach out and ask questions. There is also the nature of engineering work, it requires quiet, focused time and we get into idea of “flow”. There is this inherent need for engineers to get focused so that they can resolve problems and being disturbed, or reaching out to get help or reaching out to talk to another person while we’re in that flow disrupts the flow state. So we have a natural desire to respect each other’s space such that we are not disturbing them. And so what we are really looking at is we need to setup mechanisms where people can get into that flow state and do their work, but also be able to stay connected to what is happening around them, so these are some of the challenges, a lot of the challenges and what we saw in organizations that were capable of doing that is they had people who took, but that was the attributes that really enabled organizations or groups to deal with that well, first of all was experience, individual and personal experience. The more experience people knew where things could go wrong, they have enough battle scars on them to know: “Yes, I should talk about this, because that is been a problem in the past” so they knew when they should start reaching out. The second was leadership, leaders played an important role and a lot of the Agile methods implicitly capture this. For example the role of the ScrumMaster is a classic one.
The ScrumMaster effectively being the person who is there to remove impediments, the ScrumMaster being the person who protects teams from outside interruptions. What we don’t talk a lot about, though, in the Agile World is the role of people who are what are called Boundaries Spanners or Connectors, people who keep the team connected to the surrounding organization. In Scrum and XP we have this cheat we call them the Product Owner and they are supposed to be this omnipotent person who knows everything about the organization and is able to communicate that to the team. Reality is I don’t think the Product Owner is a doable job, not by one person; and what we found in the most successful organizations is a number of people, the thought leaders in an organization, they took it on themselves, it wasn’t a role you gave a person, it was a role to a person took on, to be that boundary spanner, to go around and keep connected to other individuals and other teams in the organization and bring that information back to the team, that is how they stay connected “wait a minute - what you’re telling me what those guys over there are doing, but that doesn’t jive with what we are doing”. Ok, there is a perspective mismatch, we got to get together with that team and talk about it.” That was the kind of thing that needed to happen.
6. And that is not embedded in our organizations, in our processes?
It’s not embedded in our organizations and in our processes and the other important part about, this is not something people naturally do like if you look at for example the ScrumMaster Training, we tell people “yes, go out and talk to people” but we do nothing to develop the skills in people to do that. One great example in the world where we do is the training that is giving to the airline pilots. About 30 years ago the airlines were shocked by the number of accidents that were happening with perfectly good working airplanes and these were been blamed on pilot error. So one classic example was an air freighter over Portland simply run out of fuel and crashed, nothing really wrong with the airplane, it just run out of fuel and crashed. And so the airlines became really concern about this asking: “What is missing about our pilot training?” The pilots are not able to handle these situations and these plains are crushing. What the airlines found out it was nothing to do with the pilot training itself, in terms of their stick and rudder skills, their ability to fly a plane.
The worst airline pilot is a really good pilot still; what they found out was it’s the inability of the pilots to work as a team. The airlines actually put a program in to help pilots work as a team to solve problems. Eventually that training expended out to the flight and the cabin crew, to the ground crew, to the dispatchers for them to work together. Talking to most pilots, this is one of the big safety innovations that they’ve had in their industry, they called it Crew Resource Management being able to get all the people in the crew to work together to resolve the situation. They had to train the people to do that and what makes us in our industry to think we are any different, we tell people “here let’s all be Agile, let’s form teams” and yet we don’t train people how to be a team member, how to work socially with their teams members. This is something that we really need to take responsibility for ourselves, we can’t just ask people to form a team and start being affective, we have to train people in team skills.
7. To be a team, does not just happen?
Not happen, and unfortunately I have seen many good people, they are made a ScrumMaster, courses made a Product Owner and the team runs roughshod over them. The team has to know how to be a team.
8. So what do we do with that?
One of the first starts is when we form our teams, yes, we’ve got the technical skills, let’s take a page from the airline industry: social training. There are numerous companies up there who do this, successful organizations focus on developing their teams. Great example from the airline industry is for example Southwest Airlines, they focus a lot on their people, developing their people to form them to work as teams, to work together to resolve problems. In fact there is an interview; a saying is attributed to Herb Kelleher who was the founder of the Southwest Airlines. Southwest Airlines started off, was and is a low cost carrier. All the major airlines try to build subsidiaries that matched Southwest Airlines cost structure and effort to compete with them. Remember United’s TED, Delta had Song; in Canada Air Canada try to do Tango, none of those worked, none of them are around today, they all failed and Herb Kelleher said: “They can match our cost structure, they can’t match our culture” that is a wild thought, culture is a massive competitor advantage and they are focusing on developing that culture. What is really exciting and cool about the Agile Practices is a lot of the practices enable good social behaviors so you are getting into social-technical systems where you are trying to develop social and technical practices together. The technical practices enabling good social behavior on the teams, that is one of the big contributions of the Agile Software Development movement.
Shane: Thank you, that is a really lightning perspective.
Thanks, I apologize, when you start getting me talking about my dissertation I go on and on, I go into academic mode.
Shane: One of the areas that I know you’ve very focused on at the moment is the concept of Value Management and the need, in fact you session here is about the flow of the Agile Business Analyst. Agile says we don’t have these Business Analysts.
I think that is a fallacy, that is a fallacy. Let’s take Scrum because everyone knows Scrum. Scrum says you got a Product Owner, you’ve got a ScrumMaster and you have a team who is capable of doing everything, that’s it. Scrum did not say: “No, there is no architect on that team, no, there is no DB analyst on that team, no, there is no business analyst on that team”. It says the team is capable of delivering a product or service. Scrum just says they work together as a team and the Business Analyst emerged to fulfill a very real need. Good solid hardcore engineers most of them are not really good at understanding the nature of the business or eliciting requirements or developing models, that’s not where their head space is at. Let’s face it, when I started in this industry, and I consider myself one of the hardcore engineers, I want to be doing cool things, I wanted to make telephone switches connect phone calls and I was far more interested in voice sampling algorithms than I was in how was the company going to make money using this thing. The Business Analyst was the person who was really interested in bridging that, could understand the business, understand the problems, had the skills to take all the competing views and distill them down into a model that would work.
9. Is this the Boundary Spanner you are talking about?
Business analyst I found actually was one of the huge Boundary Spanners, particularly one side I was studying I remember the one Business Analyst, she was amazing because she was probably the person who kept eight teams focused. She was like this little bee going around from team to team, she had the vision of what the system was in her head and she will be working with the teams and then almost making sure that they understood what they were coherent with the vision for the system and also hearing what each of different teams were doing and going wait a minute: “What you are telling me what that team over there is telling me is different, we need to talk”. That was a huge role, to me that was one of the most important roles of the Business Analyst, keep everyone on a coherent vision of what that system was about, get the perspectives aligned.
10. So what is this flow, the flow of value from the Agile Business Analyst?
The flow of value comes from the idea that we are living, we are working in a chaotic environment now, change happens rapidly. Thirty years ago when we were in this industry change did happen a lot more slowly and the methods and practices we had then served as well. Today change is happening very, very fast and the focuses now on delivering value, the problem is what is value, what is valuable and the only way sometimes to find out what the value of something is, is put it out in the marketplace; whether that be literally the real market where we put something out for sale or we are getting feedback from potentially users.
What is this, what you had in mind and getting that feedback and finding out what is really valuable because the nature of a lot of our systems is that up to 60% of the features, the elements in a given system may never be used or contribute to any value and that is shocking because all of us are complaining that we can’t hire enough developers, we’ve got far more work that we can handle and yet 60% maybe is not valuable, so why we building it? Why are we wasting huge resources building things that contribute no value and it’s because generally when we are building our systems we are building at the point of maximum ignorance, this all goes back to Barry Boehm and software engineering economics and the cone of uncertainty and it’s just saying that at a beginning of a system when you are trying to lock down the requirements, when you are trying to lock down the budgets, this is why we are so bad at budgeting, this is why we take a shotgun approach to requirements. We don’t know so we hedge our bets, we try develop everything with the idea that hopefully one of these things will catch. Why not take a learning approach, why not change that cone of uncertainty into a cone of certainty, take advantage and learn: what do we think is valuable, let’s put it out there, let’s get feedback on it, “oh that wasn’t valuable, oh, but you want these changes on it”, fine, manage our value so rather then just Business Analyst being someone who takes big requirements and just chops and down into little requirements and feeds rights an SRS and hands the SRS over to a development team.
The Business Analyst now is part of a process where we are trying to find out what is truly valuable to this organization, what are the practices that we can put in place such that we can find that true value and use our precious development resources to be focused more on what the valuable features are than the features that are less valuable, and generate that value in a steady stream by getting constant feedback. This is what Erik Ries and Lean Startup was all about and I think unfortunately Erik chose a very bad title for its book, called Lean and it should be Lean Flow or something, the moment he put the word “startup” a lot of people tossed away his ideas and yet it’s valuable whether you startup in a garage or a large scale enterprise, validated learning, that is how we found out where the values and we generate that steady streams of value.
11. One of the statements I know that you’ve made is the Danger of a Large Backlog, why?
A Large Backlog does not allow you to change, you build up a huge inventory of work and what happens is now you find out that what is in this Backlog is no longer valuable or the market is change or you don’t get feedback in time, so there is a number of issues with respect Large Backlogs. Donald Reinertsen does a lovely theoretical treatment of this in his book, Product Development Flow, but the bottom line on having large Backlogs (which is what a lot of organizations end up having) is you have to manage that Backlog. First of all you simply wasting resources managing that Backlog, you have to generate that Backlog so you waste a large number of resources generating that Backlog in the first place, and worst of all because you’ve invested so much time and effort, your sunk cost in developing that Backlog, you are going to be reluctant to take advantage of new information and say: “Oh my god, most of this is garbage”.
What do we do, we take a learning approach. We understand what the large scale features are, we don’t have to break them down into find grain detailed features all up front, what we are saying by learning is: “Yes, what are the things we really know about now and what enables us to learn more about what we think might be important months down the road, let’s work on those now, put it in front of people, see how it takes, does it work? Yes, what does it reveal? That over there is important, not that”, so this should move up and we should refine and elaborate this and work on this next, and constantly learning so we get into a rote, we keep our Backlog short by keeping fine grained stories that we can implement now at the top and the large scale features that we maybe implementing off in the future, fairly large and fuzzy, and we progressively elaborate them so the BA is involved on going, we are not front loading the system anymore, we are doing all the analysis upfront, we are doing the analyzes progressively as the system is being developed. In the construction industry they call it “Fast Tracking”.
Shane: Steve thank you very much for taking the time to talk to InfoQ today, we really appreciated and look forward to your talk!
Shane thank you so much, I really appreciate the opportunity!