This article is part of the Agile Manifesto 10th Anniversary series that is being published on InfoQ.
The authors of the agile manifesto asked us ten years ago to rethink the way we (programmers) collaborated with our customers. I, along with my PhD advisors, Robert Biddle and James Noble, were intrigued, hopeful, and also simply keen to understand how it all worked in practice. Not how people think it should have worked, as is so often reported, but how it really worked. We spent the next six years understanding this, and over that time, visited eleven agile teams[1] in five different countries. We visited teams across a variety of industries, with the teams ranging in size from 5 to 60 people.
So what made agile so different? Firstly, timeboxes (iterations or sprints depending on your preferred terminology) allowed the customer to learn over time what they really needed the software to do; what was the “right thing” to build. One of the customers we studied explained it this way:
“If we’d had to come up with requirements to the [nth] degree right at the beginning, I think it would have been very difficult because I hadn’t worked with [new software / business process] so a lot of the time I was sort of learning as I went and once … I saw the bare bones thing, it was much easier to make more detailed decisions for the next level.”
—Customer, KiwiCorp
Agile also encouraged us, as programmers, to consider daily interaction with, and feedback from, our customers as normal. We should no longer expect a document to be “thrown over the wall” that described the customer’s perfect system. Instead, we would be an integral part of the customer learning what the system needed to be. The good news is, that in our research, we found that customers loved this new way of developing software:
“Overall – I love this approach to development and I’d certainly like to use it again in any future projects I am involved in”
—Customer, KiwiCorp
Customers also specifically mentioned how much they valued the new collaborative relationships they were establishing with the programmers, for example:
“[Previously] you write a document, you get feedback on the document, it goes into a traditional waterfall … technical design specification, and it’s not until the product comes out at the end that you realise, well somebody screwed something up …. But you get the pleasure through XP of having that close, close, close relationship in defining the product right at the time … one of the great powers of XP for me was that … I could leverage … the collective intelligence of the whole developer group that I work with …”
—Customer, EagleCorp
So was everything perfect in the teams we studied? No. We are human. People are not perfect, no matter how much we would like them to be. Humans are also great problem solvers, and in this article we will describe two practices that emerged within the successful whole teams that we studied.
The first of these whole team practices is one we have labeled Customer’s Apprentice. This practice is best illustrated with the story from the team we initially encountered it in. In this team, the programmers were becoming increasingly frustrated with their customer. The customer was not providing enough user-stories to keep the programmers busy. After a few months, one of the programmers became so frustrated he volunteered to help the customer write user-stories. The coach of the team related the story as follows:
“[Programmer] said well why don’t I spend a day and go and write [user] stories with our customer and that’ll help him out, that’ll help him out in terms of (a) they’ll just be another … warm body writing [user] stories (b) [they can answer] any technical questions he has and (c) it gives me an insight into how difficult or not difficult it is. And if it turns out to be really easy then we’ll continue to hit our customer and we’ll track spikes through his chest. But that’s not what happened. He came back and said he kind [of] felt that oh wow, there's really kind of a process to writing [user] stories and to mine requirements and go hunt down the people in the business that want something and to get them to explain what it is that they want and so on and so forth. And it made them … it humbled them a little bit I think and that made the relationship much more productive … And it completely turned things around for that group. In that sense the developer had a little bit of insight into the pain in the customer’s world.”
—Coach, SwiftCorp
In becoming the Customer’s Apprentice, for what ended up being two iterations, the programmer did indeed help the customer move forward. But, perhaps more importantly, he gained a deeper understanding and appreciation of the extent of the customer’s task. In this story, two companies had recently merged, and this system was to become the standard system for the resulting company. The customer was encountering a lot of politics. The larger business community was very afraid of what the change might mean for their jobs. The programmer was exposed to the fine line the customer was walking to develop this system, and he was able to take that understanding and empathy back into the team.
In another story, the programmers were not frustrated with their customer, but the customer was overloaded, and the programmers noticed. Two of the programmers volunteered to become Customer Apprentices for an iteration. The programmers worked closely with the customer, helping the customer in their day-to-day tasks; helping to shoulder some of the load.
It is this type of practical help from programmers that helps to create whole teams that include the customers. Contrast these two stories to a story from another team, where the customer had been working 70-80 hour weeks for the last twelve months. In this case, the programmers (and the coach!) who religiously enforced a 40-hour week for themselves were not even aware that the customer was overloaded.
Sometimes, as programmers, we are not always aware of why a customer is annoyed with us, or is too busy to meet our needs. This practice, allows programmers to “walk in the shoes” of a customer, just for a short period, to develop insight and empathy for the work the customer does, that might otherwise be hidden from us in our day-to-day job as a programmer. We found that the most effective teams used this practice at least once every three months. It doesn’t have to be an overly formal practice, a programmer simply volunteering to “help out” can work wonders.
The second of these whole team practices is one we have labeled Programmer On-site. Again, this practice is best illustrated with a story; in this case, from a team I recently coached. One of the programmers on the team was very concerned about the user interface of the application. From his perspective the interface was poorly designed, as the interface required the call centre operators to scroll through pages of data while on a call. Nothing the on-site call centre representative could say persuaded him that it was an effective design. The programmer raised his concern regularly during the next month. The tensions between the programmer and the call centre operators were rising, effective collaboration was stalling.
An opportunity arose for the programmer to visit the call centre. The call-centre was located about a 3-hour train ride from the project office. Once he arrived at the call centre, the programmer was assigned to observe one of the call centre operators. The programmer started the session by suggesting to the operator that the user interface must be “awful to use”. The young operator replied: “no, it isn’t … let me show you why”. The programmer then spent the next hour watching the operator take calls and use the application, and he suddenly realized why the user interface had been designed the way it had. Despite all of the previous discussions, it only really made sense to him once he had seen the user interface in action. But, our story doesn’t stop there. While the programmer was observing the operator use the application, he observed a step in the process he thought he could eliminate with automation. It wasn’t something the operators had thought to raise in the workshops. Before the programmer left the call centre, he worked with the operator to write a user-story to capture the idea. The team developed that user-story in the next iteration, and when that feature was released a month later, it reduced the amount of time the operators needed to spend on each call.
It isn’t enough to simply “talk to” end-users. We sometimes forget that deciding what to build is one of the hardest parts of software development:
“The hardest single part of building a software system is deciding precisely what to build … the most important function that software builders do for their clients is the iterative extraction and refinement of the product requirements. For the truth is, the clients do not know what they want. They usually do not know what questions must be answered, and they have almost never thought of the problem in the detail that must be specified”
— Fred Brooks, 1995. The Mythical Man Month: Essays on Software Engineering, Anniversary Edition. Addison-Wesley.
Programmer On-site is a simple practice, which along with time-boxed or iterative development, helps the customer work with us to determine what the “right thing” to build is. In practice, it is surprisingly easy to organize programmers to observe end-users in action. Sometimes we need to think a little “outside of the box”. For example, a team that worked in the energy-trading domain initially thought it would be impossible to observe traders, as the trading floor was “off-limits” during trading hours. A quick chat with traders, however, led to a plan that worked for everybody. One person would video a trader during a high volume trading session. The trader was then invited to a lunchtime session with the programmers where he explained what he was doing (and thinking) as the programmers watched the video. The trader commented that he felt like a “star”. He loved all of the attention. More importantly, however, he was suddenly able to communicate with the programmers why he found the system so frustrating to use. He had specific examples to discuss, and the programmers could see the direct impact of what he was saying. Finally, they were on the same page, and able to collaborate together towards a solution.
To conclude, ten years ago the authors of the agile manifesto asked us to rethink how we (software developers) interacted with business people. We have made huge strides forward in creating collaborative partnerships. It is not perfect. Human relationships are involved. It is important to remember that we are “in this together”, and it is not an “us” versus “them”. This article has outlined two practices that help us remember that. Our intent with agile development is to make the most of our differences to help create great software that solves end-users problems.
About the Author
Angela Martin wrote her doctoral thesis on “The Role of Customers in Extreme Programming Projects” at Victoria University of Wellington, New Zealand. This article is based on her thesis. She is currently a lecturer at Waikato University in Hamilton, New Zealand. She also teaches the Agile Methods course within the Software Engineering Programme at Oxford University in the United Kingdom. She has over fourteen years of industry experience, including as an agile coach, and has served a two-year term on the Agile Alliance Board of Directors.
[1] We use a number of quotes from the interviews to illustrate our findings; names have been avoided or invented to preserve anonymity.