Originally a Botany graduate, Paul moved into computing in the hope that it would offer paid work. In a 18 year IT career he has worked in many roles on many and varied projects, frequently working as an independent consultant allied to Informate International of Brussels. Paul is the expert on domain modeling in UML and contributes strongly to the areas of process dealing with transition from business requirements to system architecture and design. InfoQ editor Kurt Christensen recently had the opportunity to ask Paul about his thoughts on agile development.
InfoQ: Tell me a bit about yourself and your experiences with agile development.
Paul Oldfield (PO): I've worked in a wide range of environments, from 'agile' to decidedly not agile. These have included things fairly close to Feature-Driven Development (FDD), and some very self-organizing teams but never Extreme Programming (XP) or anything overtly claiming to be agile. Currently I am associated with the outsourced software development for a customer in the UK, who has an embryonic interest in Lean Software Development that might grow into something interesting and useful.
InfoQ: In your experiences as an agile practitioner and a leader within the agile community, have you seen a backlash against agile? If so, in what ways have you seen this manifested?
PO: I have met quite a few people who are very dismissive of Agile as soon as they hear the word. At the other end of the scale, I have met quite a few people who are immediately positive about Agile. Of these, very few in either group have a real in depth understanding of Agile. Of the many people I come across in the forums, blogs and cyberspace in general, this pattern seems to repeat, though by my own selection I come across a larger proportion of people who DO understand Agile.
As far as I can see, most of these views, both positive and negative, are based in hearsay; very few people have really been doing Agile as I understand it. Many more people think they are being, or doing, Agile. Those lucky enough to have interactive access to someone who does really understand Agile have a good chance of becoming Agile. Of the rest, some will get there through reading, observation, adaptation in response to perceived results.
Where there is a backlash, it appears to be in response to those people claiming to be Agile without actually being Agile. There are also a few instances of folk who reject the ideas because of philosophical differences. At some level this may underpin many of the objections to Agile.
InfoQ: Of the criticisms that you've seen leveled against agile, which - in your experience - carry some weight?
PO: All objections to Agile carry some weight, in that they will spread unless countered. They can gain momentum. Most objections to Agile also have some degree of justification in respect to "Agile as it is done here".
InfoQ: You said "Most objections to Agile also have some degree of justification in respect to 'Agile as it is done here'," but this becomes a criticism of agile methods - that if the problem is that teams aren't correctly practicing agile methods, then how are teams supposed to know when they *are* doing it right? And if it's so hard to do right, then is that a problem with the practitioners, or is it a problem with the practice?
PO: In my opinion, there's no such thing as "best practice" in a development world - this idea comes from manufacturing. If I sound like I'm going off-topic, bear with me.
In manufacturing, one expends a lot of effort early on to remove as much as possible of the variation in context, reaping the benefit by standardization and automation and the resultant reduction of unit costs. In development, the opportunities to reduce variation in context are limited, particularly if we wish to be at all innovative.
As a result, there is the possibility that we could have a team using a set of practices, and using them correctly, but still getting poor results because the practices are not appropriate in the context. When we factor in the possibility that the team is not using the practices correctly, we begin to see that there are many ways of going off track.
If we compare agile development to agile aircraft, the aircraft is inherently unstable, but has a really good feedback system to spot when it goes off track, and make the appropriate adjustment. This ability to make the appropriate adjustment is in short supply, and is, in my opinion, a major cause of the negative responses we get to agile approaches.
So, back to your question: the problem with the practices is that they are not universally applicable out of the box, though in general they are a very good starting point. One still needs the 'inspect and adapt' cycle. Unfortunately, many of the adaptations chosen by teams make things worse; they often choose to fall back to old, incompatible but familiar practices.
If a team is getting the results they should get, then they are doing it right enough for the moment. If they seem to understand what others are saying, and continue getting results after inspecting and adapting, then they are probably in a good position. If they still have problems reconciling what they are being asked to do in Agile development with their view of how the world works, then I'd recommend sticking to the prescribed practices and not trying to adapt.
InfoQ: Let's assume that when teams fail with agile methods, it's because they were failing in their execution of agile methods. Where do teams normally fall down? What aspects of Agile are hard for teams to get right?
PO: There are some classic mistakes. Pair programming doesn't mean having one person do the work while the other watches; it must be a conversation. I recall the first time I tried it after hearing a lot about it. I knew what ought to happen, up to a point. It turned out that I knew more about the algorithms, my pair knew more about the syntax of the language, so we both saw that we were going a lot faster than we would on our own. We rapidly found what each could contribute, where the other was having problems. Pairing with other novices doesn't always lead to so productive a conversation; it's a skill that might need to be learned.
I learned to do refactoring many years ago, when compilers were slow and we had to plan to recompile over natural breaks (sometimes overnight). All my refactorings were big. No small steps then re-test. Of course things broke - but the expense of having to fix and recompile drove us to take great care - and to fix all the breaks in a single pass if we possibly could. Later, structured approaches concentrated on getting the design right first time. Refactoring was a thing to be avoided. By going agile, the responsibility for a disciplined approach falls back on the individual. A common mistake I hear about is thinking of Agile as undisciplined.
The adjustments that the team needs to make are hard to get right. They require a degree of knowledge and understanding that is not very common. In many respects it would be good for the team to stick to the practices 'out of the box' until the necessary level of understanding has been developed.
InfoQ: You mentioned earlier that sometimes people will have a negative view of agile methods because of philosophical differences. Can you elaborate on this? Ignoring simple prejudices against Agile from those who have never tried it themselves, what - if any - are some legitimate philosophical differences?
PO: I'm not sure that I'd admit to any of the philosophical differences being legitimate, but then I would say that, wouldn't I?
I commonly come across people who do not make the distinction between manufacturing and development. They see the best practice and highly automated production line processes, and want to apply the same sort of processes. Mentioning no names, I have come across situations where payment was for time and materials. Profit was related to head count. Here, efficiency meant reduction in profit, and was only attempted when the customer could no longer accept that the head count was necessary.
Occasionally, I think back to the times when I first heard of some of the agile practices. Some of them just did not fit in with my world view. Luckily, I have a personal philosophy to investigate challenges to my world view, in case I got it wrong rather than the other person. Every challenge starts off looking like a new religion, but a bit of investigation can eliminate those that are pure belief. As far as I can see, the vast majority of people choose whose opinions they will believe. They accept or reject new ideas at a religious level rather than at a scientific level. They may apply some superficial reasoning process, but they believe they don't have time to make a proper reasoned investigation. This applies to agile adopters to the same degree that it applies to agile rejecters.
Emergent Architecture was an idea I initially found did not match my world view. Eventually, I discovered that what its proponents were suggesting was in essence how I'd been working on research projects for seven years, where the architecture evolved as the solution took shape. Of course the agile proponents applied a lot more discipline than I had at the time. My later, pre-agile efforts looked better because they applied discipline. This discipline was at the cost of flexibility, and was enabled because the degree of innovation in these later systems was far lower. The agile proponents had approached discipline from a different, non-manufacturing angle, and had thus retained flexibility. It took me a long while to turn this corner, because few if any of the practices worked in isolation. I needed to understand a complete working set before I could see how it could possibly work.
Of course, there will also be those who have tried Agile (or who think they have) and who failed. It is hard for folk to admit they did not understand. Surely the problem must lie elsewhere? I don't propose to take that line of thought forward, it doesn't get us anywhere. I think it accounts for a small but significant amount of the negative attitude, though. Their opinions also reinforce the prejudices of those who haven't tried it.
About the author
Kurt Christensen has been building software for twelve years. He's worked as a coach, trainer and computer programmer on a variety of different projects, with big and small teams at big and small companies. Kurt currently lives in St. Paul, Minnesota with his wife and two children.
