Today my focus is still on improving software development best practices. And I am working on what I would call next generation software development process, and I'm also working with aspect orientation of software development that would probably be two of the most important things. I have a number of minor things that I'm working with.
2. What are some of the problems with the AGILE movement?
First of all we are all Agile; we all claim we are Agile today. Saying that you are not agile, is like saying you are not potent. So we're all agile. And I have personally written papers about things like a resounding "yes" to agile processes. But also to the next big thing, which is the title of a paper. So we all stand behind agile principles , we can all subscribe to the agile manifesto, who doesn't agree that working software is more important than documentation, so that's not the point. The real difference is that when you talk about methods some actually classify themselves as agile methods and others don't do that, but actually there's a whole spectrum, there are basic unified processes that I've been working with for so many years, agile versions of unified processes. And actually I don't think there should be anything else than agile versions of unified processes to be frank. And the same thing with agile methods, some agile methods are more like traditional in process too. And that's the whole spectrum. But if we talk about the real fundamental difference between the different methods today we take the extreme agile methods and we take, on the other hand, the rational unified process for instance, the big difference is how you deal with knowledge; both agree we need knowledge, both agree we need experience to develop good software. We need competent people to do that. Without people we actually develop nothing. But the way you treat knowledge is fundamental different. With agile methods you primarily rely on tacit knowledge. Isn't that a wonderful world to be able to rely on tacit knowledge? Whatever you have in your head is useful. The other approach is relied on explicit knowledge, primarily, not only, because there's no one that can only rely on explicit knowledge, and explicit knowledge is structured one way or the other, it may be as in unified process a meta-model, process meta-model that describes different kinds of artifacts, and how they are related and so on. So that's the fundamental difference. The other things are: we borrow from one another, I borrow from the agile methods good ideas, like test first design and things like that, or test driven development, they borrow from us like user stories and other ideas as well. And that's how it should be. The fundamental difference is exactly that: relying on tacit knowledge or relying on explicit knowledge.
3. What is the fundamental difference between tacit and explicit knowledge?
A method that relies on tacit knowledge, how do you teach that? How do you know what it is since it only is in the head of people? How do you, if you put together a team which all come with tacit knowledge, how do you get them to agree on what we should do? And if you don't know what knowledge to base your project on, how can you possibly grow that knowledge? How can you possibly,[ there are tools], if you don't know the knowledge and tools you are going to work with? So that is the primarily problem with methods that rely on tacit knowledge. IF you take a unified process, the fundamental problem with that, you actually need to select what part of unified process you should use. The unified process is a framework, process framework comprises lots of things you can pick and choose; and you leave that to the project to identify what to pick and to choose. That takes time. You need to know the whole process to actually decide what I'm going to use. And that's not easy. Then you have to learn what to use, and you have to apply the knowledge. And finally you have to control it; it means change it because nothing is ever stable. That doesn't make a process agile, so unified process is perceived because of this size to be not agile. And as I've said for many years I've never really believed that a unified process or rational unified process as such, would be applicable to the whole world and software development because it actually requires an effort to adopt it. I said something like 15-20 years ago when I believed that a maximum 10-15% of the population will ever adopt it (and that doesn't sound good right?), but I knew at that time there's a way to change it. And that's what is really the future of a unified process thanks to being based on explicit knowledge, there is a future. If it wasn't based on explicit knowledge, but tacit knowledge I can't see how you can evolve that. What will Agile methods look like in ten years? I mean how do you grow these methods so they really move to produce better software, faster and with higher quality and with more fun?
The reason is that the investment it takes to learn a unified process as it is today, is a little to high,Small teams, small campaigns will find it too expensive to invest in that kind of training, and tooling and so on. So they don't do it. Larger companies do it and they do it all around the world all the time. That means I may be wrong in where we have programmers but I believe that bigger teams can do it, but smaller teams won't do it. However there's a way to change that. But right now I believe it costs adopting something. You have to study it, because there's something to study. With agile methods there's not much to study, thin books, beautiful books but thin! Whereas if you go to unified process, in particular rational unified process there's no one that ever will be able to read it completely. So, but if you try to really learn it, it takes some time. If you're going to train, because training in it, it takes time to go through classes and learn about it; and finally if you want to adopt it, there's no way you can adopt it without having coaches or mentors that help you. I wouldn't even try to adopt unified process without having people who have done it before. It's too expensive and too risky to do it. But that is how it is today; it won't be like that in the future.
5. What will be the next generation process?
I strongly believe that the next generation process will be what I call a smart process and smart is basically agile + + +, it means it has the properties of agility but it actually also help you to use knowledge. Imagine that we, in some magical way, could teach people and learn people to teach unified process, to apply unified process, to control it etc. We could teach them to do that. And only that part of unified process you really need and only when you need it just at the right incontext when you need it means small snippets of knowledge, coming when you need it and not before. Imagine that you can get this, does it then matter the size of a process? Does it matter if a process is 50,000 pages, 100,000 pages, one million pages actually I want as a big process as knowledge as possible, but I want it served when I need it and not before. So size, the bigger the better, not as we tend to say now today with agile methods, the less the better. Because the knowledge, assuming that the knowledge really gives you something, that's the beginning of a smart process, that you get knowledge when you need, not before. So the technology to do that means intelligent agents or similar technology. Intelligent agents are like in any role you play, you don't only have a pair programmer, the pair is actually the agent; you have a pair analyst, pair designer, pair architect, pair project manager, anything. And that means you can get support in doing your job anytime. So you have basically a virtual mentor waiting for you, to be ready to help you anytime. And there are scientific evidence that a pair of a human mentor combined with a virtual mentor is much more effective than just human mentors, human coaches. There are lots of scientific evidence, I can give you reference.
6. What actually is intelligent agent, is like the paper clip in Microsoft Word?
No. That is the exact example of something that we don't want people to even think about, when they think about the intelligent agents. These are intrusive. Intelligent agents are, I mean you can look upon it from a piece/pieces of software and you can look upon it as a component or as a class and you instantiate it and an instance of such a thing is helping the human in doing/playing some particular role in software development. Like doing requirements or testing and so on. The software here is similar to other software but it has more; it has, is driven not only by procedural code but actually by rules. These rules have explicitly formulated goals . For instance, the goal could be to produce a component, that component should not just be a component, but a good component. And then there are rules for how we create good components and these drive you to develop a good component. So agents are nothing really dramatic in you, it stands on top of experience and knowledge which we've had for 20 years, like Expert system, Rule-Based systems and so on. But now it's been componentized, so the problems with these old systems were that they became monolithic and used systems and they're hard to change and control. Now we have componentized them and the trick is to find good agents. But that is similar to finding good objects, good components so there're some rules to identify good agents as well, and some techniques to do that.
Agents are there to support the people who develop software in different roles. Typically there will be an agent to help you do the use-case if you do a unified process and then that agent will know what a good use-case is. And we have programmed this agents to do what we think are good ways of doing use-cases. It could be, if you think a good way to do a good use-case is to do a user-story, a very light version of use-case, you can do that. If it's a good way to write a more precise use-case with pre-condition, post-condition and defining all the different things that will later become test cases, then you do that. Because, basically use-cases are test-cases, so it's lots of rules how to do good test-cases. So you will have that for basically all the roles we play in software development following then best practices of what we believe is good best practices.
There are scientific proofs that in some cases humans really don't do as good work, only, but actually if you work with some virtual mentors instead of human mentors you get a much more productive result. I can tell from my own experience that if you use coaches to help people adopt any kind of methodology; these coaches usually get the same questions over and over again. So for instance people who work with unified processes they have answered the question "What is a good test-case?" a hundred times and then they have to discuss how to do a test-case and so on. They are elementary things. That became boring and you don't like to do that, so if you instead can have a virtual mentor that takes all this burden, to discuss what this well-known knowledge is, instead of using humans to do that. And actually that virtual mentor is never tired, is available night and day to work with. Instead the human mentors are doing more advanced things, like "change management", how do you change the organization, how do you make people and adopt the principles of software development and so on. That is much more qualified, that is our own experience. So when we go in and work with a team, we rather have fewer human mentors in an organization and instead everyone gets a virtual mentor, and we get a better result and is much more cost-effective to the company. The other thing is, if you look upon it, there are scientific results, I think from Penn State University, that they actually have measured and studied what happens in a commanding-control system, how users react when they are in a fight situation, and the result was that humans, when they are in a normal situation, no stress, work very well together, it's not a real problem, but as soon as they get into stress situations, they take wrong decisions and they actually, there's high risk that they do things that kill people. So, whereas with virtual mentors or agents to help, they get a much better result. So I think that pair programming, if that is to convey knowledge from an expert to a less expert new guy, it's basically a very expensive way of doing it, because we talk about mouth to mouth, where if you have an expert he should maybe coach the whole team or at least a part of the team, instead of just coaching one individual at a time. And it also is very boring to this guy to teach the same thing over and over again. Personally I would never stand it, and I know many other people that would never like to do that. So I think, I wouldn't exclude to have some support in terms of pair programming but it's probably more effective to have one expert to work with many people. And then the social part of it, we can't talk about it; I really don't want to get into that.
It does. Because, first of all I don't look upon process as most people do, I look upon process as a knowledge, knowledge-base. It's not about controlling. When I started to develop OBJECTORY that became rational unified process. I did it not because, I actually didn't know much about project management, I was project manager but I didn't know much about rules for doing that so I never really thought about spending a lot of energy on formalizing the boring process stuff, the thing that all developers think is boring and hate. I only focused about one thing, that I really wanted to do: how do you do good things, good components, good interfaces, good operations, good attributes whatever, good code, good test cases, good use-cases and so on, all of these things. I have never seen anyone really said formally in any book of any substance, until we did it. So that is really what I was interested in. Don't look upon process as the project management or something for managers, look upon it at knowledge-base. So of course if you are knowledgeable, you develop it better software , but if you're not knowledgeable. For me that's the simple answer.
10. Do you agree that XP is largely driven by developers, whereas unified is more driven by managers?
Actually no, I don't do that. I think that is a perception and whereas a unified process, basically you can not really have a project of any nature where you have costs involved, people want to pay you money to do the job, developed management is involved; they want of course to make sure they get something for their money. But in all unified process projects that I know of, when it comes to the technical work it's driven by the experts, the technical people, so I don't see any difference with the extreme programming in that sense. There's a perception now let me say that. One of the things I haven't seen in many years but we are back there now, is that there's so much religion about methodology; during the early '90s when we had all these different methods about doing object oriented software development, you know we had a Grady Booch, Rumbaugh, I had my book, there were ten of us, that has silenced completely for many years now, and it was religious war, now we are back and have religious war again because it becomes religious when you have a lot of arguments that are not really based on facts. I try to stay out of these discussions and focus on discussing what actually I think I can measure or have evidence for.
OK. Let's first say that yeah it's true that we put together in the late '90s a proposal for a standard and that was adopted and that was, as you know, it was originally started by Grady Booch, Ian Rumbaugh, and myself . Then we got a lot of adoption. Basically, every methodology at that time gave out their own notation and we actually worked hard with them to make sure they could use UML to replace our own notation and our own language. In that camp there was another competitor that was really serious and that was STL, STL is the predecessor of UML that became the standard in telecommunication back in 1976, and then '80 and so on. I actually participated in developing STL, so that was my first modeling language, basically, that became a standard. What we did, we knew that we've developed a new kind of language, because typically these languages are developed without having an underlying detailed semantics; reason is language should be able to work in front of many other languages like, at that time we talked about CHILL, ADA and so on. They could have many different underlying semantics and you don't want that to be reflected in the modeling language. But UML won that vote too so now STL is part of the UML, included in the UML family. So UML has really been successful in the modeling language family. Now what's the critique against UML? Where does it come from? It definitely comes from people who've never used the modeling language, and it comes from people who have been working with formal semantics of formal languages. Well I've done that myself too, I did my PhD in formal language on formal semantics, and when we did the STL we used formal semantics we actually used Vienna development method to specify both the syntax, the abstract syntax, the concrete syntax, the static semantics, the dynamic and operation semantics, and we did that down to nitty gritty details. With UML we didn't go that far because we actually felt that very few people ever read formal mathematical semantics. But we had abstract syntax, we had static semantics in a formal way. Now you go on and you look upon programming. How many programming languages out there have ever had a formal semantics? Very few of them. I think these languages are not really used anymore. Does C+ + have a formal semantics? No way! Not even C has. Maybe it has. JAVA? Is there formal semantics with Java? No way! So we have still people use these languages with success. So a major critique to UML is that is not precise enough. Well that's one critique coming from people who've never really used modeling language like that. Then the critique comes from vendors, which actually want to develop their own modeling language. That's very natural too. It has to do with their interest in having customers locked into their solution and not so much about the language. If there was a problem with language, they could fix it. Now there is one relevant critique, and that's the size of UML. UML 2.0 has become very large, it has become heavy, it's very hard to learn and study. So what do you do about that? Well, with intelligent agent you fix it. You know that right? The intelligent agents fix a lot. Now having said that I don't think these are silver bullets, but they actually help you in adopting things, they don't solve other problems but they help you adopt, they help you to learn, they help you to apply technologies.
12. UML seems to be becoming a programming language. Is that appropriate? What's your take on that?
It's a little too early. I think we need better platforms to get there. But what's good about it is that what really remains to make UML an executable language is really very little. But the sooner they'll make it an executable language, it has to be supported on platforms. So it's too early to do it today because I would like to see Microsoft and IBM agree on something like that. But if they did that so we could have one executable language, the good thing with UML is that you can actually add and extend it, so it would be able to support many different programs and styles, so you could have profiles of UML and different variants of it and you can pick and choose. But it is too early; we need better platforms than what we have today. So I'm looking forward to when we have better platforms.
13. So what do you think about MDA?
MDA is based on something we've basically have been hearing for many years, model driven development. I have a little problem with calling it model-driven architecture, because it's not only about architects it's about everything we do. That is an old idea; I mean we all believe that is good, I think everyone will stand up and say that model-driven development is good because we do that, we've done it in many different disciplines not only in software. You start by drawings, and you move from drawings to other kind of drawings and so on. The interesting thing with how OMG has presented it, is you move from different kind of models and you have transformations from these models, and you can define rules for doing these transformations. I strongly believe that we can do that. Actually we have adopted part of it already, so we make an analysis model, which is basically a model where we don't really care about anything else other than the problem, we specify a problem in a formal way and we actually make it executable, we do that by using just Java, plain Java, and this model is then the beginning for making a design; so we add things like persistence or cross-cutting concerns, as we talk about it in the aspect world, we add security, we add all kinds of monitoring, and we add distribution and we do that as aspects, so we don't really change the core, this original specification implemented in Java , but we add it in form of aspects. And that makes it very easy. It's very easy for customers to do; we keep the core of the problem nice and understandable and we add the specific stuff. I believe in it and I believe in it because I think I've seen we can do it and we are not really all the way as OMG originally described it, but if we take it in a more modest way we can do it a part today and we can of course be able to do more and more in the future. All the big vendors are doing it, some kind of modern architecture, a modern development. I think the critique against this is coming from the same camp that criticizes UML.
14. What do you see about AOP is it going to become as mainstream as OOP?
First of all there's a huge difference in what it means to adopt AOP and OOP. I actually don't like to talk about it as a programming technique, I view it as a software development practice, but the big difference is that you don't really need to think much differently it's not the new paradigm shift. You can think in terms of objects and adding aspects to objects is just a refinement of the technology. Instead of dealing with objects that participate in many different use-cases or implement many different features, you actually slice these things into the object, so a use-case may hit many different objects and you can keep these separate from one another. And you compose them through aspect mechanism. This is not a new idea, aspect-orientation, there have been similar solutions to this problem about cross-cutting concerns and separation of concerns, 20 years go. My first paper talking about very similar language as you do when you talk about aspect-orientation was written in 1986 and I'm pretty sure this would become a very important technology, but when, this depends on vendors. I'm really worried for the way the programming community, the aspect-oriented community treated, it treated too much as low level technology and they also want to move this technology upstream now people talk about early aspects so aspects are moving to all the disciplines and they do it on the conditions over programming, over implementors, whereas it needs to be a technology that supports the whole software development life cycle, from requirements all the way down to executables and testable codes.
15. Any final words?
The final words would be my standard final words. I really believe there's a lot of interesting technology. The last 30-35 years I've been working with software have been very exciting, we have moved very slowly, components were already there in 1970, the use-cases and many of the techniques, UML was there, but it has moved slowly. But we have never been as far as we are today when it comes to best practices and adoption of best practices. I believe that the 30 years I've been working with software have been very exciting, but the next 35 years will be even more exciting. But where we are moving with all this is to make the world more human. Today we are very machine-centric and I want to move away from a machine centered world to a more human centered world.