I am here at QCon, first of all, to talk about a topic which is Sustainable Software Development and also to have a little booth with my company Hello2morrow. Hello2morrow is here in QCon for the third time, so we’ve been in San Francisco before, we’ve been in New York this June and now it’s our third appearance here, we like the conference, very good atmosphere, lots of good speakers and good networking and of course we try to… we are also here to win new customers, for our products, for studying this analysis tool suite called Sonargraph and that’s working on pretty well right now.
I think there is still a big tension between the two camps. There are people who are living according to the Agile manifesto. This movement I think came out of the failed waterfall processes where people could very clearly see the wasted, enormous amount of effort at the beginning of the project, to plan everything to death and then find out that it wouldn’t work at the end when the project was finished so that obviously was not a good solution. Now the Agile movement was born, I think they went a little bit to the other extreme, no planning at all and we ‘re just the only user stories, we don’t talk about architecture. Architecture is something that’s created on the fly, as a side product, more or less, and I think, while that might work pretty well for a smaller project, we have two or three people working together, creating something like fifty, maybe a hundred thousand lines of code. That is something that it still might work pretty well. But if you are talking about larger projects, we would be talking about several thousand lines of code, millions of code lines and thousands of people involved, I think you need to find some middle ground where you spend some time for planning and creating usable blueprint and still using Agile techniques like having sprints or iterations and getting business functionally out as early as possible.
First of all, rapid release cycles are a good idea. You want to have proven functionality. You want to have something that you can show. What I think you should reserve a certain amount, a certain percentage of your time in each Sprint for what I call code hygiene, architecture, non functional requirements, scaffolding, blueprinting, all this kind of stuff and what we know from our own experience, how we develop our tool Sonargraph , is that we’re using these prints, we are using the time box iterations but in the beginning of something we’ve spent a relatively high percentage for the planning, for the scaffolding, for architecture, so the first sprint might seem we talk about 50% about the time is going into how do we build this, how is the foundation going to look like, what are the guiding principles for a design, so what are the cornerstones of that building we are going to build and then in the first few iterations we do a lot of that and we also try to get the first vertical cut through where we try to look about a complex scenario and trying to at least get a minimum implementation of that complex scenario done in the context of what we think the blueprint of that application should be. So it’s not going to be perfect, it’s not going to be pretty but it’s kind of you want to make it prove of concept, you want to make sure you are on the right track. I’m not saying that you should have a finished architecture on sprint number one, I’m far from that, but you should have a pretty good idea of how the foundation of your project is going to look like and then you use the next sprints and iterations until we find those ideas. So it’s going to be something like in the beginning you spend a lot of time for planning and architecture and stuff like that and it’s going to get less and less, but every sprint you should have some time for code hygiene.
Architectural work, I wouldn’t say it’s coding right away. Architectural work is more a discussion about the principles that you want to apply to your application, how you want to design your application, what kind of layers am I using, what kind of sub-systems do I have on my system, how are those different parts interrelated to each other and I think it’s a good idea to have a team discussion about that, because I think most people can contribute to this, everybody has its own ideas and if the team is able to come to a consensus about how to build this, it would be much easier. So I think it’s a good idea to have this in the team discussion. Of course led by an architect who maybe has a little bit more experience, who has done other jobs before and who can bring in additional value, but I think everybody should be involved in that process.
Harry: So as the process goes on, more people can have their hands on the actual implementation.
Exactly. I’m a strong proponent of that an architect should be coding. I always consider architects as non commissioned officers, so they should lead their troops in the field and they should make sure they are successful but they should be coding and should understand how to… the architect should be the best guy in the team so that other people could look up to him and take him as an example.
Harry: If you do a plan before, and obviously you should, you might end up with very large plans… I’m trying to get back into the planning…
I think, if we make a short break, when I’m talking about planning, I don’t mean making Gantt diagrams or something like this, I’m talking more about something like scaffolding blueprint for the application, so how are the different building blocks for this application looking like and what kind of infrastructure do they use to pin that altogether. I’m not talking about making project plans in that sense because that’s all given by scrum those methods and methodology, you have your back log and of course then you plan your sprints and they say “these sprints we’re going to do that, that…”
Harry: Right and then you have to help stack rank the back log and…
I think the important point is that a back log should actually have items for this non functional stuff, like the back log item should not only be coding items, should be items like designing the application and talking about quality is also a very important thing. What kind of technical qualities guide lines do we have in our process.
I see them converging actually, so I see that architecture is going to play a more important role in Agile, in this sense I was just describing that the first sprints, you do a little bit more of architecture and structure and design, also quality rule is very important, we didn’t talk about that yet. And then as your sprints continue, this is just something like continuous maintenance work, which you have to plan, if you plan your iterations you have to plan a certain amount of effort for every sprint for this kind of task.
That’s always a big question, the “Gretchenfrage” as we would say, because it basically means that if you add time for non-functional requirements, for quality management, for architecture management in your process, it’s going to slow you down. It’s going to make you slow in the beginning, there’s no doubt about it. So the benefits you’ll get out of it won’t come for free. The benefits are quite enormous, so we know from experience from our customer base that people really do technical quality planning, technical quality management, architecture management in their daily routine and in all of their sprints they can save up to 66% of maintenance cost, increased team productivity by 30%, just one customer, Black Dot software in northern Massachusetts and now Burlington the meantime. They’ve confirmed to me that they saved enormous amounts of maintenance costs, with just adding this to each of their sprints. But the point in that case we have much more a political and a social problem than a technical problem. The technical problem can be solved easily. Just define your quality standards, measure them with a tool base, so you have to add the tools into your development process and act on the results coming out of this tool base. So that’s not really difficult.
The difficult part is to bring the stake holders into the board. Because, since we are living in a very short term minded community here, everything is about the next sprint, the next quarter and not so much about the long term sustainability of what we are building; it’s a tough sell. But I think the solution for that would basically be to create more transparency so the stake holders could actually see the value that is produced by having better quality, having better architecture, having better maintainability and the key to that is software metrics. As soon as you make your metrics publicly available, if you measure them every day, publish them on a website, for example using Sonar as a very good platform for communicating this kind of metrics. Everybody knows where it stands and the quality is not a hidden issue anymore, it’s something that becomes visible and then people can also see the value. So if you talk about real numbers, it might be easier to get management and stake holders involved in that thing. At the end it’s still something to need to sell but I think the best way is to try that on a smaller scale, to see how it works out for you. I’m pretty sure that most people will agree after sometime that it works out in a very good way and then throw it down in a larger scale.
I think static analysis is a topic that is a toolset that should accompany the process. I think that’s very important because right now if you are building software systems, it’s all about functionality. The hero is the guy who gets the job done, so everything is about getting the job done, without really specifying what that means. And I think the big problem right now, of so many software projects is that getting the job done doesn’t contain any criteria for technical quality of what comes out of it. It’s all about functionality. You push that magic button, something magic happens in the background and the application is considered to be done. Of course we don’t know how it looks behind, if this is a big mess, if this is a beautiful application, if it’s well designed, if it’s badly designed. All these questions are not really asked.
Just imagine you would build airplanes this way. You would build an airplane and the airplane is an airplane and it looks like an airplane, nobody would want to fly in one of those things. So in all other disciplines where we’ve built physical items, quality is a visible attribute. You can actually very easily find is something is crap or not. And the software system is much harder because all is hidden behind the beautiful interface whatever, and you don’t know what’s behind it. And I think that’s a very important issue, that if you plan for building an application, you not only should talk about architecture, you should also talk about technical quality rules, that will find how this code should look like, what are the quality criteria, what are the hurdles the code has to jump over to be considered as done. I think it’s very important to add this dimension to the whole planning thing. I talked in the beginning about something like between 20 and 30% of sprint times for code hygiene, that is not only architecture that includes all this questions about technical quality, the measurement of technical quality which should be totally automated, but if this automated process produces findings, somebody has to get all those findings and fix the findings to make sure that we stay on track. It’s what you want to achieve.
Harry: It’s a great answer. Thanks a lot, Alexander and I hope you have a great QCon!
Thank you!