Menno Vis, IT director of bol.com talked about aligning over multiple Scrum teams and with projects and programs at the Bits&Chips Software Engineering conference.
InfoQ did an interview with Menno Vis about the benefits of increasing agility, how bol.com deploys Scrum, using roadmaps with agile, the challenges that have been faced when scaling agile, the main focus area’s at bol.com for agile scaling, establishing loosely coupled teams, and the things that bol.com does for their people to have fun while doing their work.
InfoQ: What makes increasing the agility so important for bol.com? Which benefits will it bring to the organization?
Vis: The ecommerce market is extremely dynamic and changes continuously. As a leading ecommerce organization we are always innovating because we want to offer our customers the best shop there is. For us innovation means experimenting by taking (small) incremental steps followed by measuring and learning from the results. This is what agility means for bol.com and it can only be achieved by making it possible for (multidisciplinary) teams to work autonomously and independently.
We have clear long term objectives but the road to reach these objectives is not always that clear. We need to be able to take alternative and/or new roads to reach our objectives which is what agile is all about. The benefit is then of course that we still will be able to reach these objectives.
InfoQ: Can you describe how the bol.com "way-of-Scrum-working" looks?
Vis: This question is not so easy to answer in a few lines but I’ll try. We have approx. 35 Scrum teams working on small and large innovations within a specific domain. Each team is responsible for several services which are related to each other and which support one (or a part) of a business process. We aim for teams to be as autonomic as possible so they can (and will) take responsibility of their domain. At bol.com a Scrum team is a DevOps team and also responsible for daily operations of "their" services in our production environment. This became possible when we insourced our datacenter and invested a lot in automating our infrastructure (see also bol.com’s DevOps Journey). Our teams are responsible for building the software, releasing this to production, running it and loving it. Every Scrum team has the ownership for making our customers happy for a specific business theme, which increases the motivation of the team .Hence our motto is: you build it, you run it, you love it!
InfoQ: How did you come to this way of working, and what did you learn along the way?
Vis: We started with agile/Scrum 7 years ago with less than 10 teams. Over the years we have changed many things but the basics are still the same: a team is responsible for their domain. One of the major changes we implemented several years ago is our Scrum innovation planning process: an agile process which is used to determine the roadmap for each team. This roadmap has a 12-Months horizon and is adjusted every 4 months in a planning session. The initiatives on the roadmap are those which have highest value compared to the (time) investment of a team (which is estimated by the team itself). It is a bit like sprint planning but on a higher level; this has helped us to give a clear direction to our teams but also has agility build in without changing our direction continuously.
InfoQ: My first thought is that a 12 months roadmap would reduce agility, but I suppose that’s not true. Can you elaborate how this roadmap support bol.com in staying agile?
Vis: The roadmap gives a team guidance and provides everyone insight in which initiatives are planned to be picked up by the team. This also helps stakeholders and other non-Scrum members with preparations which might be required (e.g. collecting data or contracts that have to be signed). However we do stay agile since the roadmap is revisited at least every 4 months. And if needed the roadmap can be adjusted whenever there is a good reason to do so; we will then not wait for the next planning session.
InfoQ: Can you elaborate on the challenges that have been faced when scaling agile at bol.com?
Vis: The most important challenge was when we passed the (for us) magic number of having 25 teams. We used to work in parallel and all software was deployed after every sprint in one major release. So every 4 weeks we had a major software release. This meant we had to ensure all 25 teams were ready on time without any major defects in their software to make it possible to move forward and release the software build by all these teams. If only one team had issues all other teams had to wait (and hence suffer). Given the fact we had many more teams than at the start the possibility of this happening had also increased significantly. We were convinced that we had to move to another model: teams should be able to build and release every change they have made independently. So we moved to a model of continuous delivery of software. For a team it means that a user story is only done when it’s in production (and is live!). Of course our software architecture had to be adopted to be ready for continuous delivery and tooling was developed and customized to support this. But more important we had to change the mindset of our teams as this is a major change in ways-of-working. At this moment we have implemented DevOps for all 35 teams, and though we are not at the end of this journey, we can see the scalability of our organization and systems has improved enormously.
Related to this we also saw that we had to automate much more than we had done in the past. A good example is testing; this used to be a manual process but that doesn’t scale very well and also will take too much time if a team has to release software continuously. We switched to a high level of automated testing which also lead to changes in the way how our teams are structured. We currently see that we need more software engineers compared to test analysts since testing has moved into development of "test-"software.
InfoQ: What are the main focus area’s at bol.com for agile scaling?
Vis: One of our key challenges is to remove horizontal and vertical dependencies by investing in tooling and technology. This is needed so our teams can work independently of each other. A good example is Mayfly, a concept we introduced last year which enables teams to develop stories in an isolated environment. The environment is created dynamically and only "lives" until the new feature is moved to production. The architecture is using tools like Jenkins, Docker, Mesos and Stash and is – for us - the tool which we use to "dockerize user stories".
Another one of our challenges is to ensure that bol.com knowledge of tools, systems and processes is available in all teams. We do this by ensuring that in every team we still have experienced bol.com engineers; which is of course not easy if the number of teams is growing fast (and many new people have started at bol.com). Scaling up does lead to many team moves and every change impacts a team; a team will have to find a new "balance". Of course we have documentation and an extensive introduction program for new bol.com employees which helps a lot. But real bol.com knowledge will only come with experience.
There are other challenges as well: finding and recruiting the best people to grow the company is one of them for instance. This costs a lot of effort but we have been very successful the last years and our IT organization already employs over 250 persons. The fact that we have many years of agile experience combined with a new and state-of-the art technical platform does attract great candidates. If interested: we are hiring!
InfoQ: Can you describe the approach that bol.com has taken to establish loosely coupled teams?
Vis: We have put a lot of emphasis on making teams to work independent of each other by focusing on people, process and technology. People because the team members have to make it happen; they should have the will to be successful and the eagerness to deliver value on a daily basis. And they have fun too while doing this! We believe in "freedom within a framework", so a high level process framework is needed within a team (and Scrum provides that of course) but we wanted some similarities in ways-of-working across teams since this would make it much easier for people to move to another team if needed or to provide some generic tooling which can be used by all teams. And technology like Mayfly since both a state-of-the-art service oriented architecture is needed and a platform which does support this.
InfoQ: You mentioned that it’s important for people to have fun while doing their work. Are there specific thing that you have done to make that possible?
Vis: We are doing many things for our people and our people are also many things themselves. We have regular internal hackathons organized by our people themselves, the most recent one took place in April 2015. We host meetups at our office on all kind of IT topics (last months about Docker, Elastic Search and GoLang). Teams organize team events for their own teams; there are regular drinks and parties. But most important is the fact that the teams work together on features (small or large) that benefit our customers and they have real influence on what and how these features are build! And that is probably more important then all parties, bbq’s and drinks.