At the GOTO Amsterdam 2015 conference Ivan Kusalic, Software/DevOps engineer at Nokia HERE in Berlin, gave a lightning talk about about putting more Dev in DevOps.
InfoQ interviewed Kusalic about why they decided to apply DevOps, how DevOps has changed their way of working, which benefits they are getting, and the challenges that they had and how they dealt with them when development and operations became one team. Kusalic also gives some advice for applying DevOps with teams.
InfoQ: You talked about having two teams, one for development and one for operations. Can you describe the problems that you had?
Kusalic: The problems were two-fold. The teams had separate Line Managers and so even the goals would not always align. More importantly, people communicate much more inside of a team than between teams. So the problem was primarily lack of cross-team communication. Members of different teams would know each other less, would not hang around together as much, there was no common retrospectives or design discussion etc.
This made both teams preform suboptimal as they didn’t share insights nor improved the current tooling and processes. We would also critically depend on each other, but if something in the realm of responsibility of other side went wrong, you’d need to wait for it to be fixed without much say in it.
Basically all well-know benefits of DevOps would be present only to a lesser degree because of the separation of the teams.
InfoQ: What made you decide to apply DevOps? What was the outcome that you were aiming at?
Kusalic: When I joined the delivery team, the Continuous Delivery was already in place. But by becoming one team developers would understand the delivery side better and would think about it while writing application code. And on the delivery side, we’d understand better how the application worked and the implications for performance, scalability and environment that are necessary to run the application without incidents.
InfoQ: Can you share your view on DevOps?
Kusalic: I strongly believe that DevOps is about working, learning and improving together. While the tooling side is important, it will naturally arise from a team that cares and tries to continuously improve. So I guess culture over the tooling is where I lean towards.
InfoQ: Can you give some examples of things that have changed with the introduction of DevOps?
Kusalic: Everyone from the delivery side has written application code and so they understand the application much better. The same is true in opposite direction - every developer understands the delivery pipelines, scaling polices etc. much better and actively improves them. In our particular case we do not even have the separation of delivery vs. development. Everybody works on both, whatever is needed next. Of course some individuals know more from one side or the other. But by communicating, planning and working together, that’s not a problem. Now we are one team that does both.
InfoQ: Where has applying DevOps helped you to solve the problems that you had? Which benefits did it bring?
Kusalic: Working together we have rewritten our delivery tooling that was brittle and hard to understand. Now it’s much easier to add new features and we introduce much less bugs while doing so.
We have improved our delivery pipelines due to a better understanding of the application - we’ve reduced the time from commit to receiving traffic in production by 30%.
Pipelines are also much more stable as everybody cares that they stay green all the time. So when developing we are much less impacted or left in the dark as to whether the code we are building on top of is stable.
InfoQ: Are there any challenges that you experienced when becoming one team? How did you deal with them?
Kusalic: We had the standard issues like any other group has when becoming one team - it takes time to build trust and to really become a team and not just a group of people put together in an org-chart.
We’ve drastically speed-up the process by working on a common project - delivery tooling. By having common planning/design discussions, pairing together, etc. the building of the trust and being comfortable communicating happened naturally faster as well.
There were some problems specific to the project we worked on as well.
We on the delivery side didn’t know the implementation language - Scala, and were less familiar with development practices the developers employed. One particularly interesting thing was containing the complexity of powerful features we used from Scala (e.g. macros). We needed to be disciplined to have the complexity isolated only in a small part of the codebase so that people who are less familiar with Scala or even with development could still contribute.
InfoQ: If people are interested in applying DevOps with their teams, can you give them some advice?
Kusalic: Try to make sure that everybody understands the benefits of using DevOps. E.g. that smaller release cycles help developers to be sure that they are not introducing performance degradations and that every increment behaves as expected; while on the delivery side the smaller increments drastically reduce the risk of deployment going wrong.
If at all possible, try to avoid silos and ensure that everybody has at least an overview of the whole system.
Try things out on smaller, less critical projects/systems. Support systems with the same tech-stack that do not need to have 99.9% availability are great for this.
Have regular retrospectives (formal or not) to keep improving.
But I guess it all boils down to: care what you do and take pride in it. The rest will follow.