Aging platforms that are managed with manual, time consuming processes can be costly. Teams can make a business case to their management based on hours lost by repetitive work or re-work caused by human error for improving their processes by introducing modern techniques like automation tools and containers, argued Michael Jenkins. The result is a predictable and repeatable process for deployment, with minimal human interaction which provides the ability to deploy more often and more confidently.
Michael Jenkins, senior systems engineer at the Walt Disney Company, spoke about injecting modern concepts into legacy processes at Craft 2017. InfoQ is covering the conference with Q&As, summaries and articles.
To change a legacy process you might first need to build a business case. The post Injecting Modern Concepts into Legacy Processes by Michael Jenkins provides ideas on how to do this:
One way to get past the allure of new tools — and find the right one—is to think about the pain points. What’s slowing you down as a developer or system administrator? What’s breaking over and over again to the point that fixing it becomes a painful routine? Will this new tool or technique solve those problems? If so, how?
In addition to focusing on the rough patches, gather metrics to show how things can improve. For example, if a manual deployment is taking hours but it can be shortened to a few minutes with an automation tool, that makes for a compelling case. Metrics also arm you with facts to back up statements and answer questions.
In the end, you’ll want to make a business case for why things need to change and how productivity will improve.
InfoQ interviewed Michael Jenkins about how to include modern techniques into legacy processes.
InfoQ: What are the main problems of legacy processes?
Michael Jenkins: There are two main problems with legacy processes: technical debt and manual work. All too often, our legacy processes are tied to a specific version of software or operating system and even specific hardware in some cases. This technical debt makes it difficult to improve the process. Also, it's largely the case that our legacy processes involve multiple steps that require manual interaction. In this case, human error presents itself and may cause problems that could be avoided with automation. As a sidebar to the manual processes, when we free developers and administrators from having to manage manual tasks, we give them more time to be innovative.
InfoQ: How can you build a business case for changing the process?
Jenkins: The best way to build a business case is to focus on the parts of a process that cost the most in terms of lost time or lost revenue. This can be difficult for some developers and admins because they may not know the overall value of the application or system they manage when it comes to the bottom line of their company. One thing that they can be aware of though, is any time lost to repetitive work or re-work caused by human error. If they can make a statement to their management in terms of hours lost to a problem that reoccurs, that’s a basis for a business case. Management should be able to interpret the hours lost into a cost factor for the company. On the other hand, if the developers or admins are familiar with any sort of revenue data in relation to the application or product they work with, they should be able to state how much it costs the company for that application to be offline.
InfoQ: How can you socialize change to get people involved?
Jenkins: It might be difficult to get people involved if the initiator feels like a whistle blower. But it doesn’t have to be that way! I would suggest to start by using any communication channels that are already in place like team meetings or one-on-ones with leaders and management. Devs and ops can share their experiences with the problems that need to be addressed; indeed, these problems will be the root of business case. For example, they can describe the number of hours they have spent solving a problem that occurs repeatedly and show how this time was taken away from other work. If the person proposing the change is willing, perhaps they can also share the business case. If the work is taking place in a team setting, it would also be useful to get input from teammates or coworkers. Perhaps the legacy process affects multiple areas of the business beyond the observer’s day-to-day work. If that’s the case, there may be other people or teams that could support the initiator and show value across multiple areas. Whatever the case, the best approach is to share experiences and find allies.
In his blog post Jenkins gave an example how operations and development can collaborate and use application performance management data (APM) to investigate where improvement is needed:
Adding APM could bring more insight into the way an application runs and the way the server running the application performs. If a problem happens and causes downtime, both devs and ops can use the APM data to figure out what went wrong and then take steps to prevent the same thing from happening again.
With the prospect of identifying and resolving problems, developers and operators can share a common, collaborative interest.
InfoQ: You talked about a company that went from manual to automated deployment. What were the biggest hurdles and how did they deal with them?
Jenkins: The biggest hurdle was not breaking the current build process. Some people like to refer to it as "changing the tires on a moving car" or something similar to that. Indeed we needed to figure out what parts of the process could be automated and how they could be gradually worked into the bigger picture without breaking anything. We dealt with this by starting small and finding the parts of the process that could stand to be changed while other parts remained the same. Over time, we were able to move to yet another part and then another until the entire process had been reworked. For example, our deployment process was the same for three different applications that ran in three different production environments. Due to misread server names or other error, it wasn’t uncommon for an admin to deploy code to the wrong servers. To fix this we used the repository URL as an input to the deployment and automated the selection of the server selection. This fixed the problem. After the correct servers were selected, the next part of the deployment process could be automated. And so on and so on until the process was automated as much as possible.
InfoQ: What benefits did they get from changing their legacy process?
Jenkins: The biggest benefit was a predictable and repeatable process for deployment. As previously described, the only input needed to the deployment was the URL to the code repository. Since every following step in the deployment process was automated, human error was essentially eliminated. With human interaction kept to a minimum, points where human error could derail the deployment were removed. The follow on benefits were less time spent fixing or rolling back bad deployments and the ability to deploy more often, more confidently. As mentioned earlier, this freed up admins for more time to innovate and contribute in other areas.