Copying and pasting code from one Windows folder to another as a deployment method can cause downtime. Jemma Hussein Allen presented how they automated their deployments and the benefits that they got from it at QCon London.
The main challenges with manual deployments of application and infrastructure code were due to environmental inconsistencies that happened due to a range of factors, of which the main one was aligning code deployment with deployments of dependent resources such as databases, Hussein Allen said. Before options such as blue/green deployment became available, changes that were not backward compatible meant brief periods where the database schema and code were out of sync and caused failed requests during deployments, she added.
Hussein Allen mentioned that general human error also plays its part, where certain file updates are missed, or incorrect versions are copied to the new environment. Human error can also be a big factor in testing if there are no mandated safety checks and local testing is the responsibility of individual developers:
Process automation helps to remove the human error component, and with intelligent deployment mechanisms, checks such as file share latency can be checked before deployments to ensure downtime isn’t an issue.
Manual deployment involving file shares can also be a big issue if the timing of file copies is out of sync, as it can result in incomplete deployments, Hussein Allen said. Updates to dependent resources can also be codified and synchronised to ensure changes are deployed in the correct order, she suggested:
These days, with deployment automation and blue/green deployments, new versions can be deployed and tested in a production environment before being made available to end users.
The first stage for automation was to document each deployment step, no matter how small, Hussein Allen said. Some key considerations were whether the step was manual or already automated, any downstream or upstream dependencies on other resources, and any access configuration, permissions, or secrets needed to carry out the step. This gave them a good overview of the current deployment method and highlighted any steps that would take a large amount of work to automate, she mentioned.
After they had a good understanding of the process, they looked into existing available tools to assess their suitability. They also looked at the existing deployment tools in use in the organisation to determine if these would be suitable, as people who were already experienced with the tool are really valuable:
Unfortunately, none of the existing tools were suitable for this type of workload, so we carried out a wider search to assess the suitability of other tools, taking into account support for the workload type and how they integrated with the existing company tooling.
Once the assessment stage was complete, they started to implement a proof of concept in the development environment.
We started with a small part of the application, and once we had successfully set things up in development for a specific application segment, we moved that deployment process to the staging environment.
As staging had slightly more policy constraints than development, testing in this environment was important, Hussein Allen mentioned.
After the proof of concept had been implemented, they then assessed how the solution could be rolled out to other application segments and environments with minimal disruption.
The main benefit from automating deployments is a quicker and cleaner release process, especially when working collaboratively in a team, Hussein Allen said. Centralised change logs and deployment trackers also provide much better deployment visibility across teams, she added.
Hussein Allen mentioned that standardisation of deployment steps can improve familiarity and understanding of a paved road and the components needed to test and deploy quality applications. The CNCF defines a "golden path" as the "Templated compositions of well-integrated code and capabilities for rapid project development". The paved road / paved path / golden path approach has become more popular with larger organisations as it is a proven method to speed up developer adoption of standardised company processes and tooling, as Hussein Allen explained:
I have seen the adoption of well-crafted templates by development teams majorly increase the speed of utilising company-specific tooling and logic in applications.
Automation will almost always save time and effort in the long run, even if it takes slightly more effort initially, Hussein Allen concluded.
