Today at Day 1 of PASS Summit 2017, Redgate Evangelist Steve Jones presented his tips for taking a DevOps approach to database management. His approach includes tips that can benefit installations of all sizes. Jones began his presentation by sharing that in 2016 Amazon deployed production-level changes every 12 seconds. One of reasons he shared this was to illustrate the rate of change that can be effectively delivered with an effective approach to change management.
Given the amount of data being stored in databases and the crucial role of databases, performing successful changes to those databases in production is key. To help illustrate the amount of data being stored in databases, Jones noted that Microsoft collects 700 terabytes of telemetry data from Azure user’s daily. (Yes, terabytes.)
Jones looks to Gene Kim who has provided the three principles underlying DevOps:
- Systems Thinking
- Amplify Feedback Loops
- Culture of Experimentation and Learning
From there Jones discussed his techniques to bringing DevOps to Databases. The first, and foundational layer, is to implement and rely on the use of a version control system (Git, SVN, etc.). Next it is important to treat database code no differently from application code. Third, setup and maintain an organized system in the VCS—in some cases this could be the same repository used for storing application code but in others it may make sense to use a separate repository. Finally, use code reviews for changes and take them seriously.
As part of testing changes in a non-dev environment prior to deployment, Jones recommends the use of a curated data set, instead of using a subset of production data. Start with empty database (DEV / UAT / STG / etc.) then add 10-40 rows that represent the use-cases of the production environment.
All code deployed in production should be done so in a consistent manner. If manual steps are needed, they should be documented thoroughly. Consistency is important, so that every deployment is always made the same way without forgetting important considerations like unit testing / proper tables with primary keys defined etc.
Jones observes that rolling back unsuccessful changes is hard, especially when the changes involve databases. Jones recommends writing rollback code that has been tested prior to deployment on (DEV/STG/etc.) so a working script is ready to go in the event it is needed. Another technique to consider is the use of dark deploys, where the changes are deployed to production but their usage is triggered through a feature flag. Until the feature is deemed ready for general availability, it is not active. According to Jones, Facebook took this approach with their rollout of Facebook Messenger to ensure that their systems could handle the anticipated user load.
