The Long Journey Towards Database Lifecycle Management

Talk about software development processes and the name Francois Castaing isn’t often mentioned. Nor is American Motors Corporation (AMC), which is surprising because if it hadn’t been for Francois Castaing’s game-changing thinking at AMC, we wouldn’t work the way we do today.

In 1985, Francois Castaing was VP for Product Engineering and Development at AMC. He was given the unenviable task of introducing a new development process for the upcoming Jeep Grand Cherokee, which would cut costs and allow AMC to compete against its much bigger rivals.

Rather than closing his eyes and hoping the problem would go away, Castaing took a different tack. Turning his back on standard practices like organizing work around departments and components, he introduced Computer Aided Design with every design and document held in a single, central database. Then he established a communication system that resolved conflicts faster.

Now all of this doesn’t sound like much until we realize what he achieved.

Firstly, the Jeep Grand Cherokee launched with a development cost far lower than its rivals. So low that Chrysler stepped in and bought the company to get its hands on the product development process.

Secondly, even though he didn’t know it at the time, Castaing invented Product Lifecycle Management, a practice that would sweep through every industry because it increases efficiencies, reduces costs, and improves performance.

Moving software development up a gear

You can probably see where this is going. Based on Francois Castaing’s work, software companies adopted Application Lifecycle Management (ALM) to carefully manage the lifecycle of software applications from the point of inception through to delivery, monitoring and maintenance.

From the very beginning of the lifecycle, ALM covers the process of generating, refining and project managing ideas and requirements for software applications.

Implemented well, it overcomes the challenges of multiple teams, disparate processes, metrics and infrastructures. In doing so, it facilitates an agile, repeatable lifecycle that is highly compatible with a continuous improvement approach.

But here’s the thing. While lifecycle management was integrated with applications, one element was left behind. The database.

Who forgot the database?

Everyone did. There was seen to be no choice because data isn’t code, it’s a living, changing, evolving thing. If code breaks, it can be fixed, like an automobile. If data breaks, there’s a problem. A car crash, if you will.

So while the journey from Product Lifecycle Management to Application Lifecycle Management was seen to be relatively easy, the leap to Database Lifecycle Management (DLM) wasn’t.

If an application has a back-end database with business-critical data which must be be safely and correctly preserved, it brings additional, database-specific challenges to the lifecycle management process. Challenges once regarded as too twisted and mountainous to navigate.

The road ahead

It is out of the complexities databases and their data bring to the application lifecycle that DLM has finally evolved. Bringing additional processes, checkpoints and controls to ALM provides a rigorous approach to managing the schema, data and metadata for the database supporting an application.

DLM takes a wide-angled view across the software lifecycle, encouraging teams to use a range of processes and technology infrastructure to manage database changes. These processes coexist with and should be managed within the processes and policies used in ALM.

The route to a DLM process is also made a lot clearer if it’s understood from the outset that it’s the same route as ALM, with a few more stops along the way. What is important for DLM, however, is the data management strategy that underlies the process. Factors like data migration, monitoring, and recovery all come into play during the development and operation phases of the lifecycle and need to be considered carefully.

Other than that, the steps are the same – and need to be the same because it’s best practice to coordinate application and database development processes at specific stages of the lifecycle wherever possible.

Following the same route as ALM

In ALM, it’s common to automate the repetitive development and testing processes teams use to deliver, manage and maintain the application. From version controlling changes to deploying the changes to different environments, and, when ready, choosing to deploy to production, ALMhelps teams reduce risk and increase both efficiency and reliability in the software release process. With similar processes, DLM can deliver the same advantages to the database environment.

(Click on the image to enlarge it)

The version control path

Version controlling database changes, for example, is a natural step for database development teams to communicate their changes with others in the team. It is does not exist to enforce rules: rather, it is to promote a transparent development approach that provides a version to roll back to if required and maintains a solid audit trail. Versioning the database objects together with the application code also provides teams with a single source of truth on which they can base development and deployment processes.

The Continuous Integration path

As soon as database changes have been checked into version control, a continuous integration process can be triggered that automatically builds and tests those changes in a CI environment. Teams can configure their CI build server to automatically pick up committed changes and run any series of tests that have been previously committed to version control via a series of build steps.

The Release Management path

Release management has been made a lot easier for DBAs with a range of release management tools that deploy database changes to various environments alongside application changes. Some tools even allow DBAs to review changes that are about to be made and check that there has been no drift in the pre-production and production environments before carrying out the deployment.

The monitoring path

After database changes have been deployed to production, it’s important to continue monitoring closely for any unexpected results or impact on the database or application. Once again, tools are out there to make monitoring so efficient that issues are highlighted before they cause a problem.

The recovery path

An additional step to help safeguard against any unforeseen problems is to take a backup prior to deployment and have practices and patterns in place for a rollback and recovery strategy. That way, should there have been a fault in the development process, the database can be restored to its original state.

The final destination

The journey from Product Lifecycle Management to Application Lifestyle Management and now to Database Lifecycle Management has been a long one.

It’s been worthwhile however, because by providing a common series of processes to manage the lifecycle of databases alongside applications, development teams now have a mutual starting point.

For too long, application development teams have been on one road, database development teams on another. Both have been trying to get to the same place, but not communicating on the way. DLM changes that reality to one where both teams are on the same road, going to the same destination, stopping at the same waypoints, and sometimes facing the same obstacles to work around.

For example, DLM promotes greater collaboration on application features and their database requirements by default. That’s important because shared goals encourage open feedback earlier in the process where it can make the most difference. With a continuing process underway, issues on either the application or database side of the equation will be highlighted sooner.

This cultural shift will not only improve quality over the course of software lifecycles. It will reduce risk at the point of release, make faster releases possible so that new features can be delivered sooner, and deliver a better service to customers.

All of which is a destination everyone can agree on.

About the Author

Ben Rees manages the database lifecycle management team at Red Gate Software, taking his experience both in product and marketing management with five years as a developer in the trenches (in SQL Server, Oracle and DB2) to make sure that in the growth of continuous delivery, the database isn’t forgotten.