Getting Agile with MS VSTS for Database Professionals

Last month, following up on an initial beta release, Microsoft made their first full release of Visual Studio for Database Professionals, one of the first toolsets offered to help teams extend Agile practices to the DBA community.  We should expect to see comparable offerings from other major players in 2007. But database professionals have often been left to the side when teams adopt Agile practices... how can we help them get up to speed on Agile practices?  We tracked down some further information to help Agile teams and their DBAs get started with VSTS.

Many features in this release use concepts developed in Scott Ambler's book Agile Database Techniques.  Ambler, who traded ideas with MS colleagues while working on his Refactoring Databases book, thinks it's about time:

I'm excited to see tool vendors, and the open source community for that matter, are beginning to provide tools such as this to enable evolutionary database development. I believe that we are at the cusp of a new age when it comes to database development, we are finally starting to throw off the chains of traditional development and beginning to take pride in our work.

How did Microsoft decide to take VSTS in this direction? We asked Sachin Rekhi, the Microsoft product manager responsible for VSTS:

When building these tools, we realized that while application development teams have successfully embraced agile development, the database development teams have really been lagging behind.  In many ways, this has been due to the fact that the existing tools did not facilitate doing database development in an agile, iterative fashion. 

When we were deciding which role to target next, we looked at Team System’s strengths and looked for where the greatest opportunity was. ... We all know that the application and database development teams often tend to be the least integrated and sometimes have the worst channel of communication. We saw a huge opportunity to help bridge the gap between application and database development teams.

In addition, we realized that the hardest part of database development today really is managing database changes. Thus Team Edition for Database Professionals is primarily targeted at supporting iterative database development in a risk mitigating way.

Database professionals have been leary of Agile practices... and perhaps with reason.  Practices and tooling for database testing and refactoring were poor or non-existent in the early years of Agile methodologies. Consider the potentially disastrous and far-reaching implications of a database change.  If one were to simply rename a column without adequate tool support, it could have tremendous unknown effects on stored procedures, triggers, and views that reference it.  Rekhi says:

I believe we have attacked the core of the problem by enabling confidence in database changes. Whether it’s through the use of the automated rename refactoring tool for making changes, or database unit tests to ensure no regressions in existing functionality have resulted, or even to the version control history of all revisions to schema objects, we have now removed that fear.

Once people really see the power of these tools to increase their confidence in database changes, I think database professionals will be sold on the approach. [Now] I expect even greater traction in bringing the application and data tiers together in their quest for agility.

How can a team get started with Agile database practices? The Agile Data site is a good online source of further information, particularly for database testing, and the agiledatabases yahoo group is a good place for novices to ask questions of people already applying the Agile Data practices.

For those interested in even greater detail on refactoring databases successfully, Refactoring Databases: Evolutionary Database Design is a good read.  Microsoft's Rekhi endorsed Ambler's techniques in the foreword to the book:

Refactoring Databases is an invaluable resource that shows data professionals just how they can leap ahead and confidently, safely embrace change. ... [it] outlines a set of prescriptive practices that show the professional DBA exactly how to bring this agile method into the design and development of databases.  Scott’s and Pramod’s attention to the minute details of what it takes to actually implement every database refactoring technique proves that it can be done and paves the way for its widespread adoption.  Thus, I propose a call to action for all data professionals.  Read on, embrace change, and spread the word. Database refactoring is key to improving the data community’s agility.

And for those who see a need to customize it before they can fit it into their process, good news: Microsoft rarely builds a product these days without providing extensibility points for others to build upon. Team System is no exception. See Rekhi's blog for a post about the extensibility points in Team Edition for Database Professionals.