Click here to monitor SSC

Divisional Manager for SQL Tools - Red Gate Software

A trip back to the sharp end

Published 4 August 2011 4:53 pm

On 1st August, James Moore, head of the team at Red Gate who brought to the world the likes of SQL Prompt, SQL Compare and SQL Source Control, and responsible for the future strategy for SQL developer tools at Red Gate, set off on a “walkabout”, leaving a sizeable team to run itself, for an indefinite period. Read this blog post to find out why James has made this decision.

Over recent years, my team, and Red Gate in general, has thrived by providing simple robust tools that solve common database development problems better than any other, and fit smoothly into a user’s existing workflow. I’ve personally talked to countless people who tell me how much time SQL Compare, for example, has saved them when migrating schema changes.

However, development practices are changing faster than ever; Agile techniques are well and truly embedded in mainstream development and evolving quickly. Cloud computing is starting to affect where we store, and how we distribute, our data. An important part of my job as manager of the team is to make sure our tools evolve in the right way; to make them as effective as possible in supporting these and many other new ways of working.

As a hot-headed developer, it could be easy to get swept up in the level of automation that’s now possible in integration and deployment of software builds, with relatively modestly-sized teams handling 10K automated builds a month, and Continuous Integration and Continuous Deployment moving from a team practice towards being an integral part of enterprise-wide software delivery. Why is the world of database development less enthusiastic in participating in these trends? Surely with the right tools, Continuous Deployment for database development can be a reality?

Certain complexities, such as the likelihood that databases are shared between applications, the need for data interchange between databases, and the requirements for maintaining the integrity of the data in all its complexity during deployments, all make the rapid refactoring of databases more difficult, and so we’re already working hard to make database refactoring, version control, and deployment easier and quicker.

However, I came to realize that, however much I’ve liaised with our existing users of our products, I don’t have enough personal experience of working in an enterprise database environment, to enable me to fully understand the exact requirements for enterprise database deployments, what the real barriers are to Continuous Deployment for databases, and whether it is even a desirable goal for many businesses.

Ultimately, though, I also believe that, despite the complications of persistence and data, the principles of refactoring, automated regression testing and so on, can be applied equally as well to database code as to application code, and to great benefit.

So this, in short, is why I am on the road. I want to visit as many organizations as I can, big and small, especially if you are pushing the barriers of database development. I need to watch, and learn how you work. I need to understand what parts of your process are automated, and what parts defy automation. If you’re already trying to continuously deploy databases, perhaps even to the cloud, I’d love to visit you. If the concept is anathema, I’d be just as pleased be able to visit you and find out why.

If you could spare even just an hour, I’d be grateful to hear from you. Please email me here or comment below and I’ll be in touch shortly.

5 Responses to “A trip back to the sharp end”

  1. Anonymous says:

    I run a mid-sized IT department in London. We’re currently doing CI for applications, but NOT on the database. If you do find any sensible people doing CI, or even CD (!) on their database, I would love to read how in a follow up blog.

    If you would like to come and visit us I have emailed you more information and direct contact details.

    Stephen

  2. BuggyFunBunny says:

    – Continuous Integration and Continuous Deployment moving from a team practice towards being an integral part of enterprise-wide software delivery. Why is the world of database development less enthusiastic in participating in these trends? Surely with the right tools, Continuous Deployment for database development can be a reality?

    If, and when, coders stop writing “SELECT * from …”, CI and CD will be possible.  In recent memory, RoR was the first framework to make DB “migration” part of the process.  It’s still a code driven, rather than data driven, facility.  

  3. paschott says:

    We’re trying to work this out now. We’re using VS2010 DB Projects, Git for source control, and setting up branches as appropriate. We find it tends to work okay if we keep our changes relatively small, use Pre/Post deploy scripts appropriately, and refactor / rename appropriately. If we start stacking or trying to pick and choose releases, it gets harder. I think we’re about to look into a “Released” branch that we’ll use to drive our environmental releases and merge into that as we deem things ready for release.

    We’re still working out the actual release automation part for the DBs. I can definitely automate the build/deploy piece on an as-needed basis, but haven’t worked that into the deploy process yet.

    The biggest challenge – we haven’t worked out how to effectively handle a commit happening mid-deploy. That’s rare, but it does happen sometimes and causes issues due to file mismatches and similar.

    We’ve pretty much limited a lot of the bad code, but I’m not really crazy about the Ruby method for roll-forward and roll-back. Roll-forward is cool. The DBA in me is very uncomfortable w/ the idea of dropping tables/data/columns after we’ve already moved forward. *shudder*

    If we figure out the DB side effectively, I’ll definitely put up a blog post about it. :-)

  4. BuggyFunBunny says:

    – We’ve pretty much limited a lot of the bad code, but I’m not really crazy about the Ruby method for roll-forward and roll-back. Roll-forward is cool. The DBA in me is very uncomfortable w/ the idea of dropping tables/data/columns after we’ve already moved forward. *shudder*

    Then you don’t know your place, young man! :) It’s the coders who decide what the schema/catalog should look like, not database experts, who are just wasted overhead. Says so in the RoR books, anyway.

    OTOH, the concept is just for data what Agile is for code: a Texas Two Step ad nauseum. Whether Agile or BDUF (see, Spolsky) is smarter is that age old conflict. Database types tend toward BDUF, while kiddie koders tend to Agile. The latter because it offers a Prime Excuse for never doing it right the first time. Thus morphed to the database.

  5. Anonymous says:

    AFAIC that’s the best asnwer so far!

Leave a Reply