Tony Davis

Simple-Talk Editor
News, views and good brews

The Fall and Rise of Log Shipping

Published Thursday, June 25, 2009 1:10 PM

When Database Mirroring was introduced in SQL Server 2005, it seemed reasonable to assume that log shipping would gradually go out of fashion. Mirroring is a way of introducing high-availability to SQL Server by allowing the secondary server to become the main server instantly, in a way that is invisible to the applications using the database. This, in theory, is the dream solution for the .NET application developer: the application needs to do no work at all to take advantage of this resilience against hardware failure.

Log Shipping has been around for a lot longer, first as a sort of "home-brew" solution that the intrepid DBA could make work with version 6.5 and 7.0, before an official version was built into the enterprise edition of SQL Server 2000. It is a method, based on SQL Server Agent jobs, by which the transaction log backups from a primary server are applied to secondary servers. In this way, one can keep one or more spare 'warm standby' servers in a state of readiness to take over in the event of failure of the primary. It works well, and requires just a few mouse-clicks to set up, but it requires some nimble footwork from the DBA to switch the standby into becoming the primary server, and generally entails 15-30 minutes of downtime. In addition, application client redirection requires some manual changes.

Curiously, despite its many stated advantages and heavy support from Microsoft, Mirroring hasn't replaced Log shipping, as predicted. Aside from the obvious fact that many shops still have SQL 2000 machines and so have no choice but to use log shipping, there are a number of other subtle and not-so-subtle reasons for its continued prevalence.

Many HA setups require more than one standby server, which mirroring cannot provide. Other DBAs require that delay time that is intrinsic in Log Shipping to protect against logical errors that, with mirroring, would have instantly been carried across to the standby. In a sense, Log Shipping also provides "free standbys", since it makes use of backups would need to be made anyway and, to the delight of many DBAs, checks the integrity of the backup for free.

This final point hints at perhaps the biggest reason why log shipping still prevails. Many DBAs regard the High Availability aspect of log shipping as a "happy by-product" of their need to offload reporting functions to a secondary server. In a mirrored setup, the target database is never online, or accessible to the end user, until failure occurs on the source. The only way to get around this is to set up database snapshots on the target, which of course incurs the expense of Enterprise Edition. With log shipping, all you need is Backups, and a few cheap standby servers running SQL Server Express, and you have an effective reporting solution, with "poor man's High Availability" thrown in!

However, although rumours of log shipping's demise have been exaggerated, it may still be only a stay of execution. Microsoft may finally relent on the Enterprise requirement for snapshots. With the advent of server virtualization technologies, it will surely become easier to maintain 'standby' virtual servers which can be deployed even more quickly in the event of either physical or logical failure.

DBAs have stuck with log shipping for good reasons that were un-guessed at by the pundits, but does it have a long-term future as a high-availability strategy?

As always, we'd love to hear your opinion. Add your comments below (you'll need to be signed in), and the best contribution will receive a $50 Amazon voucher.

Cheers,

Tony.

Comments

 

mksql said:

> With log shipping, all you need is Backups, and a few cheap standby servers running SQL Server Express, and you have an effective reporting solution...

In this scenario, there are still latency / availability issues to address with the reporting data correct? It has been a while since I last dealt with this, but IIRC log shipping targets must be in STANDBY mode, and either must disconnect clients when logs are restored, or the restores remained queued until there is a window with no client connections.

Do you have any suggested documentation on utilizing Log Shipping specifically for reporting purposes?
June 25, 2009 10:43 AM
 

Daily Links for Friday, June 26th, 2009 said:

June 26, 2009 6:34 AM
 

SnipStorm said:

Thank you for submitting this cool story - Trackback from SnipStorm
June 26, 2009 8:16 PM
 

nkav said:

High Performance mirroring is also only available in Enterprise.  For me this is the only version of mirroring that's an acceptable alternative to log shipping.  Most DBA's aren't prepared to take the performance hit that's required with high safety/ high availability mirroring that comes with SQL Server Standard.

Log shipping is also one of those things that many DBA's would do with home-grown scripts, even with 2000 only enterprise supported log shipping via the GUI.  Finally in 2005 log shipping was officially supported in Standard.

IMHO most people don't want reporting to double as stand-by because it requires an additional license, where as having a stand-by doesn't.  Your legally allowed to run off the stand-by without an additional license hit for 30 days, which is more then enough time to move things back to your primary.

The real nail in the coffin for log shipping will be high-performance mirroring in Standard.   The more prolific SQL Server becomes, the smaller the businesses are that use it, many of them simply can't justify the price tag of enterprise.


June 28, 2009 7:11 AM
 

Phil Factor said:

Good points, nkav,
On your last point, I was wondering whether you thought there would still be a requirement for log shipping, even with cheap Standard mirroring, where one or more stand-bys are required as well as the single mirroring stand-in.
June 28, 2009 5:38 PM
 

nkav said:

I'm probably not a good person to ask as I've never had to maintain more then a 1 to 1 mapping for a primary to a secondary db server for HA/DR.  In what situations would you need such a setup?   High performance mirroring doesn't support such a setup, but I'm sure if customers demand it, MS would make it so... eventually.

Moving forward I think we'll all be waiting for what MatrixDB has to offer for multi-master/ distributed databases.
June 28, 2009 8:50 PM
 

hi_abhay78 said:

Even though log shipping and mirroring serve the same purpose , I think we can still scaleout more in log-shipping than in Mirroring besides being the fact that snapshots are allowed in ENT edition .

I will give you an example .

Unfortunately , we had a clustered instance of 2000 with 800+ databases (foolish right , but thats was it was since MS claime 35000+ databases per instance).

i was given the task of creating the DR site for all those databases .

So ,

-> Mirroring was not possible (espacially on 32 bit where you have limited VAS) due to its high resource consumption.
-> Replication was also not possible for the same reason .
-> The last option was Log shipping .But it had its complicatiosn since all the 800 x 3 jobs will kick off at the same time (backup , copy and restore) .Finally , i ended up automating the whole setup in such a way that all the 2400 jobs will be executing serially now .

So the summary is that LogShiping is still surviving and should continue to survive for quite some more time until Microsoft makes some good code changes in Mirroring for more scalablity and make it more light weight + add some features from ENT to STD (as rightly pointed by others ).

Or Microsoft should make some good changes to Log shipping like automatic failover and an option to serially execute the jobs so that it does not bring down the servers where we have high number of databases .

But , eventually Logshipping will phase out .

Regards
Abhay
June 28, 2009 10:22 PM
 

Stan said:

What about using a combination of two; I do it for some customers where we host the databases as mirrored databases (sync with witness) in one site. On top we use LogShipping from either of these databases (which ever is running primary) to a secondary site... to do so Standard Edition is sufficient (that is, if transaction latency from synchronous mirroring isn't causing you problems).
June 29, 2009 1:31 AM
 

ML-DBA said:

I did convert some implementations of log shipping to mirroring for some databases I am supporting.

One of the biggest advantage of log shipping over mirroring is that we have a better control of the primary and secondary database synchronisation. As example, for business reasons, you may need to have both databases synchronized at specific time during the day, mirroring does not support this particularity.

I do support this kind of applications since more than a year now without problem.
June 29, 2009 7:15 AM
 

PCNuttallDBDev said:

Actually, Log Shipping has been around longer than Microsoft SQL Server.
I worked with a database developer back in 1992 who developed a UNIX script based Log Shipping using Sybase.  
July 1, 2009 4:47 PM
 

Saggi Neumann said:

As mentioned above this comment, log shipping has its advantages over mirroring (multiple secondaries, ability to read from them without expensive licensing, restore delay, doesn't affect the primary as much as mirroring does in the std. synchronous mirroring, isn't affected by network latency, doesn't require full recovery model and can work well with the bulk logged recovery model so index rebuilds aren't as messy as they may become with mirroring) but it's also got its disadvantages (requires more disk space for log backup copies, no automatic failover/failback, data loss may be incurred) but these can be overcome with some 3rd party tools and backup compression (either native or 3rd party) and very frequent log backups.

I like to think of log shipping as a DR solution rather than a high availability solution which can also help with scale outs (and also to have a copy to start ETLs from instead of reading off of the production db).

All in all, both are great options, and their pros and cons should be considered anytime you're looking for a DR/HA/scaling solution. Log shipping is not going to disappear - it's so simple to maintain there's no reason to make it go away and in case MS does that, there are tons of 3rd party options ;-)

Cheers,
S. Neumann
July 11, 2009 11:45 AM
You need to sign in to comment on this blog


















<June 2009>
SuMoTuWeThFrSa
31123456
78910111213
14151617181920
21222324252627
2829301234
567891011
Finding Stuff in SQL Server Database DDL
 You'd have thought that nothing would be easier than using SQL Server Management Studio (SSMS) for... Read more...

Mission Critical: SQL Server 2008 Performance Tuning Task List
 In which Buck Woody imagines how the US military would have tackled DBA checklists for... Read more...

Simple Query tuning with STATISTICS IO and Execution plans
 A great deal can be gleaned from the use of the STATISTICS IO and the execution plan, when you are... Read more...

Switching rows and columns in SQL
 When they use SQL Server, one the commoner questions that Ms Access programmers ask is 'Where's the... Read more...

Writing Efficient SQL: Set-Based Speed Phreakery
 Phil Factor's SQL Speed Phreak challenge is an event where coders battle to produce the fastest code to... Read more...