The Great PST Migration

Having recently been on the front lines of a massive PST import operation, Sean Duffy offers advice and points out pitfalls. More than anything, he wishes he had a simple tool with which to banish PST hell, and finishes with some hard-won guidelines.

Every SysAdmin knows what kind of a nightmare PST files can be. In our case, the nightmare began around one year ago, when a new client joined our virtualised hosting platform. We inherited six of their physical servers, one of which was their monstrous 6U Exchange and file server. I say ‘monstrous’ not only because of the large rack size for a file server, but also because of the sheer mass of data on this server’s RAID array.  And the Pièce de résistance? Buried in this hefty pile of information lay 100 gigabytes of PST files. Any SysAdmin’s worst nightmare.

The issue at hand

Now, managing terabytes of customer information is all fine and dandy, until you come across large batches of PST files that are being accessed over your network, as everyone knows (or should know) how badly PST files can degrade the performance of your IT infrastructure when accessed in this fashion.Take a look at this Technet article when you get a chance to see what I mean; just to pick an example at random, one behaviour Microsoft attributes to PST files over a LAN is that “Write operations can take approximately four times longer than read operations”. They are essentially just poorly designed, unreliable database files and, in our case, they were scattered all over the network.

After getting all of the servers converted into virtual machines, we had more time on our hands to further investigate the intricacies of the IT platform we had just inherited. The bulk of the PST files were all dumped on the main file server, but there were still a significant number unaccounted for. While checking references in Outlook across many of the employees’ mail profiles, we discovered that PSTs were also being held in local profiles on each of their Terminal Servers. There was clearly no standardised method or location for storing PST files, and the search for all of these files was a mini project in itself.  Eventually we got all 100GB’s accounted for and stored in one central share, on a dedicated and fast random read/write LUN running RAID10.

The next issue was the method these poor souls were using to archive their e-mail. Their previous IT support team had half of them using their main Exchange mailboxes as a temporary “buffer” area to keep a week’s worth of e-mail, and anything older than that they were told to dump into their “Archives” (The PST files). The other half just had a combination of severely bloated Exchange mailboxes and PST files. On average, the mailboxes were around 2GB in size, with some as large as 11GB. Thankfully (can you hear the sarcasm?), the PST files were mostly between 1 and 2GB in size.

Clearly, an archiving solution was needed. For this we chose Red Gate’s Exchange Server Archiver, and in no time we had all of the exchange mailboxes over the First and Second storage groups archived with a seamless user experience. However, we were still left with one more problem; The PST files.

Naively, I produced a flawlessly written PDF tutorial document on how each user could identify their PST files, sizes and locations, detach them from Outlook, locate them in the network share and then import them back into their main Exchange mailboxes, ready to be archived by the Exchange Server Archiver overnight. This was all very well in theory, but in practise it turned out to be yet another nightmare.

After sending out the detailed instructions, complete with illustrated screenshots, and asking the users to import their own PST files into their mailboxes, our support team began receiving phone call after phone call, day in and day out, from users stuck on one of the particular steps, or even confused as to where to begin

Seeing as this company was quite IT “illiterate”, each user had to be spoon fed when it came to getting their PST files imported. We ultimately had to remote control each user’s session and perform the PST imports ourselves. This ended up consuming much of the support team’s precious time – time they could have spent helping other clients or working on proactive projects. More to the point, it took quite a lot time to deal with 100GB + of PST files. I even remember tweeting about this horrid experience.

Uncouth and frustrated tweets aside, another issue we ran into was that some of the PST files also contained non-Mail items, such as Calendars, tasks and contacts, so we had to add a special note in the instructions for the users who encountered these to contact us. This way, we could connect with these users and ensure that these curveball items were imported correctly into Exchange, and not into a “Mail and post item only” folder.

Last but not least, we found that the exchange and file servers were being hit by drops in performance as disk I/O spiked – the result of large PST files being copied between servers at random times throughout the day.

Looking back on this entire epic experience, we could really have done with a PST Searching and importer tool, so it was a shame that the Red Gate’s PST Importer Early Access Program had not yet opened to the public!

Solution

I have since had a fair amount of time to tinker with Red Gate’s PST Importer via that same Early Access Program, and am pleased to say that most of the headaches usually experienced when dealing with PST files can now be avoided. Let me briefly run you through how to use the PST Importer, and you’ll see what I mean.

After starting up the PST Importer console, you are greeted by a clean, simple-to-use interface, where you can quickly set up your first PST search or work with an existing “Import list” (which I’ll describe in a moment). If you’re going down the search route, you should install the PST Agent that comes with the Importer on the machines you wish to search. This is a tiny search assistant agent that runs as a service.

After selecting a new PST search, your active directory domain is searched for computers and a list is quickly displayed, allowing you to choose exactly which machines on your network to search for PST files. You can also choose whether you want to omit certain locations on the computers that are searched. I found that by choosing obvious locations to omit, such as the Windows and System directories, I was able to speed up my searches quite significantly.

Once you’ve made your decisions (and got the PST Agent installed), you are now able to kick the search off manually or on a predefined schedule. This second option is quite useful if, like me, you find that searching file servers during business hours causes (somewhat unsurprisingly) slowdowns because of the higher disk I/O. The scheduling option is greyed out in the Early Access Program, so I didn’t get a chance to play with it, but I’ll look forward to it in subsequent releases. After the PSTs have been discovered (with minimal effort on your part, I might add), the next step is to select the ones you want to import and push them over to a new or existing “Import list”.

Once you have your PST Import list set up, the next step is just a simple case of choosing where you want the PST files to go. There are quite a few options to choose from here, and this is where the PST Importer really shines, in my opinion. You can choose which mailbox individual PST files get imported to, as well as which Archive store they move into (using the Exchange Server Archiver). Alternatively, file attributes can be used to automatically determine which PST files go into which mailboxes – for example by using the “File owner” information on the PST file. Non-mail items, such as Calendars, contacts, tasks and notes, can also be automatically imported from the PST files. The Importer will also create the placeholders that Exchange Server Archiver uses, if you choose to do so. It is a remarkably pain-free experience.

Listed below is a brief summary of some of the features that Red Gate are offering in their PST Importer tool. I have added sub-points to each feature indicating how this would have made life easier for us had we used this tool to begin with.

  • PST searching across the domain.
    • The ability to have been able to search for PSTs across all servers and through all the user local/roaming profiles at specified times would have been an absolute godsend, and  would also have allowed us to keep server disk I/O to a minimum during work hours.
    • We could have omitted certain obvious locations that PSTs would definitely not be found, like the Windows or System directories, which would naturally have sped up our hunt for the elusive PST files.
  • Central management console to work with all PST files
    • A central console to manage all PST files across the network would have helped us keep everything organised and planned correctly. Not to mention providing a centralised location to check when we were after certain details or looking to import certain files.
  • Direct PST Importing to Exchange Server Archiver.
    • This is one of the most important feature in my opinion. I personally spent many hours manually importing PST files into Outlook Exchange mailboxes and writing up documentation to help users try (and apparently fail, on a semi-consistent basis) to do this themselves.
    • The users that did manage to get PSTs manually imported themselves also wreaked havoc on the exchange server. Imagine how you’d react if you started getting calls from every seemingly single user at a large company, each complaining that their “E-mails aren’t working”. Unexpected PST imports happened on two different occasions, filling the exchange server’s mailstore drives up, which subsequently dismounted the mailstores due to lack of disk space.
    • Having a set schedule, together with control over all of the PST importing ourselves would have prevented this, as we could have properly planned for a mass PST import overnight. Specifically, extending the virtual disk drives holding mail beforehand.
    • Disk I/O was also spiking at random times between the file server and the exchange server, as the more IT-literate users imported their PST files via Outlook. Again, a set schedule would have solved this.
    • Of course, the PST Importer also greatly speeds up the entire import process by importing and archiving in one clean sweep, and mailing in the placeholders! The destination detection is also great, as based on file permissions or “used by” information, the importer can figure out which PST file goes into which user’s mailbox.

Conclusion

To finish off, all I can say is that I hope you can learn from our experience. Going into battle with large batches of PST files can be daunting, and potentially cause serious problems for your IT infrastructure. Now, of course you can do your best to avoid this frustration by spending more time educating your users, planning ahead, doing your best to prepare your infrastructure, and giving your support team pep talks and Red Bull. But the amount of time that takes will increase exponentially as you support more users and, let’s be honest, they’re not all going to listen to you; sooner or later something will break. In situations like this, and so centralizing the project ‘s processes and control, and introducing little or no disruptions to regular workflows is the well-trodden path to keeping your sanity. Do yourself a favour and use a PST Importing tool.

Tags: , , ,

  • 14019 views

  • Rate
    [Total: 18    Average: 4.1/5]
  • boitumelo Tshwenyego

    Thanks a lot
    Good advices Sean! I had such an experience with PST files that are scattered across the organisation.Our companies were merging to form one company I had to recover them from individual user machines (about 200 of them) and back them up in an external Hard Drive. The other challenge was to mount them up in the new infrastructure, some were more that 2GB, and this was hell.

  • Shogan

    Multiple PCs
    Hi Boitumelo,

    Sounds like you also had a bit of a struggle there. Hopefully next time you’ll be able to plan for the situation and use PST Importing software. Here is a useful article on getting the PST Agents installed across across multiple PCs (as you mentioned you had in your case. http://sysadmin-talk.org/2010/03/using-microsoft-system-center-configuration-manager-to-deploying-pst-importers-agents/).

    Regards,
    Sean

  • MyOpinion

    Just found this ..
    Hi,
    Having read this, and going through a migration ourselves, we chose to use PST Accelerator (www.pstaccelerator.com) to do our migration through one of their partners, and are finding it a blessing as it addresses many of the things mentioned in here very nicely.

  • peterkozak

    PstFlightdeck
    Hi all,
    you should also consider if you are doing PST Migration in larger scale, to have a look on PST Flightdeck from QUADROtech.
    (http://www.quadrotech-it.com)