Reconciling Data Across Systems Using a Reconciliation Hub

In many enterprises, where there are a number of separate systems engaged in processing data, there arises the daunting task of checking and reconciling data as it flows between systems. Discrepancies in data must be detected, tracked and corrected as quickly as possible: there is no room for error in doing this. A Reconciliation Hub can provide the answer, as Rahul Gupta explains.… Read more

How to get ETL Horribly Wrong

ETL ( Extract, transform, load) doesn't have to be like a spell on hell. To make a success of ETL systems, you need the freedom and ability to make graceful U-turns when you detect a mistake in architecture or configuration: to fix the root problem rather than to merely tackle the symptoms. Feodor lists the eight most common root causes of failure in ETL systems, and how to fix them.… Read more

SSIS 2012 Projects: Deployment Configurations and Monitoring

The Project Deployment Model introduced in SSIS 2012, which was explained in the first part of this series, speeds up the deployment of database projects in which there may be hundreds of SSIS packages per project. Not only that, but deployments can be configured differently for each environments such as test and staging, and there are now ways of monitoring the status and performance of packages and of versioning the SSIS Catalog.… Read more

SSIS 2012 Projects: Setup, Project Creation and Deployment

It used to be that SQL Server Integration Services (SSIS) packages had to be deployed individually. Now, they can be all deployed together from a single file by means of the Project Deployment Model introduced in SSIS 2012. Where there are tens or even hundreds of SSIS packages to deploy, this system is essential. Feodor Georgiev talks us through the basics in the first of a three-part series.… Read more

Developing a Custom SSIS Source Component

SSIS was designed to be extensible. Although you can create tasks that will take data from a wide variety of sources, transform the data is a number of ways and write the results a wide choice of destinations, using the components provided, there will always be occasions when you need to customise your own SSIS component. Yes, it is time to hone up your C# skills and cut some code, as Saurabh explains.… Read more

SSIS Basics: Using the Merge Join Transformation

SSIS is able to take sorted data from more than one OLE DB data source and merge them into one table which can then be sent to an OLE DB destination. This 'Merge Join' transformation works in a similar way to a SQL join by specifying a 'join key' relationship. This transformation can save a great deal of processing on the destination. Annette Allen, as usual, gives clear guidance on how to do it.… Read more

Working with SSIS Data Types

In order to be able to take data from a variety of sources, manipulate it, and then export it to one of several destinations, SSIS has to use its own data types. Generally, the conversions of data types is handled implicitly, but if you hit problems, then you may need to intervene in one of several ways to ensure an appropriate conversion.… Read more

SSIS Basics: Setting Up Your Initial Package

When working with databases, the use of SQL Server Integration Services (SSIS) is a skill that often needs to be acquired quickly, from scratch. Up until now, it has been a curiously frustrating search to find out the basics, fast, in order to get up and running quickly. No longer, as Annette comes up with a simple introduction for the rest of us.… Read more