Andras

Software Architect - Red Gate Software

Pivots with Dynamic Columns in SQL Server 2005

Published Friday, September 14, 2007 9:28 AM

Pivots in SQL Server 2005 can rotate a table, i.e. they can turn rows into columns. PIVOTs are frequently used in reports, and they are reasonably simple to work with. However, I've seen quite a few questions about this operator. Most questions were about the column list in the PIVOT statement. This list is fixed, but many times the new columns are determined by the report at a later stage. This problem is easily solved when we mix pivots with dynamic SQL, so here is a very simple example about how to dynamically generate the pivot statement:

PIVOT allows you to turn data rows into columns. For example, if you have a table like this (I use only three months here for simplicity):

CREATE TABLE Sales ([Month] VARCHAR(20) ,SaleAmount INT)

INSERT INTO Sales VALUES ('January', 100)
INSERT INTO Sales VALUES ('February', 200)
INSERT INTO Sales VALUES ('March', 300)

SELECT FROM SALES
 

Month             SaleAmount
----------------  -----------
January           100
February          200
March             300 

Suppose we wanted to convert the above into this:

 
January     February    March
----------- ----------  ----------
100         200         300

We can do this using the PIVOT operator, as follows:

SELECT  [January]
      [February]
      [March]
FROM    SELECT    [Month]
                  SaleAmount
          FROM      Sales
        p PIVOT SUM(SaleAmount)
                    FOR [Month] 
IN ([January],[February],[March])                   ) AS pvt

However, in the above example, I have the column names fixed as the first three months. If I want to create a result in which the columns are dynamic (for example, they are read from a table), then I need to construct the above query dynamically. To demonstrate this let’s look at the following example:

In the first table I have the column names I want to use:

CREATE TABLE Table1 (ColId INT,ColName VARCHAR(10))
INSERT INTO Table1 VALUES(1, 'Country')
INSERT INTO Table1 VALUES(2, 'Month')
INSERT INTO Table1 VALUES(3, 'Day')

In the second table I have the data. This consists of a row identifier (tID), a column ID (ColID) that refers to the column type in Table1, and a value:

CREATE TABLE Table2 (tID INT,ColID INT,Txt VARCHAR(10))

INSERT INTO Table2 VALUES (1,1, 'US')
INSERT INTO Table2 VALUES (1,2, 'July')
INSERT INTO Table2 VALUES (1,3, '4')
INSERT INTO Table2 VALUES (2,1, 'US')
INSERT INTO Table2 VALUES (2,2, 'Sep')
INSERT INTO Table2 VALUES (2,3, '11')
INSERT INTO Table2 VALUES (3,1, 'US')
INSERT INTO Table2 VALUES (3,2, 'Dec')
INSERT INTO Table2 VALUES (3,3, '25')

Now I would like to retrieve data from these two tables, in the following format:


tID         Country    Day        Month
----------- ---------- ---------- ----------
1           US         4          July
2           US         11         Sep
3           US         25         Dec 

In other words I want to turn the data rows in Table2 into columns. If I had a fixed set of columns for the result, i.e. the columns Country, Day, and Month were fixed, I could use SQL Server 2005’s PIVOT operator in a query like:

SELECT  tID
      [Country]
      [Day]
      [Month]
FROM    SELECT    t2.tID
                  t1.ColName
                  t2.Txt
          FROM      Table1 AS t1
                    JOIN Table2 
AS t2 ON t1.ColId t2.ColID         p PIVOT MAX([Txt])                     FOR ColName IN [Country][Day],                                      [Month] ) ) AS pvt ORDER BY tID ;

However I need to construct this query dynamically, because the column names Country, Day, and Month are specified in a table, and can be changed independently from my query. In our case these columns are given in Table1.

In the first step to generate the final pivot query I need to create the list of columns, in this case [Country], [Day], [Month].

Since there is no string concatenation aggregator in SQL (a concatenation aggregator would not be deterministic without some order restriction), and since the column names are stored in rows of a table, I need to flatten these columns into a single row or variable. There are various solutions to achieve this. One solution would be to use a query like:

DECLARE @cols NVARCHAR(2000)
SELECT  @cols COALESCE(@cols ',[' colName ']',
                         '[' colName ']')
FROM    Table1
ORDER BY colName

This query works both on SQL Server 2000 and 2005. It is efficient, but some may not like it because it uses the same variable (@cols) on both sides of an assignment. Another solution that works on SQL Server 2005 only is to use XML PATH.

DECLARE @cols NVARCHAR(2000)
SELECT  @cols STUFF(( SELECT DISTINCT TOP 100 PERCENT
                                '],[' t2.ColName
                        FROM    Table1 AS t2
                        ORDER BY '],[' t2.ColName
                        FOR XML PATH('')
                      ), 12'') + ']'

This second query (I’ve seen this posted by Peter Larsson) has the advantage that it does not use the @cols variable on the right hand side. I like this solution more, since this can be extended as a general string concatenation aggregate in more complex queries.

Both of the above queries generate, from Table1, the string: ‘[Country],[Day], [Month]’. This column list is used twice in the pivot query that we aim to construct. Once it is use in the list of columns that we want to retrieve, and once it is used as the list of values that should become columns. Having constructed this list of columns above, we can just concatenate it with the missing parts of the pivot query like:

DECLARE @query NVARCHAR(4000)
SET @query N'SELECT tID, '+
@cols +'
FROM
(SELECT  t2.tID
      , t1.ColName
      , t2.Txt
FROM    Table1 AS t1
        JOIN Table2 AS t2 ON t1.ColId = t2.ColID) p
PIVOT
(
MAX([Txt])
FOR ColName IN
( '+
@cols +' )
) AS pvt
ORDER BY tID;'

Executing this with

EXECUTE(@query)

will give us the expected result: a table that is pivoted and shows columns that were specified in a table:

tID         Country    Day        Month
----------- ---------- ---------- ----------
1           US         4          July
2           US         11         Sep
3           US         25         Dec
by András

Comments

 

Adam Machanic said:

I know this is just a touch on the anal-retentive side, but as currently written your queries are technically a vector for an injection attack (think of a malnamed column). To thwart this issue, please use the QUOTENAME function instead of concatenating square brackets on either side of the column name.

September 17, 2007 12:18 PM
 

András said:

There is no such thing like paranoia :). You are right Adam. While in my example the columns are not meant to come form a user directly, and I assumed it is "clean", there is no reason not to fortify code by default. Thanks for your comment.
 Andras
September 18, 2007 9:17 AM
 

Cheverton said:

Your approach is clever and instructive but isn't the following simpler?

DECLARE @query NVARCHAR(4000)

Select @query = N'SELECT tID
     , [1] as ['+ a.ColName + ']
     , [2] as ['+ b.ColName + ']
     , [3] as ['+ c.ColName + ']
FROM    ( SELECT    t2.tID
                 , t2.ColId
                 , t2.Txt
         FROM      Table2 t2
       ) p PIVOT ( MAX([Txt])
                   FOR ColId IN ( [1], [2], [3] ) ) AS pvt
ORDER BY tID ;'
From
Table1 a, Table1 b, Table1 c
where
a.ColId=1 and b.ColId=2 and c.ColId=3;

exec( @query)
September 18, 2007 9:41 AM
 

Cheverton said:

I needed to think 2 seconds more. Of course, your approach extends dynamically for any number of columns, not just any names of columns.
September 18, 2007 9:53 AM
 

» Pivot Tables with Dynamic Columns » Flixon.com » Blog Archive said:

October 29, 2007 4:53 PM
 

Praddumna said:

Hi
Can we insert a new column in our query result and have a default value to it.




December 1, 2007 12:34 AM
 

Pivots with Dynamic Columns in SQL Server 2005 « Quarksoft’s Notes said:

January 7, 2008 1:41 PM
 

Ferenc said:

Dear Andras!

For me isn’t working.
It is said The name ‘Select …’ is not a valid identifier.

When I copy the @query content SQL syntax, and run in a new query it’s completted succesfully.

Can you help me?
January 28, 2008 7:34 AM
 

suruchijain said:

Hi András!

Well I tried using ur tick, I guess the "pivot" keyword in the @query is not accepted by SQL Server 2000. the error I get on executing :

this is what I do :

DECLARE @cols NVARCHAR(2000)
SELECT  @cols = COALESCE(@cols + ',[' + colName + ']',
                        '[' + colName + ']')
FROM    Table1
ORDER BY colName
--print @cols

DECLARE @query NVARCHAR(4000)
SET @query = N'SELECT tID, '+
@cols +'
FROM
(SELECT  t2.tID
     , t1.ColName
     , t2.Txt
FROM    Table1 AS t1
       JOIN Table2 AS t2 ON t1.ColId = t2.ColID) p
PIVOT
(
MAX([Txt])
FOR ColName IN
( '+
@cols +' )
) AS pvt
ORDER BY tID;'

exec @query

and the error I get is:

Server: Msg 203, Level 16, State 2, Line 26
The name 'SELECT tID, [Country],[Day],[Month]
FROM
(SELECT  t2.tID
     , t1.ColName
     , t2.Txt
FROM    Table1 AS t1
       JOIN Table2 AS t2 ON t1.ColId = t2.ColID) p
PIVOT
(
MAX([Txt])
FOR ColName IN
( [Country],[Day],[Month] )
) AS pvt
ORDER BY tID;' is not a valid identifier.

Could u plz tell me where I m going wrong ?
February 6, 2008 4:11 AM
 

Ferenc said:

Hi Suruchijain!

The solution is: Execute(@query) instead of exec @query!
February 7, 2008 1:54 AM
 

paulie said:

This is great, kudos to you sir! I've been searching for a solution like this for ages. This is a real enabler, thanks so much for sharing your knowledge.
March 1, 2008 2:03 PM
 

http://www.simple-talk.com/community/blogs/andras/archive/2007/09/14/37265.aspx said:

March 16, 2008 3:27 PM
 

thot said:

Hi!

This is a great article.

But i'm having problems to do this in my query.

I doing and quiz analysis and i want to do this dinamically.

i have 2 tables in the DB like this:
  -- Aswear (IDAsw, IDQuiz, IDOption, IDQuestion)
  -- Option (IDOption, IDQuestion, Item, Order)

And i want something like this:

Sex     | Yes | No
M        | 2    | 2
F         | 3    | 6

The column "Sex" have IDQuestion = 1 and the other question have only 2 option (Yes or No) and have the IDQuestion = 11

This is possible by using pivot??
Anyone can showme an example to get this??

Regards

April 15, 2008 12:24 PM
 

alangrn said:

Thanks. This is a great article. How do I wriet these results to a table? I am trying to get the results from the cross tab to be linked to another query.
July 3, 2008 7:36 AM
 

BRebey said:

Excellent article!  Thanks a million for all the work!  

If you ever discover a way to use dynamic columns without the size limitation of a local variable, please let me know!  This solution won't work for me, as my requirements are to support 2880 columns - i.e., two days of data samples taken at 1-minute intervals, with each time interval in a column.  

Thanks again!
August 2, 2008 12:58 AM
You need to sign in to comment on this blog

About András

András Belokosztolszki is the architect of SQL Compare versions 4, 5, 6 and 7, SQL Log Rescue and SQL Refactor. He is focused on database internals, database synchronization and database schema evolution.

















<September 2007>
SuMoTuWeThFrSa
2627282930311
2345678
9101112131415
16171819202122
23242526272829
30123456
Ziggurats, Batman and the Town Crier
 We asked Brian for a description of the Help System for the software he's working on and ends up... Read more...

The Future of NET Reflector
 Simple Talk asked freelance writer Bob Cramblitt to sit down with the two people behind the agreement... Read more...

Software Tool Design: Remote User Testing
 If you are developing a software product, you'll know that the sooner you can get feedback from the... Read more...

SQL Toolbelt 2008: Predominantly an Engineering Task
 The conversion of the Red-Gate tools to be compatible with SQL Server 2008 might not seem, on first... Read more...

Andrew Tanenbaum: Geek of the Week
 Andrew Tanenbaum has had an immense influence on the way that operating systems are designed. He... Read more...