Av rating:
Total votes: 19
Total comments: 5


Pop Rivett
Pop Rivett and the Case of the Rogue SPIDs
22 March 2007

My dear fellow, you know my methods…

Pop Rivett put down his violin, stretched back in his fireside chair, and stared severely at me. "My dear fellow, you know my methods. The mystery of the rogue SPIDs has been one of the more exacting tasks it has been my fortune to be given.

A process in a complex database occasionally, and apparently randomly, manages to put table locks on vital tables. The result? Several applications are brought to a complete halt. Tracking down the problem was indeed taxing, but through the faculties of deduction and of logical synthesis which I have endeavoured to apply to the task, I consider that I helped the DBA, Lestrade, with some small points in connection with it."

I smiled ruefully, and shook my head. "I was relying on traces and found myself completely inundated and overwhelmed by the detail. I doubt I ever would have solved the mystery!"

"Not so, you are too modest. However, I allow that a certain selection and discretion is necessary for producing a rapid result".

Rivett strummed a few notes on his violin before continuing. "In solving the mysteries of the performance of a database application, one often needs to view a list of the processes currently running on your server; how long each process has been running, what the command was, what is blocking, and how much I/O it is causing."

"How in the name of good fortune would you know all that, Rivett?"

"Elementary, dear fellow. All you need is a stored procedure that will report any SPIDs that are blocking others, the last command executed, and other useful diagnostic attributes. You then need to run it every minute or so on the SQL Server agent, inserting the results into a log table which has a 'timestamp' column, until you catch the process 'in flagrante delicto'. You will then be able to see the command that is doing the blocking, and the processes being blocked."

"Surely, Rivett!" I interjected, "sp_who and sp_who2 will show you the basic details of the processes, and the SPID is just the primary key of the sysprocesses table"

Rivett looked at me pityingly. "Quite! But they will show you all the processes running, not just the blocking processes. Furthermore, they will not tell you what was being executed will they? Try out my sp_SpidByStatus procedure. It requires little skill.

You'll want to call the procedure with a 'blk' parameter to reveal all blocked and blocking spids. The default status is 'runnable' as that is the usual requirement, but you can call the procedure with 'all' as a parameter, so that all SPIDs are displayed. If you call the procedure with a SPID, then just that SPID is displayed.

Examining the sp_SpidByStatus procedure

Take a look at the procedure. I'd show you it on my laptop except the damned battery is flat! Note form will have to suffice…

if exists (select * from sysobjects where id = object_id(N'[dbo].[sp_SpidByStatus]')
and OBJECTPROPERTY(id, N'IsProcedure') = 1)

drop procedure [dbo].[sp_SpidByStatus]

GO

 

Create procedure [dbo].[sp_SpidByStatus]

@status varchar(20) = 'runnable'

as

/*

exec sp_SPIDbyStatus              -- all spids whith status runnable

exec sp_SPIDbyStatus 'sleeping'   -- all spids whith status sleeping

exec sp_SPIDbyStatus 'background' -- all spids whith status background

exec sp_SPIDbyStatus 'sleeping'   -- all spids whith status sleeping

exec sp_SPIDbyStatus 'all'        -- all spids

exec sp_SPIDbyStatus 'blk'        -- all blocked or blocking spids

exec sp_SPIDbyStatus '74'         -- an individual spid - also gives subthreads

select * from master..sysprocesses where spid = 56

*/

set transaction isolation level read uncommitted

set nocount on

 

declare       @cmd varchar(1000)

 

declare @buf varchar(1000) ,

       @id int ,

       @spid int ,

       @maxSpid int

       create table #spid (spid int, command varchar(1000) null)

       create table #temp (x varchar(100), y int, s varchar(1000), id int identity (1,1))

       create table #spids (spid int)

       if isnumeric(@status) = 1

       begin

              insert #spids select @status

       end

       else if @status = 'blk'

       begin

              insert #spids

              select spid from master..sysprocesses where blocked <> 0

              union

              select blocked from master..sysprocesses where blocked <> 0

       end

       else

       begin

              insert #spids select spid from master..sysprocesses
where (status = @status or @status = 'all') and ecid = 0

       end

       select        @spid = 0 ,

              @maxSpid = max(spid)

       from   #spids

 

       while @spid < @maxSpid

       begin

              select @spid = min(spid) from #spids where spid > @spid

 

              select @cmd = 'dbcc inputbuffer (' + convert(varchar(10),@spid) + ')'

 

              delete #temp

 

              insert #temp

              exec (@cmd)

 

              select        @id = 0 ,

                     @buf = ''

              select @buf = @buf + replace(replace(s,char(10),'|'),char(13),'|')

 

              from #temp

 

              insert        #spid

              select @spid, @buf

       end

 

       select        blk = case when s.blocked <> 0 then convert(varchar(3),
s.blocked) else '   ' end ,

              spid = convert(varchar(4),#spid.spid) ,

              s.physical_io ,

              status = left(s.status,12) ,

              last_batch = convert(varchar(23),s.last_batch,121) ,

              s.cmd ,

              #spid.command ,

              login_time = convert(varchar(23),s.login_time,121) ,

              s.HostName

       from   #spid ,

              master..sysprocesses s

       where  s.spid = #spid.spid

       and    (ecid = 0 or isnumeric(@status) = 1)

       order by s.status, #spid.spid

 

       drop table #spid

       drop table #temp

 

go

 

/* And you will need a table to hold the list of blocking,

and blocked, processes */

 

CREATE TABLE [dbo].[BlockingSPids](

       [BlockingSpids_ID] [int] IDENTITY(1,1) NOT NULL,

       [blk] [varchar](3)| NOT NULL,

       [spid] [varchar](4)| NOT NULL,

       [physical_io] [int] NOT NULL,

       [status] [varchar](12)| NOT NULL,

       [last_batch] [datetime] NOT NULL,

       [cmd] [nchar](32)| NOT NULL,

       [command] [varchar](1000)| NOT NULL,

       [login_time] [datetime] NOT NULL,

       [HostName] [nchar](256)| NOT NULL,

       [insertionDate] [datetime] NOT NULL

              CONSTRAINT [DF_BlockingSPids_insertionDate]

                                  DEFAULT (getdate()),

 CONSTRAINT [PK_BlockingSPids]

                                  PRIMARY KEY CLUSTERED

(

       [BlockingSpids_ID] ASC

) ON [PRIMARY]

) ON [PRIMARY]

 

/* finally, all you will now need to do is to place this code

on the SQL Server agent to catch and identify those Rogue Spids.*/

 

insert into BlockingSpids(

blk,spid,physical_io,status,last_batch,cmd,command,login_time,HostName)

       execute sp_SpidByStatus 'blk'

To keep a record of what is happening on your server you can, of course, schedule the call on the SQL Server agent and output the result of this procedure to a table. In short measure, you should have trapped the identity of that rogue process, and the other processes that it is blocking."

"Well, now you explain it, Rivett, it all seems so simple. How silly of me to be overawed by the problem"

"I begin to think", replied Rivett wistfully, "that I make a mistake in explaining. 'Omne ignotum pro magnifico'. My poor reputation, such as it is, will suffer shipwreck if I am so candid."



This article has been viewed 3849 times.
Pop Rivett

Author profile: Pop Rivett

Pop spent his formative years working in assembler on IBM Series/1 but retrained in VB when that went out of fashion. He soon realised how little relational database expertise existed in most companies and so started to spend most of his time working on that. He now sticks to architecture, release control / IT processes, SQL Server, DTS, SSIS, and access methods in VB/ASP/.NET/Crystal Reports/reporting services. He has been involved with SQL Server since the old days of v4.2 to v2005. He tries to stay away from anything presentation oriented (see www.mindsdoor.net). Theoretically he is semi-retired but seems to keep being offered potentially interesting work.

Search for other articles by Pop Rivett

Rate this article:   Avg rating: from a total of 19 votes.


Poor

OK

Good

Great

Must read
 
Have Your Say
Do you have an opinion on this article? Then add your comment below:
You must be logged in to post to this forum

Click here to log in.


Subject: Useful
Posted by: LeeM (view profile)
Posted on: Thursday, March 22, 2007 at 11:52 AM
Message: Very helpful for a problem I've been experiencing with one of my in-house databases.

Thanks Pop!

Subject: SP4
Posted by: ACALVETT (view profile)
Posted on: Thursday, March 22, 2007 at 12:19 PM
Message: Very useful procedure.

One thing to consider that may confuse people is that when running SQL 2000 SP4 a spid may appear as blocking itself. This is described at http://support.microsoft.com/default.aspx?scid=kb;en-us;906344
Would be very easy to make a small update to cater for this scenario.

If people wanted to further enhance could also slip in the results of ::fn_get_sql to complement the inputbuffer.

A

Subject: small changes needed
Posted by: Anonymous (not signed in)
Posted on: Friday, March 23, 2007 at 5:54 AM
Message:
hi

just tried it, small changes for anyone else (maybe just my system specific)

1)
create procedure needs [dbo].<proc name>
2)
last 2 lines, change the exec 'proc name' to the correct name (to sp_SPIDbyStatus and not sp_nrSpidByStatus)
3)
for completeness, grant execute for SP

useful stuff in the armoury

Subject: re: small changes
Posted by: Tony Davis (view profile)
Posted on: Friday, March 23, 2007 at 12:15 PM
Message: Hi,

Thank you for pointing out these mistakes. I have corrected them in the article and the CODE DOWNLOAD.

Best,

Tony (Simple-Talk Ed.)

Subject: Inserting results to a table
Posted by: shepton (view profile)
Posted on: Wednesday, April 04, 2007 at 10:44 AM
Message: Hi there. Very interesting + useful stuff + very timely for me as I am getting blocking issues on a production server but I have a problem when I try to use INSERT... EXECUTE... to save the results of the proc to a table - basically SQL Server won't let me :-

An INSERT EXEC statement cannot be nested.

Incidentally, I have also amended your proc slightly to include dbid in the result set as the server I have problems with has 14 application databases running on it and it is not always apparent to me what code relates to which database.

Mark

 









Phil Factor
The Data Center that Exploded
 A while back, in a Simple-Talk editorial meeting, someone bet Phil that he couldn't come up with a Halloween story.... Read more...



 View the blog
SQL Toolbelt 2008: Predominantly an Engineering Task
 The conversion of the Red-Gate tools to be compatible with SQL Server 2008 might not seem, on first... Read more...

Audit Crosschecks
 In this short article, the second of a 2-part series, William suggests a solution, using SQL Data... Read more...

SQL Response: The dim sum interview
 Richard Morris met David and Nigel of the SQL Response team, in a dim sum Restaurant in Cambridge. They... Read more...

XML Jumpstart Workbench
 In which Robyn and Phil decide that the best way of starting to learn XML is to jump in and take a ride... Read more...

Discovering Security Uses for SQL Compare
 Much of the security of SQL Server is implemented as part of the database schema. This provides some... Read more...

Beginning SQL Server 2005 Reporting Services Part 1
 Steve Joubert begins an in-depth tour of SQL Server 2005 Reporting Services with a step-by-step guide... Read more...

Ten Common Database Design Mistakes
 Database design and implementation is the cornerstone of any data centric project (read 99.9% of... Read more...

SQL Server Full Text Search Language Features
 SQL Full-text Search (SQL FTS) is an optional component of SQL Server 7 and later, which allows fast... Read more...

Beginning SQL Server 2005 Reporting Services Part 2
 Continuing his in-depth tour of SQL Server 2005 Reporting Services, Steve Joubert demonstrates the most... Read more...

Executing SSIS Packages
 Nigel Rivett demonstrates how to execute all SSIS packages in a given folder using either an SSIS... Read more...

Over 150,000 Microsoft professionals subscribe to the Simple-Talk technical journal. Join today, it's fast, simple, free and secure.

Join Simple Talk