08 September 2011

SmartAssembly Error Reporting: with the bug-finding power of more than 20 testers

At Red Gate, we use the SmartAssembly automated-error reporting system in the applications we sell.  When an error or exception occurs ‘in-the-wild’, it is reported back to Red Gate HQ, along with details about the error (stack trace, deployment environment, local variables, and more). Using the ‘SmartAssembly Sync for JIRA’ system, these errors are then automatically filed as bugs in our JIRA bug-tracking system. I should point out that Sync-For-JIRA is rather clever – it adds stack traces, it doesn’t file duplicates, etc.

We have been using this system now for about a year, and were recently surprised to discover that this system files more bugs than our top 20 testers put together. SmartAssembly actually finds about 15 times as many bugs as one of our more prolific testers.


66% of issues (bugs, enhancements, etc) raised in the past year come from SmartAssembly Error Reporting.

In the past year, we have learned (rather embarrassingly) of thousands of different errors in our products.  In the past we would have had to rely on good-hearted customers sending us in limited amounts of reproductory data, but thanks to SmartAssembly Automated Error Reporting we can automatically get all the data we need to reproduce the bug.

However, the type of bugs found by this handy tool tends to be different:  unhandled exceptions, crashes, and unexpected environments. On the other hand, ‘flesh-and-blood’ testers tend to find UI glitches, workflow problems, etc.  So, SmartAssembly error-reporting definitely complements real testers, rather than replaces them.

For more articles like this, sign up to the fortnightly Simple-Talk newsletter.


  • Rate
    [Total: 0    Average: 0/5]

Jason Crease studied maths and computer Science at Cambridge University and joined Redgate several years ago as a Test Engineer. He specialised for some time in Testing .NET applications and was part of the team responsible for the development of ANTS Profiler and .NET Reflector. After working as a DevOps engineer supervising Redgate's own IT systems, he joined the DLM team, working on the practicalities of the management and automation of the application lifecycle.

View all articles by Jason Crease