Automated Error Reporting in .NET Reflector – harnessing the most powerful test rig in existencePublished 17 June 2010 12:23 pm
I know a testing system that will find more bugs than all the unit testing, integration testing, and QA you could possibly do. And the chances are you’re not using it.
It’s called your users.
It’s a cliché that you should test so that you find your bugs rather than your users. Of course you should. But it’s also a cliché that no software is ever shipped bug-free. Lost cause? No, opportunity!
I think .NET Reflector 6 is pretty stable. In fact I know exactly how stable it is, because some (surprisingly high) proportion of its users tell me every time it crashes:
If they press “Send Error Report”, I get:
And then I fix it. As a rough guess, while a standard stack trace is enough to fix a problem 30% of the time, having all those local variables in the stack trace means I can fix it about 80% of the time.
How does this all happen? Did it take ages to code this swish system? Nope, it was one checkbox in SmartAssembly. It adds some clever code to your assembly to capture local variables every time an exception is thrown, and to ask your user to report it to you, with a variety of other useful information.
Of course not all bugs show up as exceptions. But if you get used to knowing that SmartAssembly will tell you when an exception happens, you begin to change your coding style. Now, as long as an exception gets thrown in any situation you don’t expect, you’ll fix it if it ever happens. You’ll start throwing exceptions liberally, and stop having to think about whether tiny edge cases are possible, as long as they throw an exception if they happen.
You can try this out for yourself by downloading the free trial of SmartAssembly.