Back in December last year, I asked myself: could it be that .NET developers think that you need three days and a PhD to do performance profiling on their code?

What if developers are shunning profilers because they perceive them as too complex to use? If so, then what method do they use to measure and analyse the performance of their .NET applications? Do they even care about performance?

So, a few weeks ago, I decided to get a 1-minute survey up and running in the hopes that some good, hard data would clear the matter up once and for all. I posted the survey on Simple Talk and got help from a few people to promote it. The survey consisted of 3 simple questions:

Amazingly, 533 developers took the time to respond - which means I had enough data to get representative results! So before I go any further, I would like to thank all of you who contributed, because I now have some pretty good answers to the troubling questions I was asking myself. To thank you properly, I thought I would share some of the results with you.

First of all, application performance is indeed important to most of you. In fact, performance is an intrinsic part of the development cycle for a good 40% of you, which is much higher than I had anticipated, I have to admit. (I know, "Have a little faith Laila!")

When asked what tool you use to measure and analyse application performance, I found that nearly half of the respondents use logging statements, a third use performance counters, and 70% of respondents use a profiler of some sort (a 3^rd party performance profilers, the CLR profiler or the Visual Studio profiler).

The importance attributed to logging statements did surprise me a little. I am still not sure why somebody would go to the trouble of manually instrumenting code in order to measure its performance, instead of just using a profiler. I personally find the process of annotating code, calculating times from log files, and relating it all back to your source terrifyingly laborious. Not to mention that you then need to remember to turn it all off later! Even when you have logging in place throughout all your code anyway, you still have a fair amount of potentially error-prone calculation to sift through the results; in addition, you'll only get method-level rather than line-level timings, and you won't get timings from any framework or library methods you don't have source for. To top it all, we all know that bottlenecks are rarely where you would expect them to be, so you could be wasting time looking for a performance problem in the wrong place.

On the other hand, profilers do all the work for you: they automatically collect the CPU and wall-clock timings, and present the results from method timing all the way down to individual lines of code. Maybe I'm missing a trick. I would love to know about the types of scenarios where you actively prefer to use logging statements.

Finally, while a third of the respondents didn't have a strong opinion about code performance profilers, those who had an opinion thought that they were mainly complex to use and time consuming. Three respondents in particular summarised this perfectly:

"sometimes, they are rather complex to use, adding an additional time-sink to the process of trying to resolve the existing problem".

"they are simple to use, but the results are hard to understand"

"Complex to find the more advanced things, easy to find some low hanging fruit".

These results confirmed my suspicions: Profilers are seen to be designed for more advanced users who can use them effectively and make sense of the results.

I found yet more interesting information when I started comparing samples of "developers for whom performance is an important part of the dev cycle", with those "to whom performance is only looked at in times of crisis", and "developers to whom performance is not important, as long as the app works". See the three graphs below.

Sample of developers to whom performance is an important part of the dev cycle:

Sample of developers to whom performance is important only in times of crisis:

Sample of developers to whom performance is not important, as long as the app works:

As you can see, there is a strong correlation between the usage of a profiler and the importance attributed to performance: indeed, the more important performance is to a development team, the more likely they are to use a profiler. In addition, developers to whom performance is an important part of the dev cycle have a higher tendency to use a much wider range of methods for performance measurement and analysis. And, unsurprisingly, the less important performance is, the less varied the methods of measurement are.

So all in all, to come back to my random questions:

.NET developers do care about performance. Those who care the most use a wider range of performance measurement methods than those who care less. But overall, logging statements, performance counters and third party performance profilers are the performance measurement methods of choice for most developers.

Finally, although most of you find code profilers complex to use, those of you who care the most about performance tend to use profilers more than those of you to whom performance is not so important.

The very best software is almost always originally the creation of a single person. Readers of our 'Geek of the Week' will know of a few of them.  Even behemoths such as MS Word or Excel started out with one programmer.  There comes a time with any software that it starts to grow up, and has to move from this form of close parenting to being developed by a team.  This has happened several times within Red-Gate: SQL Refactor, SQL Compare, and SQL Dependency Tracker, not to mention SQL Backup, were all originally the work of a lone coder, who subsequently handed over the development to a structured team of programmers, test engineers and usability designers.

Because we loved .NET Reflector when Lutz Roeder wrote and nurtured it, and, like many other .NET developers, used it as a development tool ourselves, .NET Reflector's progress from being the apple of Lutz's eye to being a Red-Gate team-based development  seemed natural.  Lutz, after all, eventually felt he couldn't afford the time to develop it to the extent it deserved. Why, then, did we want to take on .NET Reflector?  Different people may give you different answers, but for us in the .NET team, it just seemed a natural progression. We're always very surprised when anyone suggests that we want to change the nature of the tool since it seems right just as it is. .NET Reflector will stay very much the tool we all use and appreciate, although the new version will support .NET 4, and will have many improvements in the accuracy of its decompiling.

Whilst we've made a lot of improvements to Reflector, the radical addition, which we hope you'll want to try out as well, is '.NET Reflector Pro'. This is an extension to .NET Reflector that allows the debugging of decompiled code using the Visual Studio debugger. It is an add-in, but we'll be charging for it, mainly because we prefer to live indoors with a warm meal, rather than outside in tents, particularly when the winter's been as cold as this one has. We're hoping (we're even pretty confident!) that you'll share our excitement about .NET Reflector Pro.

Check it out with a free download of the Beta.

Cheers,

Laila

Have you noticed occasions when your browser suddenly starts eating memory? Even the latest versions of Firefox, Opera or Internet Explorer can cause a PC to grind to a halt by grabbing all available memory. Browsers are greedy by their nature. They grab memory for such things as the Back-Forward cache in order to speed performance. However, they also suffer from JavaScript programming errors that prevent JavaScript and DOM objects from being recycled once they have fallen out of use. This is most likely to happen if you have created reference loops via closures that involve both DOM and JavaScript objects. This normally involves event handlers, but can happen in a variety of circumstances.

JavaScript has its own built-in mark and sweep garbage collection that works pretty well. The programmer does not normally have to explicitly signal that an object needs to be recycled: it is done automatically once all objects that reference the object go out of scope and its resources are released.

The problems happen because most browsers have separate systems for recycling resources in the DOM and JavaScript. The recycling of DOM objects is usually done via a reference-counting system. This system returns the memory to the heap once the reference count falls to zero. If a circular reference is detected within either recycling system, then all is well and the circularity is detected and managed, but if it involves both JavaScript and the DOM, as when a DOM object has a reference back to a closure, such as an event handler, then the objects can't be released even after the browser has navigated away from the page that caused the problem.

The seasoned JavaScript programmer will break circular references by assigning a null to the reference to the object once the object is no longer required. However, the best solution for the JavaScript programmer is probably to use a framework such as jQuery which manually releases event handlers that are assigned via the framework. This is little consolation to the user who needs to access many sites, few of which have taken any steps to ensure that their JavaScript avoids memory leaks.

Browsers have become so central to the work we do on PCs that we have become increasingly tolerant of the demands that these applications make on PC memory. However, it seems absurd that one should need to keep closing and re-starting the browser just to prevent the memory-drain from throttling the PC. It is particularly galling when the cause is programming error.

Do you have any ideas on how the browsers excessive appetite for memory can be suppressed? It would be great to hear what you think.

Cheers,

Laila

In this guest editorial, our very own Chris Allen brings us the first of hopefully many Thoughts from the Help Desk. As Halloween has just gone by, he decided it would be appropriate to discuss black magic, a frightening thought, and how to bring your source code back from the dead.

Attention all developers! Unless you're using a military-grade code-protection environment or a seriously sophisticated code-protection tool, your code is vulnerable to decompilation and disassembly; in short: all your source are belong to us. But everyone knows this, right? I didn't think I'd need to give anyone a wakeup call but, as I found out recently on the help-desk, there's still a smattering of people who still think compilation equals obfuscation. I have some sympathy with such n00bs; I can still remember the awe in which I held the first decompiler I ever used- one for the Psion's EPOC language. I wondered how on earth it was possible to reverse the process of compilation; Type and Data Flow Analysis sounded like so much black magic. But, as in law, ignorance is no defense. It will always be the case that dynamic code is vulnerable to decompilation and discovery (with the honorable exceptions above), and this is especially true with interpreted languages. So, I write this so that we can all draw a line under any illusions we've clung to, deal with this fact of developer life and then - embrace it! Soon enough, you'll be very glad of it - for example, recently my colleague gave me a small application that did a great job of customizing our SQL Compare engine. He only had the runtime assembly and had lost the source code but I really needed to understand what he had done - one quick flick of the wrist later and Reflector had not only recovered the source, but had created the Visual Studio project for it too (I *really* love that feature)! And I've heard many other stories about why Reflector is so useful - the one comment that sticks in my mind is from the developer who said, "If it wasn't for Reflector, I'd be doing a different job".

So how does this 'Dark Art' work? As I say- code has always been vulnerable and "reversible" but, until the invention of the technology behind intermediate languages (such as the .NET languages), the job of reversing the code was akin to decryption (think WWII, "Enigma" code-breaking; better still, don't think - just watch the film :-) ).  Intermediate languages don't directly generate the machine code (which is the really hard bit to reverse-engineer)- they generate "IL", each line of which has a reasonably clear derivation (often a one-to-one correspondence with source code, in fact). This intermediate level of code is generally relatively easy to pull apart, and if you find this technology a little frightening, it's maybe comforting to know that decompilation is not an exact science. There isn't a 100% one-to-one correspondence and, sometimes, decompilation is equivalent to the classic Halting problem. It's still hard to do very well, but we think we're still on top of the game. Welcome to Reflector Pro.

-Guest post by Chris Allen

Most so-called memory leaks in .NET applications aren't really leaks. After all, if an object has a valid reference to it within the current scope, then the object is being used, even if you didn't intend it. The Garbage Collector will shrug and assume you need it. If your application continues to create objects that never leave the active scope, or creates a host of class global variables, or stores a large block of data in a class global collection, your profiler will show a growth in memory consumption. It certainly will look like a leak, but the programming errors are more fundamental. You can reduce the problem by setting the unused variable to Null (or Nothing in VB.NET) but the answer is really a refactoring to restrict object scopes as much as possible.

With unmanaged resources such as a file handle, window handles (HWND), database connections, or network connection, it is a different problem. These require explicit cleanup. The garbage collector doesn't know how or when to do it. An object that encapsulates an unmanaged resource requires a public Dispose method to clean up the unmanaged resource, which has to be called when users of the object are finished with it. This can catch out the programmer who has become less used to thinking in terms of disposal of resources after use. Often, a NET object that contains references to unmanaged resources requires the use of the object's Dispose method, via either an explicit or implicit disposal, in order to clean up these references. Any object that encapsulates an unmanaged resource must allow implicit disposal, using the Finalize method, which is a second line of defense against a leak if the programmer fails to call Dispose. This should free any external resources that the object owns and has held onto.

Calling functions in unmanaged code that return data from .NET can cause problems because it is not always clear who owns the memory and how it should be disposed. When, for example, interacting with a COM object that is passing arrays to .NET, it is important to Marshal the array onto the managed heap and dispose of the original with the Marshal.FreeCoTaskMem method.

When leaks occur in .NET applications, they can often cause more grief than in an unmanaged application. In the latter case, the programmers are alert for all dangers of failing to dispose of resources. In a .NET application, the wonderful feeling that it is all done for you can blind you to those occasional times when you need to manage memory resources explicitly.

I'd be fascinated to hear of other types of memory problem that can happen in a managed application.

Cheers,
Laila

We've already had some emails sent to us with a gentle reproach that we are not giving enough coverage to the F# language (F for Functional). F#, if you're not familiar with it, is likely one of the great pleasures of Visual Studio 2010. It started as an open research project that adapts the classic functional programming language ML, via OCAML and Haskell, to enable it to use NET assemblies. It is now out as a fully-supported language in Visual Studio 2010 Beta1, and as a plug-in for as a fully-supported language in CTP Update for Visual Studio 2008 in the form of the F# May 2009 CTP (MSI, ZIP).

Anyone who has used ML, Haskell or Caml will recognize F#. Even Phil Factor was young when ML appeared at the University of Edinburgh in 1973. Functional programming is quite unlike the more common imperative model. It is oriented around expressions, as it consists of functions operating on simple data structures and avoids the use of variables (as generally understood) or operations with side effects. This ensures that the same function with the same input will always produce the same output (deterministic). Functional programming has evolved in parallel with imperative programming (e.g. Pascal, C#) and declarative programming (e.g. SQL). It is the ideal language for slow typists. Algorithms are cleanly expressed without having to wrap stuff up in fluff such as class definitions, and without having to type everything, since the system does that automatically for you.

It has survived the lean years by being the ideal advanced teaching language but has been used commercially for a variety of purposes that rely on advanced analysis techniques, particularly in financial systems. It has also been adopted by many as the ideal language for writing compilers. By adapting OCAML, a dialect of ML with Objects, to support .NET types and objects, Don Syme and his team at Microsoft has necessarily compromised the purity of ML by giving support for NET types and objects, using an imperative object-oriented style of programming.

Please don't get the impression that F# is an academic language. It is a well-proven, general-purpose language with a power that is equal to C#, but which is fundamentally designed for parallel processing. It is economical too. A script to manipulate an Excel spreadsheet or copy a database via SMO is even shorter than the C# equivalent. What is more, it is interactive, and adopts a programming paradigm that is surprisingly intuitive for any SQL Developer, with its built-in support for tuples, record and lists. It is in other senses quite unlike the more common languages as it is both functional and imperative. You can build F# applications for .NET 2.0/3.0 and 3.5 and 4.0

The classic textbook on ML is Lawrence Paulson's ML for the working programmer. The title was, one suspects, originally an ironic comment on its practicality for commercial use, rather than the theoretician in logic. The title no longer seems ironic, since, with its descendent F#, we have a tool that is of immediate use for any .NET programmer. We'd love to publish some articles on F# by experienced 'working programmers'.

Cheers,

Laila

One of the most interesting throw-away lines at the recent TechEd show was that Entity Framework was about four times slower than LINQ for SQL, because Entity Framework was a generic solution that could be used with a variety of data stores, and wasn't optimized for SQL Server.

This set us thinking about what the basis on which such statements are made.

LINQ-to-SQL generated a lot of interest from users but "isn't getting the love" at Microsoft because it was always seen as a stop-gap. By contrast, Entity Framework is the apple of the Microsoft eye, but has never received much affection from the users. What most programmers wanted was a simple abstraction layer to allow to them to focus on the 'application domain' and not have to immerse themselves too deeply in the arcane workings of relational databases. The project to provide this simple layer has been fumbled, mainly because there isn't a clear solution for all common application architectures.

The trouble is that abstraction layers come at a cost: they make applications run slower, but how much slower? The message from Microsoft and the MVPs about the architecture of data-driven applications seems confused, possibly because it isn't easy to quantify this performance cost.

All attempts at abstraction, including LINQ, seem to result in a solution that is considerably slower than that which can be achieved through intelligent use of SQLDataReader, or a simple interface based on SQLClient and stored procedures. However, performance comparisons between the various architectures seem to be either anecdotal or dependant on too many variables. Comparisons between LINQ and EF, for example, vary wildly according to whether LINQ queries are pre-compiled, and whether the SQL Query plan is cached and reused. Brian Dawson's benchmarks of Entity Framework against SQLClient and LINQ solutions, which are the most widely-known, illustrate the range of factors that can change the results of timings. For example, he was able to show a 28% performance gain in EFs performance by merely pre-generating the views!

It is extraordinarily difficult to apply a benchmark to the various ORM solutions in a fair and consistent way, but that doesn't mean that the community of users shouldn't be doing it. I'd like to see a production-size application being used, with the full range of timing criteria being decided 'up-front'. In other words, we need to decide before we start on what timings are fair measures of performance, and then stick to that definition. After all, you don't decide where the finishing post is after the horse-race is over, do you?

I'd be fascinated to know what you think about this. What are your experiences of nHibernate, Entity Framework and LINQ to SQL? Do you think that the obvious benefit of a full abstraction layer is really worth the performance cost of all the additional overhead?

Cheers,

Laila

Clearly, Microsoft is putting a lot of energy into Silverlight. The product, best described as a cross-browser, cross-platform implementation of the .NET Framework, is growing up. It gives the appearance of having been produced by a more progressive and less process-bound company than Microsoft: The excitement is still there. The scope of the product has increased greatly, and has grown out from its initial remit of providing a platform for rich interactive multi-media web-applications within a browser add-in. It is becoming able to support the entire spectrum of applications from games, through business to the scientific, on a PC or Mac.

The Silverlight 3 features that were announced for the Beta at MIX09 have wide implications. Probably the most significant introduction into Silverlight 3 are the .NET RIA services. These are designed to bring together the ASP.NET and Silverlight platforms by providing a pattern to write application logic that runs on the mid-tier and controls access to data for queries, changes and custom operations. It also makes the bread-and-butter tasks of data validation, authentication and roles much easier by integrating with Silverlight components on the client and also with ASP.NET on the mid-tier. If you add to this the new Data controls, including a DataPager and a DataForm, and you have a simple, versatile, platform that you can confidently use in client-server applications.

Silverlight applications are likely to look better. Cleartype support will be in, there are a number of new controls, some glorious Pixel shaders, and the perspective 3D support is much better. You'd have thought that we'd be immune to the 'ponytail' features offered by animation easing, offering bouncy and wobbly menus and buttons, but somehow, Silverlight encourages experimentation, and it is nice to know that you can use these features when you want.

Clearly, Microsoft have been startled by the popularity of Adobe Air, and so the 'out of browser' feature is an important way of outflanking it within the Windows Platform. A Silverlight 3 application can run as a desktop application, even on a Mac if it subscribes to a series of APIs and defines the application in the manifest. If you combine this with the 'Network Status' API that tells you when the network isn't or is available, you have an easy way of creating an 'occasionally-connected' application that can adapt to online/offline working.

Although you cannot compare such different technologies as Adobe Air and Silverlight, it will be fascinating to see if developers succeed in using Silverlight to create applications that are as compelling as some of the Adobe Air applications. This will be the acid test. In the meantime, the Silverlight 3 Beta is well worth a look.

Cheers,

Laila