There are user interfaces that just seem to work naturally, such as Spreadsheets, email clients, or word processors. Even some development tools have reached a shared maturity. What about memory profilers? How can you represent the way that memory is being used in an application? How can you make it simple to use, yet sophisticated enough to track down complex memory problems?
“What would I like to see in a memory profiler? How about a big green button that says ‘find my memory leak’ that you just press and it takes care of the rest!”
There is no getting away from the fact that memory profiling can be difficult and not a task that people willingly start or approach with great enthusiasm. The request for the ‘big green button’ is just wishful thinking of course but it certainly would be a killer feature!
The reason for the negativity is simple – memory profilers do not have a great reputation generally in terms of their usability or have the features needed to find and fix a memory problem quickly. Memory leaks can almost be devious in their manifestation, popping up unexpectedly or only under certain conditions making them difficult to investigate. Some people may go so far as to simply ignore a memory problem, restarting their application or rebooting a service when the memory climbs beyond acceptable levels. Cheaters! In contrast to performance profilers where a list of performance ‘hot spots’ can be flagged immediately as candidates for optimization, a memory issue can only be truly identified by the user – a magic wand solution is almost out of the question.
Memory profiling instead requires more of a Sherlock Holmes approach - identifying and disregarding suspects until the culprit is found with their hand in the proverbial cookie jar. By developing a memory profiler fit for the job however we can equip Sherlock Holmes with the latest in crime scene investigation tools to give him the added clues to make solving the problem much less stressful and frustrating. Over the last few months I’ve spent a large amount of time conducting research and testing designs in preparation for the release of the next version of ANTS Memory Profiler and one thing became very clear – designing an application that makes memory profiling easier wasn’t going to be a walk in the park.
A problem with two sides
During the time spent talking to people who were specifically trying to track down a memory leak it became obvious that there were two main obstacles to overcome. The first is identifying objects of interest which may be leaking amongst a large amount of object ‘noise’. Certain memory patterns were commonly responsible for a number of the memory issues an application would have – patterns we could potentially automatically check for and highlight to the user…
Once an object has been found that may be involved in a memory leak, the second step is to then find out why that object isn’t being garbage collected. It was in this step that people were having the most trouble with current memory profilers. It became clear that having an adequate representation of how memory is being used in an application was an area where we could make giant leaps forward from every other profiler out there (but more on that later).
Problem 1: Trying to solve the signal to noise ratio problem
"Eliminate all other factors, and the one which remains must be the truth."
Sherlock Holmes
Even a relatively small .NET application can easily generate a few million objects and herein lies the problem. The feeling of trying to find a needle in a giant haystack is common when it comes to memory profiling. With so much other information potentially masking the problem how do we tell what’s worth examining further and what can be safely discarded? The answer is to provide people with meaningful shortcuts of what to look at first, what to disregard and critically – to present that information simply.
Identify specific memory patterns:
Memory leaks can typically have a specific number of common causes and as such we can use memory heuristics to check for particular patterns. Being able to ignore which objects are less likely to be involved in a memory problem can be as powerful a filter mechanism as finding objects which are involved. For the new memory profiler we decided that we would provide a set of out of the box filters that can decrease the amount of noise often involved in trying to find a memory leak by providing a set of filters that a user can immediately apply at the click of a button (or check box in this case). For example, applying the “Dispose was called but are still in memory” filter will show a filtered list of classes with objects that potentially should have been garbage collected but are still in memory. Used in conjunction with the other filters can help narrow the search down further until the most likely candidates remain.
Figure 1: The Customizable Filter Panel to Help a User Rapidly Exclude Objects
Tailor filtering to a project:
To increase the power of this mechanism is the ability to tailor the filters to a specific project. Take the scenario where memory always seems to increase due to a certain operation that would have to involve a specific set of objects created at that point in time. With the new memory profiler filter feature you can show objects most recently created that are only referenced via a particular object type as shown below in figure 2. The user can build their own filter statements based on their own investigations so the case against a particular culprit builds. Adding (or removing objects) from the filter will allow a project specific memory leak finder that will be extremely powerful and useful to the user not only in the short term but over the course of the current development cycle and beyond.
Figure 2: The object type means a user can tailor the filtering to specific objects in their own project. Take that memory leak!
Figure 3: With the object type added simply apply the filter to see which objects are remaining to investigate. Powerful stuff indeed.
Increasing the visibility of change between snapshots:
Memory profiling also works primarily in taking ‘snapshots’ of memory at key points in time and comparing changes in the size or number of instances of particular classes. A list of numbers can be processed and scanned quicker when supplemented by a number of small visual cues than when numbers are shown just by themselves. Tagging these numbers with performance indicators might sound like a minor and obvious improvement but they can make a big difference in spotting outliers amongst the noise as shown in figure 4.
Figure 4: Class list with visual cues that allow rapid scanning of the list
The diff columns show if a particular class has increased or decreased in size or instance count. The up/down arrows immediately draw your eye to patterns amongst the numbers to the point where you almost ignore the numbers as you interpret the patterns. The ‘Live Size (bytes)’ column also indicates that although it currently isn’t sorted by the values, what would happen if it became sorted by that column.
Problem 2: Using the right representation to solve the problem
The natural representation of memory is in graph form. Looking at the most popular .NET memory profilers currently on the market one thing becomes clear. None represent memory in the format which is naturally afforded by the problem. As a consequence the underlying structure of memory is not properly represented in the UI and does not directly map onto how people naturally think of memory. In the case of memory this is problematic as the solutions offered do not allow the rapid exploration and visualisation of the current state of memory. Memory is a set of objects connected to each other in a complicated array of relationships. Filtering down the number of objects and then creating a memory graph from what’s left is the best way of inspecting memory without being totally overwhelmed.
Figure 5: The ANTS Memory Profiler Object Reference Graph – a killer feature
With the rapid zoom and browse features of the graph as well as the clear identification of root objects, identifying the cause of objects to still be held in memory should be made dramatically easier and will certainly provide people with a much more powerful tool than they have had before in the .NET area when trying to find out the reason for leaked objects. Criminal objects beware, hiding just became a great deal tougher.
Conclusion
Attempting to develop a memory profiler that is both simple to pick up but sophisticated enough to do the job is no easy task. There are many other factors involved in the development of an easy to use profiler and indeed one of the key components will be the provision of online help and great tutorials to teach users about the strategies which work best amongst the other factors to consider. It’s a myth that people don’t read help documentation and any one who believes that has clearly never been involved in the development of a memory profiler.
Time will tell if we have achieved what we set out to – tame the beast that is memory profiling. One such way is to invite contributions from the .NET community on what we have done so far and ask for ideas about how to improve the alpha version we currently have available. We have started to release early access builds of the memory profiler and want your feedback to make it even better. You can download the latest build here:
Maybe you have some advice on tracking down memory leaks and you would like to share your experiences? Speak now or forever hold your peace…
