.NET Finalizer Memory Leak: Debugging with sos.dll in Visual Studio

Normally I write about issues that only manifest themselves in production environment, issues that you can’t really reproduce in a controlled dev environment every time you perform a certain action. In those cases you need to use tools like windbg to gather dumps and do post-mortem debugging.

Windbg works really well for those types of issues, but it has its shortcomings since it is not really a managed debugger so it is much harder to set breakpoints in .NET code or step through code, or even inspect objects in a visual way like you can in a managed debugger like Visual Studio.

Visual Studio on the other hand doesn’t allow you to do post-mortem debugging the same way windbg does, and there is no easy way to view information about the domains loaded in the process or to view information about the objects on the .net heaps.

My colleague had an issue that was pretty easily reproducible but he needed both worlds, i.e. stepping through code to a specific point, looking at the objects on the stack visually and at the same time he needed to view the contents on the heap, so he resorted to having two debuggers attached, visual studio debugging managed, and windbg debugging native with sos to view the managed heap. That is pretty nasty, and there is a much easier way to combine these two worlds… debugging with sos in Visual Studio.

To illustrate how it works I’m using a sample from Ingo Rammer.

If you want to follow along you can download the sample code here.

The link is no longer available - I will move this to a repo and update this.

The specific sample I am using is FinalizerProblem which is basically the win forms equivalent of my post on Unblock my Finalizer.

Problem description

When I click on the button “Do Work” it creates a number of instances of the class MyBusinessObject. Even though I know that the objects should no longer be referenced after that they don’t appear to go away even if I invoke a GC.Collect() with GC.WaitForPendingFinalizers(). Why are my objects not released?

Debugging the issue

In this case we could easily attach windbg and use sos.dll per my post above to figure out that the reason these objects are sticking around is because of a blocked finalizer, but I will use Visual Studio in order to show you how to load up sos in it and run sos commands.

Step 1: Enable Native Debugging for the project

In order to load an extension like SOS.dll you have to be debugging in native mode, so before starting the debugger go into Project/Properties/Debug on the context menu for the project and check the box for Enable unmanaged code debugging.

Step 2: Debug and Break

Debug the problem as you normally would in Visual Studio until you have reproduced the issue (i.e. in this case click on “Do Work” to instantiate the objects, followed by “Run GC” to perform the garbage collection.

Break into the process (Debug menu/Break All)

Step 3: Load sos

In order to load sos.dll you have to open up the Immediate Window (Debug/Windows/Immediate or Ctrl+D, I) and type

.load C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\sos.dll

This should yield the response

extension C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\sos.dll loaded

Step 4: Debug with sos

Now we are ready to debug with sos.dll.

There are a few rules here

1. You can not run native commands like kb, dc etc.
2. You can not run ~* e which means you cant run ~* e !clrstack to see the stacks on all threads

You can however run all the commands in sos.dll like !dumpdomain, !dumpheap etc.

To switch the thread context for thread specific commands like !clrstack and !dumpstackobjects you can open the threads window (debug/windows/threads) and double click the thread you want to switch to. I will show an example of that later.

The first thing we want to do is find our objects on the heap

!dumpheap -type MyBusinessObject
PDB symbol for mscorwks.dll not loaded
 Address       MT     Size
027437e4 01d6683c       12
02743830 01d6683c       12
0274387c 01d6683c       12
...
02747d6c 01d6683c       12
02747db8 01d6683c       12
02747e04 01d6683c       12
02747e50 01d6683c       12
02747e9c 01d6683c       12
02747ee8 01d6683c       12
total 30 objects
Statistics:
      MT    Count    TotalSize Class Name
01d6683c       30          360 FinalizerProblem.MyBusinessObject
Total 30 objects

Then we can grab one of those objects and run !gcroot on it to find out why it is still around

!gcroot 02747d6c
Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Error during command: warning! Extension is using a feature which Visual Studio does not implement.

Scan Thread 7092 OSTHread 1bb4
Scan Thread 6864 OSTHread 1ad0
Finalizer queue:Root:02747d6c(FinalizerProblem.MyBusinessObject)

In this case it is rooted in the finalizer queue which means it is waiting to be finalized and if we look at the finalizequeue we can see that we have 69 objects that are waiting to be finalized so the question that remains is why we aren’t finalizing them…

!finalizequeue
SyncBlocks to be cleaned up: 0
MTA Interfaces to be released: 0
STA Interfaces to be released: 0
----------------------------------
generation 0 has 0 finalizable objects (002906d0->002906d0)
generation 1 has 36 finalizable objects (00290640->002906d0)
generation 2 has 0 finalizable objects (00290640->00290640)
Ready for finalization 69 objects (002906d0->002907e4)
Statistics:
      MT    Count    TotalSize Class Name
7b47f8f8        1           20 System.Windows.Forms.ApplicationContext
...
7910b694       10          160 System.WeakReference
7b47ff4c        4          224 System.Windows.Forms.Control+ControlNativeWindow
01d6683c       22          264 FinalizerProblem.MyBusinessObject
01d65a54        1          332 FinalizerProblem.Form1
7b4827e8        2          336 System.Windows.Forms.Button
7ae78e7c        8          352 System.Drawing.BufferedGraphics
...
Total 105 objects

If we were debugging in windbg, the next natural step would have been to run !threads, figure out which one was the finalizer and look at what is is running.

Since we are in Visual Studio instead we can open the threads window, navigate to the thread that is performing Finalization and look at what it is doing. If you can’t determine that by the function that is currently on the top of the user-code part of the call stack you can still identify it with !threads.

The cool thing here is that we jump straight into the code, where it is blocking and can use everything we are used to in Visual Studio like the watch and locals window or stepping in code for example, but without loading up sos.dll we would not have been able to track down why our MyBusinessObject instances were still around.

Final words

I know many people are a bit hesitant to start with windbg since it means you need to learn a whole new tool set and perfectly honestly windbg is not as “beginner friendly” as for example visual studio is, then again, its meant to be used mostly for post-mortem debugging which is not exactly an everyday task for most people.

Hopefully though with the info above you can still begin to use sos.dll in the cozy and familiar visual studio debugging environment.

Speaking of making things a bit more user-friendly and visual, Ingo Rammer who wrote the demo has developed a tool called SOSAssist which is kind of a GUI interface to sos.dll.

Edit 2020: this tool is also no longer available

Until next time,

Tess