Who is this OutOfMemory guy and why does he make my process crash when I have plenty of memory left

To answer this question, there are a few concepts we need to discuss.

Working on a 32-bit system, you can address 4 GB of memory, out of this 2 GB is typically reserved to the Operating System and 2 GB are allowed for each user mode process, such as w3wp.exe (asp.net) for example. This memory is called virtual memory and the 2 GB’s are 2GB independently of how much RAM you have added to the system. The amount of RAM simply decides how much paging and swapping you will do, i.e. how fast memory access will be.

When a process allocates memory it does so by first reserving virtual memory and then committing memory out of this chunk (this is the memory actually used). The committed memory is called private bytes.

The virtual address space is used for a number of different items in the process such as:

• Dlls
• Native heaps (non .net heaps)
• Threads (each thread reserves 1 MB for the stack)
• .net heaps (for managed variables)
• .net loader heap (for assemblies and related structures)
• Virtual allocations made by com components

Virtual memory allocations are not necessarily (or very seldom) lined up nicely in memory. For example, dlls have preferred loading addresses so gaps are left between them, and virtual allocations that have been de-allocated will also leave gaps. This means that even though you can address 2 GB of virtual memory you may not be able to use it all since when enough memory is used, the memory will look somewhat like a Swiss cheese and your plug might have a big enough hole to fit in.

This is what happens when you get an OutOfMemory exception.

I will likely talk more about .net memory management later, but for now I’m going to make it very brief since there are several very good blogs about this such as Maonis CLR Performance blog and YunJins blog.

In the .net framework the garbage collector which is our memory manager reserves virtual memory in heaps. Once these heaps are created and we create a new instance of a .net object this object is stored in these heap segments and memory is committed.

In the .net framework 1.1 (server version) the regular .net heaps are created in 64 MB segments. (variations occur if you have 8+ processors or have changed the settings manually but I will ignore this for now and talk about the common case)

These 64 MB segments need to be allocated in one big chunk, you can’t allocate 32 MB here and 32 MB there, so when you have filled up one of these 64 MB segments and a new segment needs to be created you will need to find an empty gap of 64 MB or more in the 2 GB memory space. If such a gap does not exist you will get an out of memory exception.

These OutOfMemory exceptions are generally not fatal but still bad as they leave the process in an unstable state. However, when the garbage collector needs to do a collection it requires space for internal structures in order to do this collection, and the more memory and objects that you use, the bigger these structures are. If the garbage collector does not have enough space for its collections you will get a fatal out of memory exception and the process will die.

As you may have gathered from the above the reserved memory (virtual bytes in performance monitor) will be much larger than the committed bytes (private bytes in performance monitor). Usually the difference will be somewhere around 500 MB when memory usage is high, and at about 1.2-1.4 GB or so virtual bytes it starts getting very hard to find these 64 MB holes which means that a normal .net process will start getting OutOfMemory exceptions at around 800 MB.

Again, this value will differ depending on your application, the dlls that are loaded and memory allocation patterns of native com components etc.

Now you know why you get OutOfMemory exceptions even though you have a lot of RAM, the next task is to find out why you are using this much memory and I will spend some time on that in future blog posts.