内存泄漏及检测（英文）

来源：互联网发布：河南壁虎网络编辑：程序博客网时间：2024/05/01 14:03

1. Introduction

1.1 What we are dealing with

Before finding memory leak, we should have an overview of what we are dealing with. Memory leak on Windows platform contains following aspects:

§ Memory leak on CRT, Global, Heap, Virtual, COM, …

§ Handles leak when using GDI, Kernel, Common control, winsock, winhttp, advapi, …

Many memory leak detection tools also detect errors when using memory, which include memory conflict, memory overrun, etc.

1.2 Leak detection tools

Figure 1 Leak detection tools

1.3 Leak category

Figure 2 Leak category

1.4 Work flow

Although we can use different tools to detect memory leak, but there are some common steps to do.

Figure 3 Leak detection work flow

2. Tools

2.1 CRT Heap Check Routine

Advantage:

Simple

Speedy

Low cost

Disadvantage:

Can only detect memory leaks allocated by new and malloc

Can only run under Visual Studio IDE

No call stack information, hard to find the leak especially when multi-allocation in same place

Detail:

Write the following code in the firstly included .h file

#define _CRTDBG_MAP_ALLOC

#include <stdlib.h>

#include <crtdbg.h>

Write the following code at last to print the result to Debug window

_CrtDumpMemoryLeaks();

Demo:

Figure 4 CRT Heap Check Routing

Notice:

If _CRTDBG_MAP_ALLOC is not defined before including crtdbg.h, the output imformation will not contain the leak file.

Analyse:

The output string in Debug window between Dumping objects and Object dump complete has the following meanings (marks are in the window above):

(1) Leak file and line number

(2) Memory allocation sequence

(3) Type of leak block

a. normal block allocated by your code

b. client block for MFC destructor

c. CRT block CRT allocated

d. free block freed block (will not appear in the output of the leak dump)

e. ignore block programmer can claim blocks ignored by CRT check routine

(4) The leak address

(5) ASCII and Binary version of leak content

2.2 Visual Leak Detector

This is a wrap for CRT heap check routines.

Advantage:

Same with CRT check routines, but provides call stack information

Disadvantage:

Same with CRT check routines, but provides call stack information

Detail:

§ Download the zip from CodeProject

§ Copy the VLD library (*.lib) files to your Visual C++ installation's "lib" subdirectory.

§ Copy the VLD header files (vld.h and vldapi.h) to your Visual C++ installation's "include" subdirectory.

§ In the source file containing your program's main entry point, include the vld.h header file. It's best, but not absolutely required, to include this header before any other header files, except for stdafx.h. If the source file, include stdafx.h, then vld.h should be included after it.

§ If you are running Windows 2000 or earlier, then you will need to copy dbghelp.dll to the directory where the executable being debugged resides.

§ Build the debug version of your project.

Demo:

Figure 5 Visual Leak Detector

Notice:

The library works only when defines _DEBUG, so the release version will not be effected even if including <vld.h>

Analyse:

The stack trace information and entire leaked data content was print to the output window in VC IDE.

2.3 Memory Validator

Memory Validator is a speedy and powerful tool for memory error detecting. It is able to monitor the following kinds of resources :

Figure 6 Memory Validator – Setting Dialog 1

Advantage

Powerful and handy

Disadvantage

Sometimes not accurate

Demo

§ Settings

o Set to monitor one specific resource one time on large scale software. See the collect setting above, and the display setting below.

Figure 7 Memory Validator - Setting Dialog 2

o Set to monitor only some module of the large scale software

Figure 8 Memory Validator - Setting Dialog 3

§ Watermark

o Check the leak during the two specific scenarios. We can take a snapshot of memory usage before doing suspicious operation and then take another snapshot when finishing the operation. Check the Compared result data provided by the tool.

Figure 9 Memory Validator – Adding watermark

Figure 10 Memory Validator - All the allocation between two watermarks

§ Objects : Tells us which kind and how many objects existed in memory or leaked after exiting the program. Sort by Number is recommended.

Figure 11 Memory Validator - Objects

§ Filters : Use filters to select the data you are interested in.

Figure 12 Memory Validator - Filters

§ For icons and other view usage, please check the help manual of Memory Validator

2.4 BoundsChecker

BoundsChecker is a memory detect tool developed by Compuware. Like memory validator, it uses hook to detect memory leaks. They are almost the same.

Here is some pictures taken from BoundsChecker

Figure 13 Setting Dialog

Figure 14 Boundschecker - Summary Dialog

Figure 15 Leak Detail

2.5 UMDH & LeakExplorer

UMDH is a tool provide by Microsoft. Through comparing two memory dump, UMDH tells us how much memory is allocated between the dump. Since not all the memory allocation is leak, so we should do the same operation between the two dump, and get the third dump. If memory increase steadily in the same context, it is quite suspicious. Since there are so much allocations information from kernel, and the output of UMDH is txt file, we have great difficulty in finding leaks. Autocad team developed a tool to analyze the output txt files by UMDH, this is called LeakExplorer.

Advantage

Accurate

Speedy

Not using hook, no impact on the program to be tested

Disadvantage

Not handy, Cost a lot of time to find the leak

Demo

UMDH.wmv

Appendix: Background

1.1 Layered Memory Management in Win32

Figure 16 Memory Layer in Windows NT based system

Local, Global memory API are provided by previously released windows api. There used to be a global heap for all the applications, which shares it concurrently, and multiple, private local heaps, one for each application. After win 2000, there is no difference between local heap and global heap. Actually there is no global heap now.

1.2 Memory allocation pair

Allocator

Deallocator

CRT

new

delete

malloc

Free

Virtual Memory

VirtualAlloc

VirtualFree

Heap

HeapAlloc

HeapFree

Global

GlobalAlloc

GlobalFree

Local

LocalAlloc

LocalFree

Com

CoTaskMemAlloc

CoTaskMemFree

SysAllocString*

SysFreeString

SafeArrayCreate

SafeArrayDestroy

Figure 17 Memory allocation pair

Most memory leak happens because of misusing the pair of allocator and free functions. For example, use free() to new(), or use delete() to malloc(), or forget to use free function after allocating memory. But there is an exception, that is Global heap and local heap memory api, as described above.

There two kinds of heaps for each application now. One is called default heap, the CRT memory API and Local, Global API allocate memory on it by default. Besides, we can also use HeapCreate to generate a dynamic heap. By using HeapAlloc and HeapFree, we can allocate and free memory on the dynamic heap. The most important difference between these two kinds of heaps is that the default heap is thread-safe. That is to say, we have to do synchronizations for the heap created by HeapCreate.

Since we are familiar with the CRT memory management, I will skip that. You can get the leak scenarios for CRT leaks in appendix. Those are tutorials from memory validator, which will be introduced in chapter 2.

1.3 Memory management in COM

1.3.1 COM pointer

All COM components use reference counting technology, they all have AddRef() to increase the count number, and Release() to decrease the count number. If the count number falls to 0, Release() will destroy the COM point and free its memory. So we should use a COM pointer like this.

pCOMNew = pCOM;

pCOM->AddRef();

…

If ( NULL != pCOM)

{

pCOM->Release();

pCOM = NULL;

}

If ( NULL != pCOMNEW)

{

pCOMNEW->Release()

pCOMNEW = NULL;

}

In most cases we don’t need to write such low level code, for example, we can use macro to replace code of xxx->Release(); xxx = NULL (like SAFEDELETE in directx). Or, we can use a simple smart pointer like this:

template <class P>

class CSmartPtr

{

P* p;

public:

CSmartPtr(P* ptr) { p = ptr; if(p) p->AddRef(); }

~CSmartPtr() { if(p) p->Release(); }

};

And for ATL : CComPtr and CComQIPtr, which are also smart pointer. Just remember when these COM objects goes out of their scope, they will destroy automatically, we needn’t destroy them again. But a lot of memory detection tools report these are memory leaks.

1.3.2 BSTR

The string operation leaks memory easily, not only because the string often used as a parameter passed everywhere, which would be easily forgotten to free, but also because not understand the library we use.

BSTR is null-terminated, two-bytes string, so its null-termination also has 2 bytes. The name of BSTR means “Basic STRing”, it can be used in Visual Basic and java.lang.string, and any other language support Automation. We usually use CComBSTR in ATL to manipulate the string. I mention it here just because it use SysAllocString and SysFreeString to allocate and free the memory, not malloc and free in CRT. So when there is a memory leak, we can’t expect the CRT memory check routine to find it.

For more information about CComBSTR, you can see Chapter 5, Data Conversion Classes in <Inside ATL>.

1.3.3 SafeArray

Like BSTR, SafeArray can also be recognized in VB and Java. We use SafeArrayCreate to allocate memory, and use SafeArrayDestroy to free the memory. In ATL, there is an alternative way: CComSafeArray. It’s a template class of SafeArray. They need special attention also because SafeArrayCreate and SafeArrayDestory are not CRT functions. Some memory detection tool will miss the real leaks, and some tools will report a correct scenario as memory leak.

1.3.4 CoTaskMemAlloc and CoTaskMemFree

As mentioned in 1.3.2, there are two kinds of heaps. The default heap meets our needs in most cases, but for COM, we usually allocate memory in one process, and free memory in another process. Clearly, a conventional heap has no way to handle this, so CoTaskMemAlloc and CoTaskMemFree is used instead. The automatically generated proxy-stub code can properly allocate and free memory across COM boundaries. COM's remoting infrastructure does all the dirty work needed to create the illusion of a single heap that spans processes.

In ATL, there is an interface called IAtlMemMgr, which has the following members:

__interface IAtlMemMgr {

public:

void* Allocate( size_t nBytes ) ;

void Free( void* p ) ;

void* Reallocate( void* p, size_t nBytes ) ;

size_t GetSize( void* p ) ;

};

We can implement it for special usage. ATL also has 5 concrete class implemented IAtlMemMgr, they can be used under different situation, as described in the following table, the left column is classes in ATL, the right column is the low level functions each class used.

Memory Manager Class

Heap Functions Used

CComHeap

CoTaskMemAlloc,

CoTaskMemFree,

CoTaskMemRealloc,

IMalloc::GetSize

CCRTHeap

malloc,

free,

realloc

CLocalHeap

LocalAlloc,

LocalFree,

LocalReAlloc,

LocalSize

CGlobalHeap

GlobalAlloc,

GlobalFree,

GlobalReAlloc,

GlobalSize

CWin32Heap

HeapAlloc,

HeapFree,

HeapReAlloc,

HeapSize

1.3.5 COM objects’ Reference count

If we found a lot leak within one class, we can infer that one instance of that class wasn’t released, and its destructor wasn’t invoked at all. Usually this can be caused by three scenarios:

§ Object pointer overwriting

§ Forgetting invoke delete operator on the pointer passed by another function

§ Incorrect com object reference count

The first two scenario is easy to detect by using memory check tools, but the third one is hard to locate where has gone wrong. Let’s see why this is difficult. The reference count itself is simple, we only should make sure that AddRef() and Release() matches on a non-smart com pointer, just like new() and delete().

There are 3 kinds of smart pointer in COM, which manipulate reference count automatically.

§ Smart pointer provided by ATL template ( CComPtr and CComQIPtr )

§ Smart pointer provided by compiler ( _com_ptr_t )

§ Smart pointer defined by ourselves

Let’s pick smart pointer from ATL template, other smart pointers are the same. CComPtr and CComQIPtr are subclass of CComPtrBase, its release method is defined like this:

// Release the interface and set to NULL

void Release() throw()

{

T* pTemp = p;

if (pTemp)

{

p = NULL;

pTemp->Release();

}

Usually we needn’t call release on a smart pointer, because it can be released automatically. But if we do call release on a class member, let’s see what happens. After calling release, the class member is assigned to be NULL. But actually its reference count may still greater than 0. When the object goes out of scope, the auto-release action will not perform, since the pointer is NULL. Finally the COM object will have no change to release because its reference count is greater than 0.

We also should pay great attention on using non-smart pointer and smart point together. “=” operator is only overloaded for assignments between two smart pointers. When the left side of “=” is just an interface, AddRef() will not be called. We should call AddRef() and Release() manually in this case.

1.4 Handle Management

Like memory allocation pairs, handles also has their creator and destroy function pair. We should use them together and make them match.

Appendix: CRT Leak scenario

n incorrect usage: double delete

1284 : char *ptr;

1285 :

1286 : ptr = new char [10];

1287 : delete [] ptr; // correct cleanup

1288 : delete [] ptr; // incorrect doublecleanup

1289 : }

n incorrect usage: double free

1082 :

1083 : char *ptr;

1084 :

1085 : ptr = (char *)malloc(10);

1086 : free(ptr); // correct cleanup

1087 : free(ptr); // incorrect doublecleanup

1088 : }

n free the wrong location

1107 : // allocate some memory then free the wrong location, inside the block

1108 :

1109 : ptr = (char *)malloc(3);

1110 : for(i = 1; i < 3; i++)

1111 : free(ptr + i); // incorrect cleanup

1112 :

1113 : // correct cleanup

1114 :

1115 : free(ptr); // correct

n delete the wrong location

1314 : // allocate some memory then free the wrong location, inside the block

1315 :

1316 : ptr = new char [3];

1317 : for(i = 1; i < 3; i++)

1318 : {

1319 : delete (ptr + i); // incorrect cleanup

1320 : delete [] (ptr + i); // incorrect cleanup

1321 : }

n realloc the wrong location

1367 : ptr = (char *)malloc(10);

1368 : p = (char *)realloc(ptr + 1, 20); // incorrect realloc

1369 : p = (char *)realloc(ptr - 1, 20); // incorrect realloc

1370 : ptr = (char *)realloc(ptr, 20); // correct realloc

n delete malloc/calloc/relloc’d memory

4708 : char *ptr;

4709 :

4710 : ptr = (char *)malloc(1000);

4711 : delete ptr; // incorrect (it will work with Visual Studio 6.0 debug, but thats relying on a side-effect caused by new wrapping free, if they change the implementation, bang codes the code!)

4712 : delete [] ptr; // incorrect (it will work with Visual Studio 6.0 debug, but thats relying on a side-effect caused by new wrapping free, if they change the implementation, bang codes the code!)

4713 : free(ptr); // correct

n free/relloc new’d memory

4764 : char *ptr;

4765 :

4766 : ptr = new char [1000];

4767 : free(ptr); // incorrect (it will work with Visual Studio 6.0 debug, but thats relying on a side-effect caused by new wrapping free, if they change the implementation, bang codes the code!)

4768 : delete [] ptr; // correct

n HeapReAlloc and HeapFree

n Memory overwritten

1502 : char *p;

1503 :

1504 : p = new char [tmd.sizeToAllocate];

1505 : p[tmd.sizeToAllocate] = '/0';

n Memory underwritten

1479 : char *p;

1480 :

1481 : p = new char [tmd.sizeToAllocate];

1482 : *(p - 1) = '/0';

n Uninitialized on stack/heap

n Heap memory overrun

1923 : // warning, the very nature of this overrun may cause this program

1924 : // to crash since it damages the CRT heap

1925 :

1926 : char *memory;

1927 :

1928 : memory = new char[100];

1929 : memcpy(memory + 3, memory, 100);

n Heap memory underrun

1952 : // warning, the very nature of this overrun may cause this program

1953 : // to crash since it damages the CRT heap

1954 :

1955 : char *memory;

1956 :

1957 : memory = new char[100];

1958 : memcpy(memory - 3, memory, 100);

1959 :

1960 : // I'll allow this to leak, since if I try to free it the damaged stack will

1961 : // be found, and if I don't try to free it, the program may not crash (for a while)

1962 : }

n Stack memory overrun

n Stack memory underrun

n Strcpy and wcscpy

5081 : char *charBuffer = new char[20];

5082 : wchar_t *wcharBuffer = new wchar_t[20];

5085 : char *charBufferSource = new char[20];

5086 : wchar_t *wcharBufferSource = new wchar_t[20];

5090 : charBufferSource[0] = '/0';

5091 : wcharBufferSource[0] = _T('/0');

5092: strcpy(charBufferSource, "some test text56789");

5093: wcscpy(wcharBufferSource, L"some test text56789");

5112: // overwrite beginning of buffer

5113: strcpy(&charBuffer[-4], charBufferSource);

5114: wcscpy(&wcharBuffer[-4], wcharBufferSource);

5115: strncpy(&charBuffer[-4], charBufferSource, 20);

5116: wcsncpy(&wcharBuffer[-4], wcharBufferSource, 20);

5117:

5118: // overwrite end of buffer

5119: strcpy(&charBuffer[10], charBufferSource);

5120: wcscpy(&wcharBuffer[10], wcharBufferSource);

5121: strncpy(&charBuffer[10], charBufferSource, 20);

5122: wcsncpy(&wcharBuffer[10], wcharBufferSource, 20);

…

5146 : // at this point, deleting the data may cause the program to crash (as the heap has

5147 : // been corrupted), if it doesn't the program will most likely crash later on

5148 :

5149 : delete [] charBuffer;

5150 : delete [] wcharBuffer;

n Forget deleting the object in an array

1863 : for(i = 0; i < 10; i++)

1864 : {

1865 : char text[10];

1866 :

1867 : sprintf(text, "%d", i);

1868 : array[i] = new CString(text);

1869 : }

1870 :

1871 : // cleanup the array, but leak the contents

1872 :

1873 : delete [] array;

n Call functions on a deleted object