The tools that check memory leak of python program

python的内存检测库
https://launchpad.net/meliae

meliae需要依赖的
http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/

Python对象引用查看库
http://mg.pov.lt/objgraph/

Garbage collection for python
http://arctrix.com/nas/python/gc/

Python垃圾回收算法描述
http://wiki.woodpecker.org.cn/moin/python_ref_circle_gc

Linux程序分析工具
http://www.valgrind.org/

如何使用valgrind来分析python程序:
——————————————————Misc/README.valgrind——————————————————
This document describes some caveats about the use of Valgrind with
Python. Valgrind is used periodically by Python developers to try
to ensure there are no memory leaks or invalid memory reads/writes.

If you don’t want to read about the details of using Valgrind, there
are still two things you must do to suppress the warnings. First,
you must use a suppressions file. One is supplied in
Misc/valgrind-python.supp. Second, you must do one of the following:

• Uncomment Py_USING_MEMORY_DEBUGGER in Objects/obmalloc.c,
  then rebuild Python
• Uncomment the lines in Misc/valgrind-python.supp that
  suppress the warnings for PyObject_Free and PyObject_Realloc

If you want to use Valgrind more effectively and catch even more
memory leaks, you will need to configure python –without-pymalloc.
PyMalloc allocates a few blocks in big chunks and most object
allocations don’t call malloc, they use chunks doled about by PyMalloc
from the big blocks. This means Valgrind can’t detect
many allocations (and frees), except for those that are forwarded
to the system malloc. Note: configuring python –without-pymalloc
makes Python run much slower, especially when running under Valgrind.
You may need to run the tests in batches under Valgrind to keep
the memory usage down to allow the tests to complete. It seems to take
about 5 times longer to run –without-pymalloc.

Apr 15, 2006:
test_ctypes causes Valgrind 3.1.1 to fail (crash).
test_socket_ssl should be skipped when running valgrind.
The reason is that it purposely uses uninitialized memory.
This causes many spurious warnings, so it’s easier to just skip it.

Details:

Python uses its own small-object allocation scheme on top of malloc,
called PyMalloc.

Valgrind may show some unexpected results when PyMalloc is used.
Starting with Python 2.3, PyMalloc is used by default. You can disable
PyMalloc when configuring python by adding the –without-pymalloc option.
If you disable PyMalloc, most of the information in this document and
the supplied suppressions file will not be useful. As discussed above,
disabling PyMalloc can catch more problems.

If you use valgrind on a default build of Python, you will see
many errors like:

    ==6399== Use of uninitialised value of size 4    ==6399== at 0x4A9BDE7E: PyObject_Free (obmalloc.c:711)    ==6399== by 0x4A9B8198: dictresize (dictobject.c:477)

These are expected and not a problem. Tim Peters explains
the situation:

    PyMalloc needs to know whether an arbitrary address is one     that's managed by it, or is managed by the system malloc.    The current scheme allows this to be determined in constant    time, regardless of how many memory areas are under pymalloc's    control.    The memory pymalloc manages itself is in one or more "arenas",    each a large contiguous memory area obtained from malloc.    The base address of each arena is saved by pymalloc    in a vector.  Each arena is carved into "pools", and a field at    the start of each pool contains the index of that pool's arena's    base address in that vector.    Given an arbitrary address, pymalloc computes the pool base    address corresponding to it, then looks at "the index" stored    near there.  If the index read up is out of bounds for the    vector of arena base addresses pymalloc maintains, then    pymalloc knows for certain that this address is not under    pymalloc's control.  Otherwise the index is in bounds, and    pymalloc compares        the arena base address stored at that index in the vector        to the arbitrary address pymalloc is investigating    pymalloc controls this arbitrary address if and only if it lies    in the arena the address's pool's index claims it lies in.    It doesn't matter whether the memory pymalloc reads up ("the    index") is initialized.  If it's not initialized, then    whatever trash gets read up will lead pymalloc to conclude    (correctly) that the address isn't controlled by it, either    because the index is out of bounds, or the index is in bounds    but the arena it represents doesn't contain the address.    This determination has to be made on every call to one of    pymalloc's free/realloc entry points, so its speed is critical    (Python allocates and frees dynamic memory at a ferocious rate    -- everything in Python, from integers to "stack frames",    lives in the heap).

在使用gc检查内存泄露时要注意模块化,否则看不到gc的debug信息:

1.py
class A(object):
def del(self):
print ‘A’

class B(object):    def __del__(self):        print 'B' a = A() b = B() a.b = b b.a = agc.set_debug(gc.DEBUG_COLLECTABLE | gc.DEBUG_UNCOLLECTABLE | \              gc.DEBUG_INSTANCES | gc.DEBUG_OBJECTS | gc.DEBUG_LEAK)

2.py
class A(object):
def del(self):
print ‘A’

class B(object):    def __del__(self):        print 'B' def make_circle_ref():    a = A()     b = B()     a.b = b     b.a = a    gc.set_debug(gc.DEBUG_COLLECTABLE | gc.DEBUG_UNCOLLECTABLE | \                  gc.DEBUG_INSTANCES | gc.DEBUG_OBJECTS | gc.DEBUG_LEAK)if __name__ == '__main__':    make_circle_ref()

虽然两个程序都存在内存泄露,但1.py中没有打印任何debug,而2.py打印出了垃圾回收
机制无法回收的对象.

python 1.py

python 2.py

gc: uncollectable
gc: uncollectable
gc: uncollectable
gc: uncollectable