pydbg的安装和编译过程

来源：互联网发布：外文检索数据库编辑：程序博客网时间：2024/06/01 10:45

文章一：

http://www.glamenv-septzen.net/en/view/13

Yesterday I tried installing pydbg and pydasm on my notepc.
pydasm is popular, famous library to disassemble machine codes (opcodes).
pydbg is also popular, famous library to build lightweight, extensible debugger for Windows platform.
Actually, pydbg is included in PaiMei, windows platform debugger framework.

The journey was hard, full of struggle and traps with python distutils.
I left these notes, memos, and traps for future person (including myself) who want installing these excellent reverse engineering tools written in python and c.

My notepc environments are:

CPU : Intel Pentium M (Centrino) 1.2GHzRAM : 1GBOS : Windows XP Professional SP3 (Japanese)Python: Python 2.5 (install from MSI installer)        Install Dir : C:\Python25Visual Studio : Visual C++ 2008 Express Edision (SP1)Subversion: TortoiseSVN 1.6.x

We require Subversion to obtain PaiMei later.

Okay, Let's begin.

Installing pydasm step-by-step
- getting libdasm
- try compile -> errors!!
- avoiding distutil's registry key checks
- try compiling again -> build success!!
- install pydasm and test ... oh-no, "MSVCR90.dll" not foud !?
- edit msvccompiler.py again, change compile option -> pydasm works correct!!
Installing pydbg step-by-step
- checkout from SVN repositry and install -> "Python26.dll not found" !?
- remove bundled pydasm.pyd in pydbg -> success!!
- note for other modules in PaiMei

Installing pydasm step-by-step

pydbg requires pydasm. We have to install pydasm before installing pydbg.
But pydasm is written by c, and no official binary release for win32 platform.

Out environments uses:

Python 2.5Visual C++ 2008 Express Edition SP1

getting libdasm

pydasm is included "libdasm", simple x86 disassembly library. Author of pydasm is another person to author of libdasm.
libdasm was written by jt (nologin.org), pydasm was written by ero (dkbza.org). pydasm was contributed to libdasm.

dkbza - pydasm (just showing libdasm download link, short overview, example codes)
- http://dkbza.org/pydasm.html

Now, let's download libdasm, including pydasm from following url:

libdasm : nologin -- code
- http://www.nologin.org/main.pl?action=codeView&codeId=49&

When download success, expand tar.gz into your favorite directory.
Yesterday, I get libdasm-1.5.tar.gz. After expanding, I got following directory trees:

libdasm-1.5\    libdasm.c    README.txt    ...    bin\    pydasm\    ...

try compile -> errors!!

Next, open command prompt, setup your VC++ environment (ex: vcvars32.bat). Confirm "C:\Python25" is added PATH environment or ".py" extension is associated with "C:\Python25\python.exe".
In your command prompt, cd to "libdasm-1.5\pydasm", and type:

> setup.py build_extrunning build_exterror: Python was built with Visual Studio 2003;extensions must be built with a compiler than can generate compatible binaries.Visual Studio 2003 was not found on this system. If you have Cygwin installed,you can try compiling with MingW32, by passing "-c mingw32" to setup.py.

Ouch!! Why? My command prompt enviroments are surely prepared for VC++ tools!!

> clMicrosoft(R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01 for 80x86Copyright (C) Microsoft Corporation.  All rights reserved....> linkMicrosoft (R) Incremental Linker Version 9.00.30729.01Copyright (C) Microsoft Corporation.  All rights reserved.

The answer is, Python's distutils tool (used from setup.py) checks Visual Studio's registry keys and confirm proper version is installed or not.
Unfortunately, registry keys installed by VC++2008 Express Edision on my environment are not what distutils expected.

avoiding distutil's registry key checks

Then how to avoid these annoying registry key checks ?
The answer is, edit distutil's library source code. Python is open source, and distutils are written in python.

(Skips my surveys and researches for distutils and its source code.)

As a result, edit following source code:

C:\Python25\Lib\distutils\msvccompiler.py

In msvccompiler.py:

...class MSVCCompiler (CCompiler) :    ...    def __init__ (self, verbose=0, dry_run=0, force=0):        ...        # comment out here!!        self.__macros = MacroExpander(self.__version)

Registry keys checks are implemented in MacroExpander() and its constructor. And, "self.__macros" is optional, not needed. So removing this line doesn't effect to compiling.
After removing unneccesary line, delete old ".pyc" file, msvccompiler.pyc, may be exists in same directory to ".py".

try compiling again -> build success!!

In msvccompiler.py, there's no registry checks when "DISTUTILS_USE_SDK" and "MSSdk" environments are set and "cl.exe" is found in PATH environment.
So we set "DISTUTILS_USE_SDK" and "MSSdk" in our command prompt window:

> set DISTUTILS_USE_SDK=1> set MSSdk=1

Every thing is okay for each environment values. msvccompiler.py only checks each key is exists, ignoring values.

Okay, let's try compiling:

> setup.py build_extrunning build_extbuilding 'pydasm' extensioncreating buildcreating build\temp.win32-2.5creating build\temp.win32-2.5\ReleaseC:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe \    /c /nologo /Ox /MD /W3 /GX /DNDEBUG \    -IC:\Python25\include -IC:\Python25\include -IC:\Python25\PC \    /Tc../libdasm.c /Fobuild\temp.win32-2.5\Release\../libdasm.objC:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe \    /c /nologo /Ox /MD /W3 /GX /DNDEBUG \    -IC:\Python25\include -IC:\Python25\include -IC:\Python25\PC \    /Tcpydasm.c /Fobuild\temp.win32-2.5\Release\pydasm.objcreating build\lib.win32-2.5C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\link.exe \    /DLL /nologo /INCREMENTAL:NO \    /LIBPATH:C:\Python25\libs /LIBPATH:C:\Python25\PCBuild /EXPORT:initpydasm \    build\temp.win32-2.5\Release\../libdasm.obj build\temp.win32-2.5\Release\pydasm.obj \    /OUT:build\lib.win32-2.5\pydasm.pyd \    /IMPLIB:build\temp.win32-2.5\Release\..\pydasm.lib

(omitting warnings from build output)

When all work correct, we'll get

libdasm-1.5\pydasm\build\lib.win32-2.5\pydasm.pyd

install pydasm and test ... oh-no, "MSVCR90.dll" not foud !?

Now invoking install command:

> setup.py installrunning installrunning buildrunning build_extrunning install_libcopying build\lib.win32-2.5\pydasm.pyd -> C:\Python25\Lib\site-packagescopying build\lib.win32-2.5\pydasm.pyd.manifest -> C:\Python25\Lib\site-packagesrunning install_egg_infoRemoving C:\Python25\Lib\site-packages\pydasm-1.5-py2.5.egg-infoWriting C:\Python25\Lib\site-packages\pydasm-1.5-py2.5.egg-info

Then invoking python and try to import pydasm:

> C:\Python25\python.exePython 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32Type "help", "copyright", "credits" or "license" for more information.>>> import pydasmTraceback (most recent call last):  File "<stdin>", line 1, in <module>ImportError: DLL load failed: 指定されたモジュールが見つかりません。>>> quit()

Oh-no, last "ImportError" says "specified module not found". And on my environment, "MSVCR90.dll not found" messagebox was shown.

Checking module dependencies:

> dumpbin /dependents C:\Python25\python.exe...   python25.dll   MSVCR71.dll   <-- Python25 depends on MSVCR71.dll   KERNEL32.dll...> dumpbin /dependents build\lib.win32-2.5\pydasm.pyd...   MSVCR90.dll    <-- pydasm depends on MSVCR90.dll !!   python25.dll   KERNEL32.dll

On my environment, MSVCR90.dll was installed to Visual Studio installed directory, but not to Windows system directory.If you've already installed VC-Runtime including MSVCR90.dll to your Windows system directory, those ImportError and message box may not be shown, all works correctly.

How do that ? There're 2 ways to resolve this error.

install VC-Runtime including MSVCR90.dll to Windows.
1. This is straightforward, but Python uses MSVCR71.dll (compiled by older version VisualStudio). I'm afraid of conflict between runtime dlls.
link VC-Runtime static.
1. This makes portable module, but need to change compile options, module file size become big.

I'm afraid of conflits between runtime dlls strongly (may be another trap), so I delve into second way.

edit msvccompiler.py again, change compile option -> pydasm works correct!!

Then how to change compile option ? What option should we choose ?
msvccompiler.py setups compiler options in MSVCCompiler class's initialize method.

msvccompiler.py:

...class MSVCCompiler (CCompiler) :...    def initialize(self):        ...        self.preprocess_options = None        if self.__arch == "Intel":            self.compile_options = [ '/nologo', '/Ox', '/MD', '/W3', '/GX' ,                                     '/DNDEBUG']            self.compile_options_debug = ['/nologo', '/Od', '/MDd', '/W3', '/GX',                                          '/Z7', '/D_DEBUG']

"/MD" and "/MDd" indicates dynamic linking multithreaded library.
For more details about runtime library and compiler options, see following MSDN.

C Run-Time Libraries
- http://msdn.microsoft.com/en-us/library/abx4dbyh.aspx
  - "MSDN Library" > "Development Tools and Languages" > "Visual Studio 2010" > "Visual Studio" > "Visual Studio Languages" > "Visual C++" > "Visual C++ Reference" > "Visual C++ Libraries Reference" > "Run-Time Library" > "C Run-Time Libraries"

This time we want to link static multithreaded library, So we choose "/MT" and "/MTd".
Edit msvccompiler.py:

...class MSVCCompiler (CCompiler) :...    def initialize(self):        ...        self.preprocess_options = None        if self.__arch == "Intel":            # change '/MD' => '/MT'            self.compile_options = [ '/nologo', '/Ox', '/MT', '/W3', '/GX' ,                                     '/DNDEBUG']            # change '/MDd' => '/MTd'            self.compile_options_debug = ['/nologo', '/Od', '/MTd', '/W3', '/GX',                                          '/Z7', '/D_DEBUG']

Delete old msvccompiler.pyc and pydasm\build directory, and try build again.

> setup.py build_ext> dumpbin /dependents build\lib.win32-2.5\pydasm.pyd Image has the following dependencies:   python25.dll   KERNEL32.dll...> setup.py install...> C:\Python25\python.exePython 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32Type "help", "copyright", "credits" or "license" for more information.>>> import pydasm>>> buffer = '\x90\x31\xc9\x31\xca\x31\xcb'>>> offset = 0>>> while offset < len(buffer):...     i = pydasm.get_instruction(buffer[offset:], pydasm.MODE_32)...     print pydasm.get_instruction_string(i, pydasm.FORMAT_INTEL, 0)...     if not i:...         break...     offset += i.length...nopxor ecx,ecxxor edx,ecxxor ebx,ecx>>> quit()

All-right, pydasm seems to work correctly.

Next, try to install PaiMei, pydbg.

Installing pydbg step-by-step

PaiMei Official Docs
- http://pedram.redhive.com/PaiMei/docs/
PaiMei Google Codes Site
- http://code.google.com/p/paimei/
Other Resources:
- http://thatsbroken.com/?p=26
- http://maliciousattacker.blogspot.com/2006/12/pydbg-in-vmware.html
- http://www.openrce.org/downloads/details/208/PaiMei
- http://maliciousattacker.blogspot.com/2007/01/setting-up-pydbg.html

checkout from SVN repositry and install -> "Python26.dll not found" !?

1st, checkout (or export) PaiMei latest source tree from SVN repositry. Use your favorite SVN tools.
Yesterday, I checkouted r248 trunk source tree.

2nd, open command prompt window, cd to PaiMei directory, and run setup.py.

> setup.py build(...)> setup.py install(...)

3rd, try to import pydbg:

> C:\Python25\python.exePython 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32Type "help", "copyright", "credits" or "license" for more information.>>> import pydbgTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "pydbg\__init__.py", line 47, in <module>    from pydbg                   import *  File "C:\in_vitro\SVNWORK\paimei\pydbg\pydbg.py", line 32, in <module>    import pydasmImportError: DLL load failed: 指定されたモジュールが見つかりません。

And message box was shown, saying "Python26.dll not found".

See carefully above python stacktrace. It is very strange that error has occurred at "import pydasm".
At previous section, we confirm "import pydasm" work correctly. What's happening?

remove bundled pydasm.pyd in pydbg -> success!!

The Answer is : "pydasm.pyd" is bundled in pydbg module directory, and it was compiled with Python 2.6.

> dumpbin /dependents C:\Python25\Lib\site-packages\pydbg\pydasm.pyd                                                    ^^^^^^^^^^^^^^^^ pydasm.pyd was bundled!!...  Image has the following dependencies:    MSVCR90.dll    python26.dll    KERNEL32.dll...

Then, we simply remove bundled "pydbg\pydasm.pyd". Close command prompt, shutdown all python program, and re-open command prompt (clear loaded dlls).

Try-again:

> C:\Python25\python.exePython 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32Type "help", "copyright", "credits" or "license" for more information.>>> import pydbg>>>

All-right, no errors!!

Finally, invoke calc.exe, get its PID, and test following example:

from pydbg import *from pydbg.defines import *def handler_breakpoint (pydbg):  if pydbg.first_breakpoint:    print "[*] Hit 1st breakpoint!"    return DBG_CONTINUE  print "[*] Hit breakpoint!"  return DBG_CONTINUEdbg = pydbg()dbg.set_callback(EXCEPTION_BREAKPOINT, handler_breakpoint)dbg.attach(XXXXX) # pid of calc.exerecv = dbg.func_resolve("user32", "ShowWindow")dbg.bp_set(recv)dbg.debug_event_loop()

Example:

> C:\Python25\python.exePython 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32Type "help", "copyright", "credits" or "license" for more information.>>> from pydbg import *>>> from pydbg.defines import *>>> def handler_breakpoint (pydbg):...   if pydbg.first_breakpoint:...     print "[*] Hit 1st breakpoint!"...     return DBG_CONTINUE...   print "[*] Hit breakpoint!"...   return DBG_CONTINUE...>>> dbg = pydbg()>>> dbg.set_callback(EXCEPTION_BREAKPOINT, handler_breakpoint)>>> dbg.attach(5084)<pydbg.pydbg.pydbg instance at 0x00BF8198>>>> recv = dbg.func_resolve("user32", "ShowWindow")>>> dbg.bp_set(recv)<pydbg.pydbg.pydbg instance at 0x00BF8198>>>> dbg.debug_event_loop()[*] Hit 1st breakpoint![*] Hit breakpoint!...(minimize, or restore calc window)...[*] Hit breakpoint!>>> quit()

note for other modules in PaiMei

In this article, we JUST ONLY want to install pydbg in PaiMei.
This article ignored to setup other modules in PaiMei, like PIDA, GUI components.

If you want to setup these modules, PaiMei framework fullset, see documents in official site or "docs/index.html" in SVN source tree.

文章二：

这两天在看《python灰帽子》，其中有章节提到pydbg的使用，于是安装之，但是安装过程中发现并没有像其他的包一样简单，还需要修改一些东西才能正常使用，参照http://www.glamenv-septzen.net/en/view/13网页方法顺利安装。现写成日志以记录安装过程，以免日后重装系统后忘记安装方法。
OS:Windows XP Professional SP3
Python: Python 2.5
Visual Studio : 2005
1.在http://www.nologin.org/main.pl?action=codeView&codeId=49&下载libdasm，因为pydbg引用libdasm库。
2.解压libdasm库到任意目录。
3.使用VS命令行提示符（在开始菜单的VS->VS TOOL目录下），进入到"libdasm-1.5\pydasm"，在命令行中键入

setup.py build_ext屏幕会回显running build_ext error: Python was built with Visual Studio 2003; extensions must be built with a compiler than can generate compatible binaries. Visual Studio 2003 was not found on this system. If you have Cygwin installed, you can try compiling with MingW32, by passing "-c mingw32" to setup.py.这是由于编译器版本不一致导致的错误。4.找到C:\Python25\Lib\distutils\msvccompiler.py文件，修改其中class MSVCCompiler (CCompiler) :的self.__macros = MacroExp这一行，将这行干掉即可。修改 self.compile_options = [ '/nologo', '/Ox', '/MD', '/W3', '/GX' ,                                      '/DNDEBUG']             self.compile_options_debug = ['/nologo', '/Od', '/MDd', '/W3', '/GX',                                           '/Z7', '/D_DEBUG']这两行中的MD和MDd为MT和MTd，这还是因为编译器的版本不一致。5.在命令行提示符下设置set DISTUTILS_USE_SDK=1 set MSSdk=1这两个变量。6.再次运行setup.py build_ext，再运行setup.py install，这样就能在C:\Python25\Lib\site-packages下生成pydasm.pyd文件了。7.使用SVN，在http://paimei.googlecode.com/svn/trunk/pydbg地址下载整个的pydbg包，将该pydbg包放置在C:\Python25\Lib\site-packages文件夹下，删除删除该包下的pydasm.pyd文件。之后import pydbg就可以使用啦，感谢那位外国作者，感谢pydbg的作者。文章三：安装pydbg pydasm
Posted on June 12, 2012 by chenlibo
  var _gaq = _gaq || [];  _gaq.push(['_setAccount', 'UA-33094566-1']);  _gaq.push(['_trackPageview']);
  (function() {    var ga = document.createElement(‘script’); ga.type = ‘text/javascript’; ga.async = true;    ga.src = (‘https:’ == document.location.protocol ? ‘https://ssl’ : ‘http://www’) + ‘.google-analytics.com/ga.js’;    var s = document.getElementsByTagName(‘script’)[0]; s.parentNode.insertBefore(ga, s);  })();
pydbg是著名的基于python 实现的调试器，是作为著名的逆向架构paimei的调试框架发布的，其依赖于pydasm，同时这个pydasm库存在于libdasm中。
整个安装过程还算顺利，主要是Python distutils的使用不熟悉，说实话，我都没怎么用过python,怎么可能熟呢？
首先下载libdasm，链接如下：
http://code.google.com/p/libdasm/downloads/list
由于需要C 编译器，所以安装 VC2010（真是重量级啊），进入Visual Studio Command Prompt (2010)，切换到下载下来的代码包中pydasm目录下，运行 setup.py install，报错无法找到vcvarsall.bat这个文件，
如下提示：
unable to find vcvarsall.bat
怀疑是环境变量的问题，所以将这个文件所在路径C:\Program Files\Microsoft Visual Studio 10.0\VC 加入到path，还是不行，开始google，发现有价值的链接如下：
http://bbs.pediy.com/showthread.php?t=133952
链接提示需要修改distutil 库源码中的msvccompiler.py文件，将删除self.__macros = MacroExpander(self.__version)，来绕过对编译环境版本在注册表中值的验证，就是必须是某个版本的编译器如VS2008，且验证本地环境VC++2008 Express Edision 的注册表键值是不是distutils 库支持的
但是在我正要实施时，发现该文中的报错似乎与我不符，我的应该在这个的错误前面，我的错误是由于还没有正确找到编译环境，所以继续搜寻，发现大多数链接都是用下载mingw来解决，包括很多国外的大牛网站，但是这只能算是绕过，根本不是解决，再说我费那么大劲装的VS2010岂不是白费了吗？继续找了一会，终于发现一个有价值的链接：
http://blog.csdn.net/ren911/article/details/6448696
这个链接就与我所遇的问题一样，采取的方法是修改文件msvc9compiler.py，如下：
修改MSVCCompiler函数：
vc_env = query_vcvarsall(VERSION, plat_spec)
为：
vc_env = query_vcvarsall(8.0, plat_spec)//由于我的是VS2010,所以改为10.0，python默认是VS2008 即9.0
还要加这句话：
if VERSION < 8.0:
raise DistutilsPlatformError(“VC %0.1f is not supported by this module” % VERSION)
看来，python3.2中的distutils只支持vc8.0以上，即vs2005以上的编译器。
修改之后安装成功！！
接下来安装pydbg，由于是paimei的调试架构，所以先要下载paimei
http://code.google.com/p/paimei/
https://github.com/OpenRCE/paimei/downloads
然后是下载pydbg
https://github.com/OpenRCE/pydbg/downloads
一开始还以为可以直接独立安装pydbg,后来发现其中源码中没有安装文件，只是源代码，参考下面链接
http://www.diybl.com/course/3_program/python/20110829/559096.html
windows下没有装SVN、git，所以直接浏览器下载安装包，解压paimei的安装包，运行setup install，简要看了下错误，像是报错找不到 pydbg的源代码文件，所以发现paimei安装包中的pydbg文件夹中是空的。将pydbg安装包的文件复制到其中，成功安装！！！
最后，检测安装结果发现，import pydbg时，仍然报错，找不到pydasm这个库，所以参考上个链接中的方法，将Lib\Site-packages\pydbg下的pydasam.pyd删掉。因为在安装pydasm之后，Lib\Site-packages\下已经存在库文件pydasam.pyd。之后运行python，调用这两个库，没有报错，如下：
文章四：ollydbg 虽然很强大, 还是无法满足所有的需求, 比如找了很久也没找到的功能就有:
1. 进入某个模块的断点. 即相当于把某个 dll 的所有函数(不光是导出的函数) 入口全部下上断点.
2. 对 COM 组件的函数下断点.
3. 内存断点的条件限制.
...
要解决所有问题的唯一办法就是写自己的调试器, 但是要从头开始写一个调试器并不是一项简单的工作. 这时, 利用 pydbg 库来实现特定的调试需求就非常方便了. 不过 pydbg 的安装也并不容易, 你不光要有 python 还得有 C 编译器才行. 下面是我安装过程的记录:
 1. 下载 libdasm (pydbg 依赖于 pydasm, pydasm 又需要 libdasm)
     http://www.nologin.org/main.pl?action=codeView&codeId=49&
     http://www.adintr.com/down/libs/libdasm-1.5.tar.gz
 2. 用 VC 的命令行进入解压出来的 pydasm 目录. 执行 setup.py install 自动编译后安装.
     如果你从没有编译个 c 源代码的 python 库, 而且你安装的 python 和你的 VC 编译器版本不匹配的话, 可能会出现找不到 C 编译器的错误. 这时就需要手动修改 python 的 Lib\distutils\msvccompiler.py 文件.
 3. 测试安装的 pydasm. 进入 python 命令行, 输入 import pydasm, 出现错误:  ImportError: DLL load failed: 找不到指定的模块.
     使用 dumpbin /DEPENDENTS pydasm.pyd 命令查找它依赖的 dll, 发现有:   MSVCR80.dll,  python27.dll ,  KERNEL32.dll (根据你的 python 版本和 VC 版本可能有不同). 缺少的应该是 MSVCR80.dll 拷贝一份到 python 目录下. 再次 import pydasm, 提示错误: ImportError: DLL load failed: %1 不是有效的 Win32 应用程序。
 4. 没办法了,  只要用我自己用源代码编译出来的 python 版本了. 自己编译出来的版本是把 VC 编译器配置好了的. 直接编译后 import pydasm 就成功了 :)
 5. 下载 pydbg, 源代码在 google code 上, 需要用 svn 下载:  http://paimei.googlecode.com/svn/trunk/.
 6. 进入源代码目录, 执行 setup.py install.
 7. 安装完成后直接 import pydbg 仍然是提示 pydasm 的 dll 无法加载. 需要把 site-packages\pydbg 目录下那一份 pydasm.pyd 文件删除.
 终于可以导入 pydbg 了: