使用gdb调试多线程死锁

gdb，神器也，必须好好深入学习之

编写一段会造成死锁的代码，如下：

#include <unistd.h>#include <stdio.h>#include <boost/thread.hpp>boost::mutex mu1;boost::mutex mu2;void func1() {    mu1.lock();    sleep(1);    mu2.lock();    printf("func1\n");    mu2.unlock();    mu1.unlock();}void func2() {    mu2.lock();    sleep(1);    mu1.lock();    printf("func2\n");    mu1.unlock();    mu2.unlock();}int main() {    boost::thread t1(func1);    boost::thread t2(func2);    t1.join();    t2.join();    printf("Hello, world\n");    return 0;}

构建程序，生成可执行文件test

g++ -g -o test test.cpp -lboost_thread

这里，使用 -g 选项，在生成的可执行文件中加入调试信息，程序中试用了boost的线程库，所以要链接boost_thread

执行程序，./test，无任何输出，死锁了

另开一个终端，执行 ps -ef | grep test，查找进程test的pid，输出如下：

</pre><pre>

simic@ubuntu:~$ ps -ef | grep testsimic     2566  2480  0 19:59 pts/1    00:00:00 ./testsimic     2653  2326  0 20:02 pts/0    00:00:00 grep --color=auto testsimic@ubuntu:~$

可知test进程的pid为2566

打开gdb

simic@ubuntu:~$ gdbGNU gdb (GDB) 7.6Copyright (C) 2013 Free Software Foundation, Inc.License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>This is free software: you are free to change and redistribute it.There is NO WARRANTY, to the extent permitted by law.  Type "show copying"and "show warranty" for details.This GDB was configured as "i686-pc-linux-gnu".For bug reporting instructions, please see:<http://www.gnu.org/software/gdb/bugs/>.(gdb)

附着到test进程上

(gdb) attach 2566Attaching to process 2566Reading symbols from /home/simic/test...done.Reading symbols from /usr/local/lib/libboost_thread.so.1.55.0...(no debugging symbols found)...done.Loaded symbols for /usr/local/lib/libboost_thread.so.1.55.0Reading symbols from /usr/lib/i386-linux-gnu/libstdc++.so.6...done.Loaded symbols for /usr/lib/i386-linux-gnu/libstdc++.so.6Reading symbols from /lib/i386-linux-gnu/libm.so.6...(no debugging symbols found)...done.Loaded symbols for /lib/i386-linux-gnu/libm.so.6Reading symbols from /lib/i386-linux-gnu/libgcc_s.so.1...(no debugging symbols found)...done.Loaded symbols for /lib/i386-linux-gnu/libgcc_s.so.1Reading symbols from /lib/i386-linux-gnu/libc.so.6...(no debugging symbols found)...done.Loaded symbols for /lib/i386-linux-gnu/libc.so.6Reading symbols from /usr/local/lib/libboost_system.so.1.55.0...(no debugging symbols found)...done.Loaded symbols for /usr/local/lib/libboost_system.so.1.55.0Reading symbols from /lib/i386-linux-gnu/libpthread.so.0...(no debugging symbols found)...done.[New LWP 2568][New LWP 2567][Thread debugging using libthread_db enabled]Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".Loaded symbols for /lib/i386-linux-gnu/libpthread.so.0Reading symbols from /lib/i386-linux-gnu/librt.so.1...(no debugging symbols found)...done.Loaded symbols for /lib/i386-linux-gnu/librt.so.1Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.Loaded symbols for /lib/ld-linux.so.20x00d19416 in __kernel_vsyscall ()(gdb)

查看当前所有线程信息

(gdb) i thread  Id   Target Id         Frame  3    Thread 0xb76f8b70 (LWP 2567) "test" 0x00d19416 in __kernel_vsyscall ()  2    Thread 0xb6ef7b70 (LWP 2568) "test" 0x00d19416 in __kernel_vsyscall ()* 1    Thread 0xb76fa8e0 (LWP 2566) "test" 0x00d19416 in __kernel_vsyscall ()(gdb)

可以看到，当前进程中有3个线程，分别为Thread 1，Thread 2和Thread 3，其tid分别为2566，2568和2567，其中Thread 1前面有个星号，表明其为当前正在执行的线程

以下依次切换到各个线程并分别查看其堆栈信息，重点观察重点观察线程堆栈中有__lll_lock_wait ()的线程，发生死锁的线程一般都有这个东东

(gdb) thread 1[Switching to thread 1 (Thread 0xb76fa8e0 (LWP 2566))]#0  0x00d19416 in __kernel_vsyscall ()(gdb) bt#0  0x00d19416 in __kernel_vsyscall ()#1  0x0027c48c in pthread_cond_wait@@GLIBC_2.3.2 ()   from /lib/i386-linux-gnu/libpthread.so.0#2  0x001ef6ad in pthread_cond_wait () from /lib/i386-linux-gnu/libc.so.6#3  0x00c9c27c in boost::condition_variable::wait(boost::unique_lock<boost::mutex>&) () from /usr/local/lib/libboost_thread.so.1.55.0#4  0x00c9857f in boost::thread::join_noexcept() ()   from /usr/local/lib/libboost_thread.so.1.55.0#5  0x0804ecd5 in boost::thread::join (this=0xbf8c8248)    at /usr/local/include/boost/thread/detail/thread.hpp:756#6  0x0804daee in main () at test.cpp:29(gdb)

(gdb) thread 2[Switching to thread 2 (Thread 0xb6ef7b70 (LWP 2568))]#0  0x00d19416 in __kernel_vsyscall ()(gdb) bt#0  0x00d19416 in __kernel_vsyscall ()#1  0x0027f0b9 in __lll_lock_wait () from /lib/i386-linux-gnu/libpthread.so.0#2  0x0027a54b in _L_lock_791 () from /lib/i386-linux-gnu/libpthread.so.0#3  0x0027a371 in pthread_mutex_lock ()   from /lib/i386-linux-gnu/libpthread.so.0#4  0x001ef8c6 in pthread_mutex_lock () from /lib/i386-linux-gnu/libc.so.6#5  0x0804e704 in pthread_mutex_lock (m=0x8057a0c <mu1>)    at /usr/local/include/boost/thread/pthread/mutex.hpp:61#6  boost::mutex::lock (this=0x8057a0c <mu1>)    at /usr/local/include/boost/thread/pthread/mutex.hpp:113#7  0x0804da7a in func2 () at test.cpp:20#8  0x0805175b in boost::detail::thread_data<void (*)()>::run (this=0x9b73250)    at /usr/local/include/boost/thread/detail/thread.hpp:117#9  0x00c97e31 in thread_proxy () from /usr/local/lib/libboost_thread.so.1.55.0#10 0x00277e99 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0#11 0x001e19ee in clone () from /lib/i386-linux-gnu/libc.so.6(gdb)

(gdb) thread 3[Switching to thread 3 (Thread 0xb76f8b70 (LWP 2567))]#0  0x00d19416 in __kernel_vsyscall ()(gdb) bt#0  0x00d19416 in __kernel_vsyscall ()#1  0x0027f0b9 in __lll_lock_wait () from /lib/i386-linux-gnu/libpthread.so.0#2  0x0027a54b in _L_lock_791 () from /lib/i386-linux-gnu/libpthread.so.0#3  0x0027a371 in pthread_mutex_lock ()   from /lib/i386-linux-gnu/libpthread.so.0#4  0x001ef8c6 in pthread_mutex_lock () from /lib/i386-linux-gnu/libc.so.6#5  0x0804e704 in pthread_mutex_lock (m=0x8057a24 <mu2>)    at /usr/local/include/boost/thread/pthread/mutex.hpp:61#6  boost::mutex::lock (this=0x8057a24 <mu2>)    at /usr/local/include/boost/thread/pthread/mutex.hpp:113#7  0x0804da2a in func1 () at test.cpp:11#8  0x0805175b in boost::detail::thread_data<void (*)()>::run (this=0x9b73078)    at /usr/local/include/boost/thread/detail/thread.hpp:117#9  0x00c97e31 in thread_proxy () from /usr/local/lib/libboost_thread.so.1.55.0#10 0x00277e99 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0#11 0x001e19ee in clone () from /lib/i386-linux-gnu/libc.so.6(gdb)

其中2568和2567的堆栈信息中都有__lll_lock_wait，为重点怀疑对象

在2567的堆栈信息中，#7一行显示了代码中的func1函数，其他各行都是库函数或者系统调用，跳转到该栈帧

(gdb) f 7#7  0x0804da2a in func1 () at test.cpp:1111          mu2.lock();(gdb)

可见，出问题的地方是源文件test.cpp的第11行语句，打印出mu2的信息

(gdb) p mu2$1 = {m = {__data = {__lock = 2, __count = 0, __owner = 2568, __kind = 0,      __nusers = 1, {__spins = 0, __list = {__next = 0x0}}},    __size = "\002\000\000\000\000\000\000\000\b\n\000\000\000\000\000\000\001\000\000\000\000\000\000", __align = 2}}(gdb)

可见，锁mu2当前的拥有者是2568，即Thread 2。由此可得出如下判断，Thread 3申请锁mu2，但mu2当前被Thread 2所持有

再来查看Thread 2的相关信息

(gdb) thread 2[Switching to thread 2 (Thread 0xb6ef7b70 (LWP 2568))]#0  0x00d19416 in __kernel_vsyscall ()(gdb) bt#0  0x00d19416 in __kernel_vsyscall ()#1  0x0027f0b9 in __lll_lock_wait () from /lib/i386-linux-gnu/libpthread.so.0#2  0x0027a54b in _L_lock_791 () from /lib/i386-linux-gnu/libpthread.so.0#3  0x0027a371 in pthread_mutex_lock ()   from /lib/i386-linux-gnu/libpthread.so.0#4  0x001ef8c6 in pthread_mutex_lock () from /lib/i386-linux-gnu/libc.so.6#5  0x0804e704 in pthread_mutex_lock (m=0x8057a0c <mu1>)    at /usr/local/include/boost/thread/pthread/mutex.hpp:61#6  boost::mutex::lock (this=0x8057a0c <mu1>)    at /usr/local/include/boost/thread/pthread/mutex.hpp:113#7  0x0804da7a in func2 () at test.cpp:20#8  0x0805175b in boost::detail::thread_data<void (*)()>::run (this=0x9b73250)    at /usr/local/include/boost/thread/detail/thread.hpp:117#9  0x00c97e31 in thread_proxy () from /usr/local/lib/libboost_thread.so.1.55.0#10 0x00277e99 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0#11 0x001e19ee in clone () from /lib/i386-linux-gnu/libc.so.6(gdb) f 7#7  0x0804da7a in func2 () at test.cpp:2020          mu1.lock();(gdb) p mu1$2 = {m = {__data = {__lock = 2, __count = 0, __owner = 2567, __kind = 0,      __nusers = 1, {__spins = 0, __list = {__next = 0x0}}},    __size = "\002\000\000\000\000\000\000\000\a\n\000\000\000\000\000\000\001\000\000\000\000\000\000", __align = 2}}(gdb)

可见，Thread 2申请锁mu1，但mu1当前被Thread 3所持有

综合一下，Thread 2持有mu2，正在申请mu1，而Thread 3持有mu1，正在申请mu2，互不相让，于是造成了死锁