Python Multi-Processing多线程编程

今天总结一下Python的Multi-Processing多线程编程。

创建多进程的方法

start new thread

使用thread.start_new_thread是一种最简单的创建多进程的方法。

#!/usr/bin/pythonimport threadimport time# Define a function for the threaddef print_time(threadName, delay):   count = 0   while count < 5:      time.sleep(delay)      count += 1      print "%s: %s"%(threadName, time.ctime(time.time()))# Create two threads as followstry:   thread.start_new_thread(print_time, ("Thread-1", 2,))   thread.start_new_thread(print_time, ("Thread-2", 4,))except:   print "Error: unable to start thread"while 1:   pass

输出结果为：

Thread-1: Wed Sep 27 19:52:57 2017Thread-2: Wed Sep 27 19:52:59 2017Thread-1: Wed Sep 27 19:52:59 2017Thread-1: Wed Sep 27 19:53:01 2017Thread-1: Wed Sep 27 19:53:03 2017Thread-2: Wed Sep 27 19:53:03 2017Thread-1: Wed Sep 27 19:53:05 2017Thread-2: Wed Sep 27 19:53:07 2017Thread-2: Wed Sep 27 19:53:11 2017Thread-2: Wed Sep 27 19:53:15 2017

如果没有程序末尾的while无限循环，thread1和thread2之外的主线程瞬间就会结束，程序就退出了，因此会什么也不打印。

threading module

#!/usr/bin/pythonimport threadingimport timeexitFlag = 0class myThread(threading.Thread):   def __init__(self, threadID, name, delay):      threading.Thread.__init__(self)      self.threadID = threadID      self.name = name      self.delay = delay   def run(self):      print "Starting " + self.name      print_time(self.name, 5, self.delay)      print "Exiting " + self.namedef print_time(threadName, counter, delay):   while counter:      if exitFlag:         threadName.exit()      time.sleep(delay)      print "%s: %s" % (threadName, time.ctime(time.time()))      counter -= 1# Create new threadsthread1 = myThread(1, "Thread-1", 1)thread2 = myThread(2, "Thread-2", 2)# Start new Threadsthread1.start()thread2.start()print "Exiting Main Thread"

输出结果为：

Starting Thread-1Starting Thread-2Exiting Main ThreadThread-1: Wed Sep 27 20:00:16 2017Thread-2: Wed Sep 27 20:00:17 2017Thread-1: Wed Sep 27 20:00:17 2017Thread-1: Wed Sep 27 20:00:18 2017Thread-2: Wed Sep 27 20:00:19 2017Thread-1: Wed Sep 27 20:00:19 2017Thread-1: Wed Sep 27 20:00:20 2017Exiting Thread-1Thread-2: Wed Sep 27 20:00:21 2017Thread-2: Wed Sep 27 20:00:23 2017Thread-2: Wed Sep 27 20:00:25 2017Exiting Thread-2

也可以通过加入线程列表的形式，等待线程执行结束后主线程才退出：

#!/usr/bin/pythonimport threadingimport timeexitFlag = 0class myThread(threading.Thread):   def __init__(self, threadID, name, delay):      threading.Thread.__init__(self)      self.threadID = threadID      self.name = name      self.delay = delay   def run(self):      print "Starting " + self.name      print_time(self.name, 5, self.delay)      print "Exiting " + self.namedef print_time(threadName, counter, delay):   while counter:      if exitFlag:         threadName.exit()      time.sleep(delay)      print "%s: %s" % (threadName, time.ctime(time.time()))      counter -= 1# Create new threadsthread1 = myThread(1, "Thread-1", 1)thread2 = myThread(2, "Thread-2", 2)# Start new Threadsthread1.start()thread2.start()# Add threads to thread listthreads = []threads.append(thread1)threads.append(thread2)# Wait for all threads to completefor t in threads:    t.join()print "Exiting Main Thread"

输出结果为：

Starting Thread-1Starting Thread-2Thread-1: Wed Sep 27 20:04:21 2017Thread-1: Wed Sep 27 20:04:22 2017Thread-2: Wed Sep 27 20:04:22 2017Thread-1: Wed Sep 27 20:04:23 2017Thread-2: Wed Sep 27 20:04:24 2017Thread-1: Wed Sep 27 20:04:24 2017Thread-1: Wed Sep 27 20:04:25 2017Exiting Thread-1Thread-2: Wed Sep 27 20:04:26 2017Thread-2: Wed Sep 27 20:04:28 2017Thread-2: Wed Sep 27 20:04:30 2017Exiting Thread-2Exiting Main Thread

互斥锁同步线程

不加锁的问题

当不同的线程操作同一批数据的时候，就有可能产生意想不到的问题。比如说：

# encoding: UTF-8import threadingimport timeclass MyThread(threading.Thread):    def run(self):        global num        time.sleep(1)        num = num+1        msg = self.name+' set num to '+str(num)        print msgnum = 0threads = []def test():    for i in range(1000):        t = MyThread()        t.start()    threads.append(t)if __name__ == '__main__':    test()    for t in threads:        t.join()    print num

理论上说，一千个线程都让num增1,结果应该是1000,但是输出的结果不是1000:(而且每次都不一样)

............Thread-998 set num to 905Thread-988 set num to 906Thread-994 set num to 904Thread-992 set num to 901Thread-990 set num to 907Thread-1000 set num to 908908

这是因为不同线程同时操作同样数据导致的线程不同步。此时需要通过加互斥锁实现线程的同步。

加锁的效果

# encoding: UTF-8import threadingimport timeclass MyThread(threading.Thread):    def run(self):        global num        time.sleep(1)    threadLock.acquire()         num = num+1    threadLock.release()        msg = self.name+' set num to '+str(num)        print msgnum = 0threadLock = threading.Lock()threads = []def test():    for i in range(1000):        t = MyThread()        t.start()    threads.append(t)if __name__ == '__main__':    test()    for t in threads:        t.join()    print num

加锁相当于每个数据在每一时刻只能被一个线程获取到，而获取不到锁的线程就会处于暂时的阻塞状态。输出结果是这样的：

.........Thread-994 set num to 994Thread-995 set num to 995Thread-993 set num to 993Thread-996 set num to 996Thread-999 set num to 997Thread-1000 set num to 998Thread-998 set num to 999Thread-997 set num to 10001000

可以看出，并不是按照线程的创建时间获取数据的处理权限，但是结果却可以保证每个线程都能依次处理数据，从而保证数据被自增了1000次。