python基础-线程创建、线程池、进\线程异步回调(add_done_callback)、进\线程数据共享、ftp线程池

线程创建

进程只是用来把资源集中到一起（进程只是一个资源单位，或者说资源集合），而线程才是cpu上的执行单位。
每个进程有一个地址空间，而且默认就有一个控制线程
线程就是一条流水线工作的过程，一条流水线必须属于一个车间，一个车间的工作过程是一个进程

多线程（即多个控制线程）的概念是，在一个进程中存在多个控制线程，多个控制线程共享该进程的地址空间，相当于一个车间内有多条流水线，都共用一个车间的资源

我们之前了解过进程的2种创建方式
下面的代码是2种创建线程的方式

from threading import Threadfrom multiprocessing import Processimport time,osdef task():    print('%s is running' %os.getpid())    time.sleep(2)    print('%s is done' %os.getpid())class Mythread(Thread):    def __init__(self,name):        super().__init__()        self.name=name    def run(self):        print('%s is running' % os.getpid())        time.sleep(5)        print('%s is done' % os.getpid())if __name__ == '__main__':    t=Thread(target=task)    # t=Mythread('xxxxx')    t.start()    print('主')

输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/1 开启线程的两种方式.py"10336 is running主10336 is doneProcess finished with exit code 0

线程进程pid

part1:在主进程下开启多个线程,每个线程都跟主进程的pid一样

from threading import Threadfrom multiprocessing import Processimport time,osdef task():    print('partent：%s self：%s' %(os.getppid(),os.getpid()))    time.sleep(5)if __name__ == '__main__':    t=Thread(target=task,)    # t=Process(target=task,)    t.start()    print('主',os.getppid(),os.getpid())

输出如下：

partent：9052 self：10120主 9052 10120

开多个进程,每个进程都有不同的pid

from threading import Threadfrom multiprocessing import Processimport time,osdef task():    print('partent：%s self：%s' %(os.getppid(),os.getpid()))    time.sleep(5)if __name__ == '__main__':    t=Process(target=task,)    t.start()    print('主',os.getppid(),os.getpid())

输出如下：

主 9052 2668partent：2668 self：8744

线程进程数据共享

进程之间数据不共享，但是进程之间可以通过ipc进行数据通讯

from threading import Threadfrom multiprocessing import Processimport time,osn=100def task():    global n    n=0if __name__ == '__main__':    t=Process(target=task,)    t.start()    t.join()    print('主',n)

输出如下：

主 100

线程之间内存空间共享

from threading import Threadimport time,osn=100def task():    global n    n=0if __name__ == '__main__':    t=Thread(target=task,)    t.start()    t.join()    print('主',n)

输出如下：

主 0

线程ftp

服务端：

import multiprocessingimport threadingimport sockets=socket.socket(socket.AF_INET,socket.SOCK_STREAM)s.bind(('127.0.0.1',8081))s.listen(5)def action(conn):    while True:        data=conn.recv(1024)        print(data)        conn.send(data.upper())if __name__ == '__main__':    while True:        conn,addr=s.accept()        p=threading.Thread(target=action,args=(conn,))        p.start()

客户端：

from socket import *client=socket(AF_INET,SOCK_STREAM)client.connect(('127.0.0.1',8081))while True:    msg=input('>>: ').strip()    if not msg:continue    client.send(msg.encode('utf-8'))    msg=client.recv(1024)    print(msg.decode('utf-8'))

线程池

from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutorfrom threading import current_threadimport time,randomdef task(n):    print('%s is running' %current_thread().getName())    time.sleep(random.randint(1,3))    return n**2if __name__ == '__main__':    t=ThreadPoolExecutor(3) #默认是cpu的核数*5    objs=[]    for i in range(5):        obj=t.submit(task,i)        objs.append(obj)    t.shutdown(wait=True)    for obj in objs:        print(obj.result())    print('主',current_thread().getName())

输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/4 线程池.py"ThreadPoolExecutor-0_0 is runningThreadPoolExecutor-0_1 is runningThreadPoolExecutor-0_2 is runningThreadPoolExecutor-0_0 is runningThreadPoolExecutor-0_1 is running014916主 MainThread

线程池ftp

服务端：

from socket import *from concurrent.futures import ThreadPoolExecutorimport osserver=socket(AF_INET,SOCK_STREAM)server.setsockopt(SOL_SOCKET,SO_REUSEADDR,1)server.bind(('127.0.0.1',8080))server.listen(5)def talk(conn,client_addr):    print('进程pid: %s' %os.getpid())    while True:        try:            msg=conn.recv(1024)            if not msg:break            conn.send(msg.upper())        except Exception:            breakif __name__ == '__main__':    p=ThreadPoolExecutor(5)    while True:        conn,client_addr=server.accept()        p.submit(talk,conn,client_addr)

客户端：

from socket import *client=socket(AF_INET,SOCK_STREAM)client.connect(('127.0.0.1',8081))while True:    msg=input('>>: ').strip()    if not msg:continue    client.send(msg.encode('utf-8'))    msg=client.recv(1024)    print(msg.decode('utf-8'))

线程的一些其他方法

from threading import Thread,current_thread,enumerate,active_countimport time,osdef task():    print('%s is running' %current_thread().getName())    time.sleep(5)    print('%s is done' %current_thread().getName())if __name__ == '__main__':    t=Thread(target=task,name='xxxx')    t.start()    print(t.name)    #查看当前活着的线程    print(enumerate()[0].getName())    print(active_count())    print('主',current_thread().getName())print()

输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/3 线程对象的其他属性或方法.py"xxxx is runningxxxxMainThread2主 MainThreadxxxx is done

异步-回调函数

ProcessPoolExecutor方式

我们之前总结的异步返回结果没有用到调用函数，接下来的是利用了回调函数

#pip install requestsimport requestsfrom concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutorfrom threading import current_threadimport time,osdef get(url):    print('%s GET %s' %(os.getpid(),url))    response=requests.get(url)    time.sleep(3)    if response.status_code == 200:        return {'url':url,'text':response.text}def parse(obj):    res=obj.result()    print('[%s] <%s> (%s)' % (os.getpid(), res['url'],len(res['text'])))if __name__ == '__main__':    urls = [        'https://www.python.org',        'https://www.baidu.com',        'https://www.jd.com',        'https://www.tmall.com',    ]    t=ProcessPoolExecutor(2)    for url in urls:        t.submit(get,url).add_done_callback(parse)    t.shutdown(wait=True)    print('主',os.getpid())

代码思路是：
t=ProcessPoolExecutor(2)开一个进程池，然后去并发下载网络数据，下载完毕后，
在主进程中add_done_callback去解析
这里由于主进程、子进程不是同一个进程空间，所以在解析数据时候，在主进程
输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/5 补充异步的概念.py"5628 GET https://www.python.org4816 GET https://www.baidu.com4816 GET https://www.jd.com[3204] <https://www.baidu.com> (2443)[3204] <https://www.python.org> (48856)5628 GET https://www.tmall.com[3204] <https://www.jd.com> (124541)[3204] <https://www.tmall.com> (212080)主 3204Process finished with exit code 0

ThreadPoolExecutor方式

import requestsfrom concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutorfrom threading import current_threadimport timeimport osdef get(url):    print('%s GET %s,%s' %(current_thread().getName(),os.getpid(),url))    response=requests.get(url)    time.sleep(3)    if response.status_code == 200:        return {'url':url,'text':response.text}def parse(obj):    res=obj.result()    print('[%s] <%s> (%s)' % (current_thread().getName(), res['url'],len(res['text'])))if __name__ == '__main__':    urls = [        'https://www.python.org',        'https://www.baidu.com',        'https://www.jd.com',        'https://www.tmall.com',    ]    t=ThreadPoolExecutor(2)    for url in urls:        t.submit(get,url).add_done_callback(parse)    t.shutdown(wait=True)    print('主',current_thread().getName(),os.getpid())

代码思路是：
t=ThreadPoolExecutor(2)开一个线程池，然后去并发下载网络数据，下载完毕后，
在主线程程中add_done_callback去解析
这里由于主线程、子线程是同一个进程空间，所以在解析数据时候，可能主线程、子线程都会解析
输出如下：

E:\python\python_sdk\python.exe "E:/python/py_pro/5 补充异步的概念.py"ThreadPoolExecutor-0_0 GET 12956,https://www.python.orgThreadPoolExecutor-0_1 GET 12956,https://www.baidu.com[ThreadPoolExecutor-0_1] <https://www.baidu.com> (2443)ThreadPoolExecutor-0_1 GET 12956,https://www.jd.com[ThreadPoolExecutor-0_0] <https://www.python.org> (48856)ThreadPoolExecutor-0_0 GET 12956,https://www.tmall.com[ThreadPoolExecutor-0_1] <https://www.jd.com> (124541)[ThreadPoolExecutor-0_0] <https://www.tmall.com> (212079)主 MainThread 12956Process finished with exit code 0