运维小白的Python之路——socket编程

一、基础概述

socket通常也称作“套接字”，用户描述IP地址和端口，是一个通信链的句柄，应用程序通常通过“套接字”，想网络发出请求或者应答网络请求。
socket起源于Unix，而Unix\Linux的基本哲学之一就是“一切皆文件”，对于文件用“打开”，“读写”，“关闭”模式来操作。socket就是该模式的一个实现，socket即是一种特殊的文件，一些socket函数就是对其进行的一种操作（读写、打开和关闭）

socket和文件的区别：
1.file是针对某个文件进行的操作——打开、关闭、读写
2.socket是针对服务端和客户端socket进行打开读写和关闭

socket小练习
server端

#!/usr/bin/env python3import socketip_port = ("127.0.0.1", 8888)sk = socket.socket() #创建socket实例sk.bind(ip_port) #绑定IP，端口sk.listen(5) #监听端口，最大连接数5print("server is waitting...")conn,addr = sk.accept() #等待客户端的链接如果没有收到消息就在这里阻塞while True:    client_data = str(conn.recv(1024),encoding = "utf8") #接受客户端的信息    print(client_data)    conn.sendall(bytes("这是server端的回话",encoding = "utf8"))conn.close() #断开链接

client端

#!/usr/bin/env python3import socketip_port = ("127.0.0.1", 8888)sk = socket.socket() #创建socket实例sk.connect(ip_port) #链接服务端while True:    user_input = input("请输入")    sk.sendall(bytes(user_input,encoding = "utf8")) #向服务端发送信息    server_reply = sk.recv(1024) #接受服务端的信息    print(str(server_reply,encoding = "utf8"))sk.close()

二、内部方法

server = socket.socket(socket.AF_INET, socket.SOCK_StrEAM, 0)
参数一：地址簇
socket.AF_INET IPv4(默认)
socket.AF_INEt6
socket.UNIX 只能用于单一的Unix系统进程间通信
参数二：类型
socket.SOCK_STREAM 流式socket，用户TCP协议
socket.SOCK_DGRAM 数据报式socket， 用于UDP协议
socket.SOCK——RAW 原始套接字。普通套接字无法术力ICMP、IGMP等网络报文，而SOCK_RAW可以；其次，SOCK_RAW也可以处理特殊的IPv4报文;此外，利用原始套接字，可以通过IP_HDRINCL套接字选项由用户构造IP头。
socket.SOCK_RDM 是一种可靠的UPD形式，即保证数据的传输但是不保证数据的顺序。SOCK_RAM用来供对原始数据的低级访问，在需要执行某些特殊操作时使用，如大宋ICMP报文。SOCK_RAM通常尽限于高级用户或管理员运行程序的程序使用。
socket.SOCK_SEPQACKET 可靠的连续数据包服务
参数三：协议
0 （默认）与特定的地址家族相关的协议，如果是0，则系统会根据地址格式和桃姐类别，自动选择一个合适的协议。
UPD演示

server端#!/usrbin/env pyrhon3import socketip_port = ('127.0.0.1', 8888)server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)server.bind(ip_port)while True:    data = str(server.recv(1024), encoding = "utf8")    print(data)server.close()

client端#!/usr/bin/env python3import socketip_port = ('127.0.0.1', 8888)client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)while True:    inp = bytes(input("请输入").strip(), encoding = "utf8")    if inp == "exit":        break    client.sendto(inp, ip_port)client.close()

sk.bind(address)

sk.bind(address) 将套接字绑定到地址。address的地址格式取决于地址簇，在AF_INET下，以元组(host,ip)的形式表示地址。

sk.listen(backlog)

开始监听传入链接。backlog制定在拒绝链接之前，可以挂起的最大连接数量。
backlog = 5，表示内核已经接到了链接请求，但是服务器还没有调用accept进行处理的链接个数最大为5，这个之不能无限大，因为要在内核中维护链接队列

sk.setblocking(bool)

是否阻塞（默认是True），如果设置成False，那么accept和recv在没有接收到数据的时候，不阻塞，直接报错。

sk.accept()

接受链接并返回(conn,address)，其中conn是新的套接字对象，可以用来接收和发送数据。address是链接客户端的地址。
接收TCP客户的链接（阻塞式）等待链接的到来

sk.connect(address)

连接到address处的套接字。一般，address的格式为元组(hostname,port)，如果链接出错，返回socket.error错误。

sk.connect_ex(address)

同上，只不过会有返回值，链接成功是返回0，链接失败时候返回编码。

sk.close()

关闭套接字

sk.recv(bufsize[,flag])

接收套接字的数据。数据一字符串的形式返回，bufsize指定最多可以接收的数量，flag提供有关消息的其他信息，通常可以忽略。

sk.recvfrom(bufsize[.flag])

与recv()类似，但是返回值是(data,address)。其中data是包含接收数据的字符串，address是发送数据的套接字地址。

sk.send(string[,flag])

将string中的数据发送到链接的套接字。返回值是要发送的字节数量，改数量可能小于srting的字节大小，即：可能未将指定的内容全部发送。

sk.sendall(string[,flag])

将string中的数据发送到链接的套接字，但在返回之前会尝试发送所有的数据。成功返回None，失败则抛出异常。
内部通过递归调用send，将所有的数据发送出去。

sk.sendto(string[,flag],address)

将数据发送到套接字，address是一个(ipaddr, port)元组，是定原远程地址。返回值是大宋的字节数。该函数主要用于UDP协议。

sk.settimeout(timeout)

设置套接字操作超时，timeout十一二浮点数，单位是秒。值为None表示没有超时日期。一般超时日期应该在刚创建套接字时折尺，因为他们可能用于链接的操作（如client操作最多等待5S）

sk.getpeername()

返回套接字的远程地址。返回值通常是元组(ipaddr, port)

sk.getsockname()

返回自己的地址，返回值通常是个元组(ipaddr, port)

sk.fileno()

套接字的文件描述符

一个关于UDP的例子

server端#!/usr/bin/emv python3import socketip_port = ('127.0.0.1', 28888)server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)server.bind(ip_port)while True:    data,(host, port) = server.recvfrom(1024)    print(data, host, port)    server.sendto(bytes('ok', encoding = 'utf8'), (host, port))client端#!/usr/bin/env python3import socketip_port = ('127.0.0.1', 28888)client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)while True:    inp = input("数据:").strip()    if inp == 'exit':        break    client.sendto(bytes(inp, encoding = "utf8"), ip_port)    data = client.recvfrom(1024)    print(data)client.close()

实例（学说话）

实现了可重复的链接服务端

server端#!/usr/bin/env python3import socketip_port = ('127.0.0.1', 28888)server = socket.socket()server.bind(ip_port)server.listen()while True:    link,ip = server.accept()    while True:        link.send("这个server会学说话".encode()) #对发送的数据进行编码，默认为utf8        data = link.recv(1024).decode()        print('data:',data)        if not data:            continue        elif data == 'q':            break        print("从客户端接收的消息:",data)        link.send(data.encode())    link.close()client端 #!/usr/bin/env pyton3import socketip_port = ('127.0.0.1', 28888)client = socket.socket()client.connect(ip_port)data = client.recv(1024).decode() #对发来的数据进行解码，默认为utf8print(data)while True:    user_input = input("请输入(q退出):")    if not user_input:        print("输入的不能为空")        continue    client.send(user_input.encode()) #当输入q时如果不先将q发到服务端再关闭socket，服务端会报(BrokenPipeError: [Errno 32] Broken pipe——管道破裂)    if user_input == 'q':        break    server_res = client.recv(1024).decode()    print(server_res)client.close()

实例（简单的ssh）

client端#!/usr/bin/env python3import socketip_port = ('127.0.0.1', 28888)client = socket.socket()client.connect(ip_port)title = client.recv(1027).decode()print(title)while True:    user_input = input('请输入命令(q:退出)：')    if not user_input:        print("命令为空，请重新输入")        continue    client.send(user_input.encode())    if user_input == "q":        break    res_len = int(client.recv(1024).decode())    client.send("数据长度接受成功".encode())    res = ''    while len(res) < res_len:        res += client.recv(1024).decode()    print(res)client.close()server端#!/usr/bin/env python3import socket,osip_port = ('127.0.0.1', 28888)server = socket.socket()server.bind(ip_port)server.listen()while True:    link,ip = server.accept()    link.send("这个server是一个简单的ssh".encode())    while True:        client_cmd = link.recv(1024).decode()        if client_cmd == "q":            break        res = os.popen(client_cmd).read()        res_len = len(res)        print(res_len)        link.send(str(res_len).encode())        res_succ = link.recv(1027).decode()        link.send(res.encode())    link.close()

socketserver模块

socketserver模块可以处理多进程的socket

ThreadingTCPServer

ThreadingTCPServer实现的Soket服务器内部会为每个client创建一个 “线程”，该线程用来和客户端进行交互。

1、ThreadingTCPServer基础

使用ThreadingTCPServer:
创建一个继承自 SocketServer.BaseRequestHandler 的类
类中必须定义一个名称为 handle 的方法
启动ThreadingTCPServer
socketserver实现服务端

只接收一个进程的情况#!/usr/bin/env python3import socketserverclass MyTCPHandler(socketserver.BaseRequestHandler):    def handle(self):        while True:#可以重复接收            #等待客户端的链接，接收数据            self.data = self.request.recv(1024).strip()            #打印客户端的地址            print("{}wrote".format(self.client_address[0]))            #打印客户端发来的信息                 print(self.data)            #将客户端发来的信息变成大写病返回                   self.request.sendall(self.data.upper())if __name__ == "__main__":    host, port = "127.0.0.1", 8008    server = socketserver.TCPServer((host, port), MyTCPHandler)    server.serve_forever()多进程的情况#!/usr/bin/env python3import socketserverclass MyTCPHandler(socketserver.BaseRequestHandler):    def handle(self):        while True: #可以重复接收            #等待客户端的链接，接收数据            self.data = self.request.recv(1024).strip()            #打印客户端的地址            print("{}wrote".format(self.client_address[0]))            #打印客户端发来的信息                 print(self.data)            #将客户端发来的信息变成大写病返回                   self.request.sendall(self.data.upper())if __name__ == "__main__":    host, port = "127.0.0.1", 8008    server = socketserver.ThreadingTCPServer((host, port), MyTCPHandler)    server.serve_forever()

ThreadingTCPServer源码剖析

内部调用流程为：
启动服务端程序
执行 TCPServer.init 方法，创建服务端Socket对象并绑定 IP 和 端口
执行 BaseServer.init 方法，将自定义的继承自SocketServer.BaseRequestHandler 的类 MyRequestHandle赋值给 self.RequestHandlerClass
执行 BaseServer.server_forever 方法，While 循环一直监听是否有客户端请求到达 …
当客户端连接到达服务器
执行 ThreadingMixIn.process_request 方法，创建一个 “线程” 用来处理请求
执行 ThreadingMixIn.process_request_thread 方法
执行 BaseServer.finish_request 方法，执行 self.RequestHandlerClass() 即：执行 自定义 MyRequestHandler 的构造方法（自动调用基类BaseRequestHandler的构造方法，在该构造方法中又会调用 MyRequestHandler的handle方法）

ThreadingTCPServer相关源码：

BaseServerclass BaseServer:    """Base class for server classes.    Methods for the caller:    - __init__(server_address, RequestHandlerClass)    - serve_forever(poll_interval=0.5)    - shutdown()    - handle_request()  # if you do not use serve_forever()    - fileno() -> int   # for select()    Methods that may be overridden:    - server_bind()    - server_activate()    - get_request() -> request, client_address    - handle_timeout()    - verify_request(request, client_address)    - server_close()    - process_request(request, client_address)    - shutdown_request(request)    - close_request(request)    - handle_error()    Methods for derived classes:    - finish_request(request, client_address)    Class variables that may be overridden by derived classes or    instances:    - timeout    - address_family    - socket_type    - allow_reuse_address    Instance variables:    - RequestHandlerClass    - socket    """    timeout = None    def __init__(self, server_address, RequestHandlerClass):        """Constructor.  May be extended, do not override."""        self.server_address = server_address        self.RequestHandlerClass = RequestHandlerClass        self.__is_shut_down = threading.Event()        self.__shutdown_request = False    def server_activate(self):        """Called by constructor to activate the server.        May be overridden.        """        pass    def serve_forever(self, poll_interval=0.5):        """Handle one request at a time until shutdown.        Polls for shutdown every poll_interval seconds. Ignores        self.timeout. If you need to do periodic tasks, do them in        another thread.        """        self.__is_shut_down.clear()        try:            while not self.__shutdown_request:                # XXX: Consider using another file descriptor or                # connecting to the socket to wake this up instead of                # polling. Polling reduces our responsiveness to a                # shutdown request and wastes cpu at all other times.                r, w, e = _eintr_retry(select.select, [self], [], [],                                       poll_interval)                if self in r:                    self._handle_request_noblock()        finally:            self.__shutdown_request = False            self.__is_shut_down.set()    def shutdown(self):        """Stops the serve_forever loop.        Blocks until the loop has finished. This must be called while        serve_forever() is running in another thread, or it will        deadlock.        """        self.__shutdown_request = True        self.__is_shut_down.wait()    # The distinction between handling, getting, processing and    # finishing a request is fairly arbitrary.  Remember:    #    # - handle_request() is the top-level call.  It calls    #   select, get_request(), verify_request() and process_request()    # - get_request() is different for stream or datagram sockets    # - process_request() is the place that may fork a new process    #   or create a new thread to finish the request    # - finish_request() instantiates the request handler class;    #   this constructor will handle the request all by itself    def handle_request(self):        """Handle one request, possibly blocking.        Respects self.timeout.        """        # Support people who used socket.settimeout() to escape        # handle_request before self.timeout was available.        timeout = self.socket.gettimeout()        if timeout is None:            timeout = self.timeout        elif self.timeout is not None:            timeout = min(timeout, self.timeout)        fd_sets = _eintr_retry(select.select, [self], [], [], timeout)        if not fd_sets[0]:            self.handle_timeout()            return        self._handle_request_noblock()    def _handle_request_noblock(self):        """Handle one request, without blocking.        I assume that select.select has returned that the socket is        readable before this function was called, so there should be        no risk of blocking in get_request().        """        try:            request, client_address = self.get_request()        except socket.error:            return        if self.verify_request(request, client_address):            try:                self.process_request(request, client_address)            except:                self.handle_error(request, client_address)                self.shutdown_request(request)    def handle_timeout(self):        """Called if no new request arrives within self.timeout.        Overridden by ForkingMixIn.        """        pass    def verify_request(self, request, client_address):        """Verify the request.  May be overridden.        Return True if we should proceed with this request.        """        return True    def process_request(self, request, client_address):        """Call finish_request.        Overridden by ForkingMixIn and ThreadingMixIn.        """        self.finish_request(request, client_address)        self.shutdown_request(request)    def server_close(self):        """Called to clean-up the server.        May be overridden.        """        pass    def finish_request(self, request, client_address):        """Finish one request by instantiating RequestHandlerClass."""        self.RequestHandlerClass(request, client_address, self)    def shutdown_request(self, request):        """Called to shutdown and close an individual request."""        self.close_request(request)    def close_request(self, request):        """Called to clean up an individual request."""        pass    def handle_error(self, request, client_address):        """Handle an error gracefully.  May be overridden.        The default is to print a traceback and continue.        """        print '-'*40        print 'Exception happened during processing of request from',        print client_address        import traceback        traceback.print_exc() # XXX But this goes to stderr!        print '-'*40

TCPServerclass TCPServer(BaseServer):    """Base class for various socket-based server classes.    Defaults to synchronous IP stream (i.e., TCP).    Methods for the caller:    - __init__(server_address, RequestHandlerClass, bind_and_activate=True)    - serve_forever(poll_interval=0.5)    - shutdown()    - handle_request()  # if you don't use serve_forever()    - fileno() -> int   # for select()    Methods that may be overridden:    - server_bind()    - server_activate()    - get_request() -> request, client_address    - handle_timeout()    - verify_request(request, client_address)    - process_request(request, client_address)    - shutdown_request(request)    - close_request(request)    - handle_error()    Methods for derived classes:    - finish_request(request, client_address)    Class variables that may be overridden by derived classes or    instances:    - timeout    - address_family    - socket_type    - request_queue_size (only for stream sockets)    - allow_reuse_address    Instance variables:    - server_address    - RequestHandlerClass    - socket    """    address_family = socket.AF_INET    socket_type = socket.SOCK_STREAM    request_queue_size = 5    allow_reuse_address = False    def __init__(self, server_address, RequestHandlerClass, bind_and_activate=True):        """Constructor.  May be extended, do not override."""        BaseServer.__init__(self, server_address, RequestHandlerClass)        self.socket = socket.socket(self.address_family,                                    self.socket_type)        if bind_and_activate:            try:                self.server_bind()                self.server_activate()            except:                self.server_close()                raise    def server_bind(self):        """Called by constructor to bind the socket.        May be overridden.        """        if self.allow_reuse_address:            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)        self.socket.bind(self.server_address)        self.server_address = self.socket.getsockname()    def server_activate(self):        """Called by constructor to activate the server.        May be overridden.        """        self.socket.listen(self.request_queue_size)    def server_close(self):        """Called to clean-up the server.        May be overridden.        """        self.socket.close()    def fileno(self):        """Return socket file number.        Interface required by select().        """        return self.socket.fileno()    def get_request(self):        """Get the request and client address from the socket.        May be overridden.        """        return self.socket.accept()    def shutdown_request(self, request):        """Called to shutdown and close an individual request."""        try:            #explicitly shutdown.  socket.close() merely releases            #the socket and waits for GC to perform the actual close.            request.shutdown(socket.SHUT_WR)        except socket.error:            pass #some platforms may raise ENOTCONN here        self.close_request(request)    def close_request(self, request):        """Called to clean up an individual request."""        request.close()

ThreadingMixInclass ThreadingMixIn:    """Mix-in class to handle each request in a new thread."""    # Decides how threads will act upon termination of the    # main process    daemon_threads = False    def process_request_thread(self, request, client_address):        """Same as in BaseServer but as a thread.        In addition, exception handling is done here.        """        try:            self.finish_request(request, client_address)            self.shutdown_request(request)        except:            self.handle_error(request, client_address)            self.shutdown_request(request)    def process_request(self, request, client_address):        """Start a new thread to process the request."""        t = threading.Thread(target = self.process_request_thread,                             args = (request, client_address))        t.daemon = self.daemon_threads        t.start()ThreadingMixIn

ThreadingTCPServerclass ThreadingTCPServer(ThreadingMixIn, TCPServer): pass

RequestHandler相关源码

class BaseRequestHandler:    """Base class for request handler classes.    This class is instantiated for each request to be handled.  The    constructor sets the instance variables request, client_address    and server, and then calls the handle() method.  To implement a    specific service, all you need to do is to derive a class which    defines a handle() method.    The handle() method can find the request as self.request, the    client address as self.client_address, and the server (in case it    needs access to per-server information) as self.server.  Since a    separate instance is created for each request, the handle() method    can define arbitrary other instance variariables.    """    def __init__(self, request, client_address, server):        self.request = request        self.client_address = client_address        self.server = server        self.setup()        try:            self.handle()        finally:            self.finish()    def setup(self):        pass    def handle(self):        pass    def finish(self):        passSocketServer.BaseRequestHandler

ForkingTCPServer

ForkingTCPServer和ThreadingTCPServer的使用和执行流程基本一致，只不过在内部分别为请求者建立 “线程” 和 “进程”。
ForkingTCPServer和ThreadingTCPServer的区别，只是将实例中的

server = socketserver.ThreadingTCPServer((host, port), MyTCPHandler)

更改为

server = socketserver.ForkingTCPServer((host, port), MyTCPHandler)