No.011-Python-学习之路-Day8-Socket网络编程

Socket网络编程介绍

背景

两台电脑之间的通信是通过网络来传输的，传输的是bit流，实现通信是使用IP+端口，把网络删的相关传输进行封装，仅暴露一些接口，这些接口即socket；

so，对tcp及udp等网络协议的封装即socket；

socket只干两件事情<发数据send，收数据receive>

两台设备间通信

步骤一：首先两台设备需要import socket

步骤二：两台设备声明socket.TCP/IP相同协议类型，声明地址簇<Socket Families>及协议类型<Socket protocol types>

步骤三：后面步骤如下图

地址簇<Socket Families>

socket.AF_UNIX 本机进程间通信,也到本地的localaddress<127.0.0.1>转一下发送给另外进程
socket.AF_INET IPV4
socket.AF_INET6 ipv6

协议类型<Socket protocol Types>

socket.SOCK_STREAM # for tcp
socket.SOCK_DGRAM # for udp
socket.SOCK_RDM # 是一种可靠的UDP形式，即保证交付数据报大拿不保证顺序；
socket.SOCK_RAW 原始套接字，普通的套接字无法处理ICMP及IGMP等网络报文，而SOCK_RAW；

注:SOCK_RAW也可以处理特殊的IPV4报文，利用原始套接字，可以通过IP_HDRINCL套接字选项由用户构造IP地址头

常用方法

sk = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM, proto=0, fileno=None)
# Creat a new socket using the given address family,socket type and protocol;
# if fileno is specified,the other arguments are ignored,return a descriptor;

addr = socket.getaddrinfo(host, port, family=0, type=0, proto=0, flags=0)
# 获取要连接的对端主机地址

sk.bind(address)
# 将套接字绑定到地址。address地址的格式取决于地址族;
# 在AF_INET下，以元组（host,port）的形式表示地址。

sk.listen(backlog)
# 开始监听传入连接
# backlog是一个整数，指在拒绝连接之前，可以挂起的最大连接数量；

sk.setblocking(bool)
# 是否设置为阻塞模式，默认为阻塞模式；
# 非阻塞模式，accept的时候不会停下等待；
# 但是如果accpet和recv异常的话，会触发BlockingIOError报错；

sk.accept()
# 接受连接并返回（conn,address）,其中conn是新的套接字对象，可以用来接收和发送数据;
# 接收TCP 客户的连接（阻塞式）等待连接的到来;

sk.connect(address)
# 连接到address处的套接字。一般，address的格式为元组（hostname,port）
# 如果连接出错，返回socket.error错误。

sk.connect_ex(address)
# 同上，只不过会有返回值，连接成功时返回 0 ，连接失败时候返回编码，例如：10061

sk.close()
# 关闭套接字

sk.recv(bufsize[,flag])
# 接受套接字的数据。数据以字符串形式返回，bufsize指定最多可以接收的数量。
# flag提供有关消息的其他信息，通常可以忽略。

sk.recvfrom(bufsize[.flag])
# 与recv()类似，但返回值是（data,address）。
# 其中data是包含接收数据的字符串，address是发送数据的套接字地址。

sk.send(string[,flag])
# 将string中的数据发送到连接的套接字。返回值是要发送的字节数量，
# 该数量可能小于string的字节大小。即：可能未将指定内容全部发送。

sk.sendall(string[,flag])
# 将string中的数据发送到连接的套接字;
# 但在返回之前会尝试发送所有数据。成功返回None，失败则抛出异常。
# 内部通过递归调用send，将所有内容发送出去。

sk.sendto(string[,flag],address)
# 将数据发送到套接字，address是形式为（ipaddr，port）的元组，指定远程地址。
# 返回值是发送的字节数。该函数主要用于UDP协议。

sk.settimeout(timeout)
# 设置套接字操作的超时期，timeout是一个浮点数，单位是秒。值为None表示没有超时期。
# 一般，超时期应该在刚创建套接字时设置，因为它们可能用于连接的操作;
# 比如因为写原因与数据的链接未建立，那么等待timeout时间后，即不再等待，返回timeout；

sk.getpeername()
# 返回连接套接字的远程地址。返回值通常是元组（ipaddr,port）。

sk.getsockname()
# 返回套接字自己的地址。通常是一个元组(ipaddr,port)

sk.fileno()
# 套接字的文件描述符

socket.sendfile(file, offset=0, count=None)
#  发送文件 ，但目前多数情况下并无什么卵用。

Socket编程实例

最简单的客户端及服务端通信的实现

#####客户端的实现#########
# 导入socket模块
import socket 
# 声明socket类型，同时生成socket连接对象(括号内为地址簇及协议类型，默认即这两种)
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# 与server地址(一个包含ip及端口的tuple)建立连接
client.connect(('localhost', 9988))
# 循环的收发数据
while True:
    msg = input(">>:")
    if msg.startswith("quit"): break
    # 发送数据
    # 注意无法send空，空server端认为你还在发，send也是有限制的
    # send是有个缓冲区的,如果一次没发完的全存在着，下次发，发的是缓冲的；
    # 这个缓冲区是有个大小的,如果缓冲区被占满，下次会扩充...<>
    # sendall 循环发send
    client.send(msg.encode(encoding='utf-8')) # 
    # 收取数据
    # 定义收多少数据，收可以很大，但也有限制，recv是有限制的,官方建议小于8192bit
    data = client.recv(1024) 
    print(data)
client.close()

#####服务端的实现#############
# 导入socket模块
import socket

# 声明socket类型，同时生成socket连接对象(括号内为地址簇及协议类型，默认即这两种)
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# 绑定要监听的端口
server.bind(('localhost', 6969))

# 监听，同时定义最多可以挂起5个链接
server.listen(5)

# 等客户端连接,并记录哪个客户端拨入！返回标记，第一个是链接的标记<conn>，第二个是对方的地址<addr>就是字符串元祖
# conn就是客户端链接过来后，服务端为其生成的一个连接实例
conn, addr = server.accept()

#接收数据方式一<错误的方式>:
# 这种的问题点:你无法标记谁打的啊，难道只能联系一个么“？如果多个呢？
# data = server.recv(1024)
# print('recv', data)
# server.send(data.upper()) # 回复消息

# 接收数据方式二<正确的方式>：
data = conn.recv(8192)
print('revc-', data)
#但是send的缓冲区是有大小的，发送也是有字节的，32768字节；
#发送的时机有两个：1.缓冲区满了会发送，2.超时会发送<而send手动触发，强制超时>
conn.send(data.upper())

# 链接关闭
server.close()

服务端可以反复的和客户端进行通信<但一次只有一个>

import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 6969))
server.listen(5)
f = open("E:\shipin.flv", 'rb')
video = f.read()
print(len(video))
f.close()
# 通过循环的方式可以反复的建立链接
while True:
    conn, addr = server.accept()
    try:
        while True:
            # 通过循环的方式可以反复的收发消息
            data = conn.recv(1024)
            print(data)
            if not data:
                conn.close()
                break
            conn.sendall(video)
    except ConnectionResetError:
        continue
server.close()

简单模仿ssh服务端及客户端

####客户端#######
import socket

ssh_client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssh_client.connect(('localhost', 7788))

while True:
    comm = input(">>:")
    if not comm: continue
    comm = comm.encode('utf-8')
    ssh_client.send(comm)
    result_len = int(ssh_client.recv(1024).decode('utf-8'))
    ssh_client.send("准备好接受数据".encode(encoding='utf-8'))
    recive = b''
    while len(recive) != result_len:
        recive += ssh_client.recv(1024)

    print(recive.decode(encoding='utf-8'))

#######服务端########
import socket
import subprocess # 了解下subprocess程序
ssh_server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssh_server.bind(('localhost', 7788))
ssh_server.listen(5)
while True:
    conn, addr = ssh_server.accept()
    print("新连接-%s:%s" % (addr[0], addr[1]))
    try:
        while True:
            comm = conn.recv(1024)
            comm = comm.decode(encoding='utf-8')
            code, result = subprocess.getstatusoutput(comm)
            # result = os.popen(comm).read()
            result = result.encode(encoding='utf-8')
            result_len = str(len(result)).encode(encoding='utf-8')
            conn.send(result_len)

            client_ack = conn.recv(1024) # wait client to ack # 可以解决粘包的现象
            print("ack:%s" % client_ack.decode())
            # socket的粘包现象，偶尔会发送这种现象，紧挨着的send会被缓冲区合并成一条来发送
            # 解决办法：send及recv交替使用
            conn.send(result)                                   
    except ConnectionResetError:
        print("连接已重置.")
        continue
ssh_server.close()

简单模仿FTP服务端及客户端

import socket
import hashlib

ssh_client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssh_client.connect(('localhost', 9898))

# 向服务器发送信息或者文件
def f_send(sock_obj, info):

    if info:
        sock_obj.send(info.encode())
    else:
        print("发送不可为空；")


# 从文件服务器上接收文件
def f_recv(sock_obj, newfile):
    # 接收文件的大小&并发送ack
    f_size = int(sock_obj.recv(1024).decode())
    sock_obj.send("ack".encode())

    # 接收模块
    r_size = 0
    new_f_md5 = hashlib.md5()
    with open(newfile, 'wb') as f:
        while True:

            if f_size - r_size > 0:
                # 精确接收
                if f_size - r_size > 1024:
                    one_time_recv = sock_obj.recv(1024)
                    r_size += len(one_time_recv)
                else:
                    one_time_recv = sock_obj.recv(f_size-r_size)
                    r_size += len(one_time_recv)
                # 写入文件
                f.write(one_time_recv)
                # 累加计算MD5
                new_f_md5.update(one_time_recv)
                print("已接收:{}byte  总共{}byte".format(r_size, f_size))

            else:
                # 文件接收完后，接收原文件MD5及对比新文件的md5
                old_f_md5 = sock_obj.recv(1024)
                print(old_f_md5)
                print(new_f_md5.hexdigest())
                print("已接收:{}byte  总共{}byte".format(r_size, f_size))
                break


while True:
    comm = input(">>:")

    if comm:

        if comm.startswith("get"):
            print("从服务器下载文件：")
            path = comm.strip().split(" ")[1]
            f_send(ssh_client, path)
            f_recv(ssh_client, path+'.new')

        elif comm.startswith("put"):
            print("向服务器上传文件:")
            pass

        elif comm.startswith('quit'):
            ssh_client.close()
            print("已退出...")
            break

    else:
        continue

FTP客户端

import socket
import os
import hashlib

# socket的连接的建立

ftp_server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ftp_server.bind(('localhost', 9898))
ftp_server.listen()

while True:
    # 建立链接
    conn, addr = ftp_server.accept()
    print("新链接已建立>>")
    try:
        while True:

            # 接收命令
            file_path = conn.recv(1024).decode()

            if os.path.isfile(file_path):

                # 发送文件大小&并开始计算MD5
                f_md5 = hashlib.md5()
                f_size = os.stat(file_path).st_size
                conn.send(str(f_size).encode())
                conn.recv(1024)

                # 发送文件
                with open(file_path, 'rb') as f:
                    for line in f:
                        # 发送一行并累加计算MD5
                        conn.send(line)
                        f_md5.update(line)

                # 发送计算完成的文件MD5
                print(f_md5.hexdigest())
                conn.send(f_md5.hexdigest().encode())

            else:
                # 返回错误，无此文件
                print("服务器上无此文件；")
                # conn.send("wrong file path!".encode())

    except ConnectionResetError:
        print("链接重置！")
        continue

ftp_server.close()

FTP服务端

socket server介绍与应用

socket并不能多并发，只能支持一个用户，socketserver 简化了编写网络服务程序的任务，socketserver是socket的在封装。

共4种类型

class socketserver.TCPServer(server_address, RequestHandlerClass, bind_and_activate=True)
class socketserver.UDPServer(server_address, RequestHandlerClass, bind_and_activate=True)
class socketserver.UnixStreamServer(server_address, RequestHandlerClass, bind_and_activate=True)
class socketserver.UnixDatagramServer(server_address, RequestHandlerClass,bind_and_activate=True)

继承关系

+------------+
| BaseServer |
+------------+
      |
      v
+-----------+        +------------------+
| TCPServer |------->| UnixStreamServer |
+-----------+        +------------------+
      |
      v
+-----------+        +--------------------+
| UDPServer |------->| UnixDatagramServer |
+-----------+        +--------------------+

BaseServer下主要的方法

fileno() # 返回文件描述符，一般用不到；

handle_request() # 处理单个请求，一般不用；

serve_forever(poll_interval=0.5) #
#poll_interval 每0.5检测，是否给我发送shutdown信号；
#3.3之后，会自动调用service_action()去清楚一些僵尸子进程

shutdown() # 发送关闭信号

address_family # 即地址簇

socket_type # 协议类型

RequestHandlerClass #请求处理类

timeout # 在handle_request()中使用

handle()

详解如下：

class socketserver.BaseServer(server_address, RequestHandlerClass)
This is the superclass of all Server objects in the module. It defines the interface, given below, but does not implement most of the methods, which is done in subclasses. The two parameters are stored in the respective server_address and RequestHandlerClass attributes.

fileno()
Return an integer file descriptor for the socket on which the server is listening. This function is most commonly passed to selectors, to allow monitoring multiple servers in the same process.

handle_request()
Process a single request. This function calls the following methods in order: get_request(), verify_request(), and process_request(). If the user-provided handle() method of the handler class raises an exception, the server’s handle_error() method will be called. If no request is received within timeout seconds, handle_timeout() will be called and handle_request() will return.

serve_forever(poll_interval=0.5)
Handle requests until an explicit shutdown() request. Poll for shutdown every poll_interval seconds. Ignores the timeout attribute. It also calls service_actions(), which may be used by a subclass or mixin to provide actions specific to a given service. For example, the ForkingMixIn class uses service_actions() to clean up zombie child processes.

Changed in version 3.3: Added service_actions call to the serve_forever method.

service_actions()
This is called in the serve_forever() loop. This method can be overridden by subclasses or mixin classes to perform actions specific to a given service, such as cleanup actions.

New in version 3.3.

shutdown()
Tell the serve_forever() loop to stop and wait until it does.

server_close()
Clean up the server. May be overridden.

address_family
The family of protocols to which the server’s socket belongs. Common examples are socket.AF_INET and socket.AF_UNIX.

RequestHandlerClass
The user-provided request handler class; an instance of this class is created for each request.

server_address
The address on which the server is listening. The format of addresses varies depending on the protocol family; see the documentation for the socket module for details. For Internet protocols, this is a tuple containing a string giving the address, and an integer port number: ('127.0.0.1', 80), for example.

socket
The socket object on which the server will listen for incoming requests.

The server classes support the following class variables:

allow_reuse_address
Whether the server will allow the reuse of an address. This defaults to False, and can be set in subclasses to change the policy.

request_queue_size
The size of the request queue. If it takes a long time to process a single request, any requests that arrive while the server is busy are placed into a queue, up to request_queue_size requests. Once the queue is full, further requests from clients will get a “Connection denied” error. The default value is usually 5, but this can be overridden by subclasses.

socket_type
The type of socket used by the server; socket.SOCK_STREAM and socket.SOCK_DGRAM are two common values.

timeout
Timeout duration, measured in seconds, or None if no timeout is desired. If handle_request() receives no incoming requests within the timeout period, the handle_timeout() method is called.

There are various server methods that can be overridden by subclasses of base server classes like TCPServer; these methods aren’t useful to external users of the server object.

finish_request()
Actually processes the request by instantiating RequestHandlerClass and calling its handle() method.

get_request()
Must accept a request from the socket, and return a 2-tuple containing the new socket object to be used to communicate with the client, and the client’s address.

handle_error(request, client_address)
This function is called if the handle() method of a RequestHandlerClass instance raises an exception. The default action is to print the traceback to standard output and continue handling further requests.

handle_timeout()
This function is called when the timeout attribute has been set to a value other than None and the timeout period has passed with no requests being received. The default action for forking servers is to collect the status of any child processes that have exited, while in threading servers this method does nothing.

process_request(request, client_address)
Calls finish_request() to create an instance of the RequestHandlerClass. If desired, this function can create a new process or thread to handle the request; the ForkingMixIn and ThreadingMixIn classes do this.

server_activate()
Called by the server’s constructor to activate the server. The default behavior for a TCP server just invokes listen() on the server’s socket. May be overridden.

server_bind()
Called by the server’s constructor to bind the socket to the desired address. May be overridden.

verify_request(request, client_address)
Must return a Boolean value; if the value is True, the request will be processed, and if it’s False, the request will be denied. This function can be overridden to implement access controls for a server. The default implementation always returns True.

View Code

建立一个socket server的步骤：

第一步：创建一个 RequestHandlerClass ，这个class需要继承BaseRequestHandler类，重写父类里的handler()<跟客户端所有的交互>
第二步：实例化4个中的任意一个socketserver, 并且传入server_address，及上面创建的请求处理类；
第三步：调用上面实例中的handle_request()->处理一个请求退出，或者server.serve_forever()->处理多个请求，永远执行
第四步：Finally, call server_close() to close the socket.

Socket server多并发依赖于多进程or多线程

让你的socketserver并发起来，必须选择使用以下一个多并发的类

class socketserver.ForkingTCPServer
class socketserver.ForkingUDPServer
class socketserver.ThreadingTCPServer
class socketserver.ThreadingUDPServer

一个简单的实例

import socketserver

class MyTCPHandler(socketserver.BaseRequestHandler): # 每一个请求过来都会实例化，这个类


    def handle(self):
        # 更客户端所有的交互都是在handle中完成的，每一个请求过来处理都是handle来做的；
        # 谁调用的handle呢？可以在BaseRequestHandler中查看到：
        # 源码如下：
        #  self.setup() #在调用handle()之前
        #  try:
        #    self.handle() # 创建实例调用handle()
        #  finally:
        #    self.finish() # 调用之后的操作；

        try:
            while True:
                self.data = self.request.recv(1024).strip() # 必须是这么来收
                print("{} wrote:".format(self.client_address[0]))
                print(self.data)
                self.request.send(self.data.upper())
        except ConnectionResetError:
            print("[{}]链接已重置！！！".format(self.client_address))

print(__name__)
if __name__ == "__main__":
    # 单线程
    # mytcp = socketserver.TCPServer(('localhost', 9988), MyTCPHandler) 
    # 多进程，每来一个请求就开启一个新的进程；
    # mytcp = socketserver.ForkingTCPServer(('localhost', 9988), MyTCPHandler)
    # 多线程，每来一个请求就开启一个新的线程；
    mytcp = socketserver.ThreadingTCPServer(('localhost', 9988), MyTCPHandler) 
    mytcp.serve_forever()

end