多进程

1、启动一个新的进程

其实启动一个新的进程和启动一个线程的格式是一样的,只不过启动进程是需要从multiprocessing中导入Process(注意此处的Process首字母是大写的)

格式如下:

from multiprocessing import Process

def f(name):
    print('The f is running !')

if __name__ == '__main__':
    p = Process(target=f,args=('balabala',))
    p.start()

2、进程间的通信

由于进程是资源的集合,所以进程之间是不能相互访问的,是独立的。

2.1 Queue 其实用法和线程中的队列是一样的,需要从multiprocessing 导入Queue

from multiprocessing import Queue,Process
import os

def f(t):
    t.put(['孙悟空','美猴王',500])
    print('The current PID f:',os.getppid(),os.getpid())

if __name__ == '__main__':
    q = Queue()
    print('The current PID:', os.getpid())
    p = Process(target=f,args=(q,))
    p.start()
    print('The data is :',q.get())

运行结果为:

The current PID: 8748
The current PID f: 8748 15712
The data is : ['孙悟空', '美猴王', 500]
可以看出,这是两个进程,进程之间的通信和线程中的队列是一个道理,一个进程把数据放到队列中,另一个程序取队列里边取数据,相当于通过中间的池子作为联系的纽带!!

2.2通过Pipes管道通讯

from multiprocessing import Pipe,Process

def f(t):
    t.send(['孙悟空','美猴王',1000])
    t.close()

if __name__ == '__main__':
    one_hand,other_hand = Pipe()
    p = Process(target=f,args=(other_hand,))
    p.start()
    print('The one_hand receive data:',one_hand.recv())
    p.join()

这个模型等价于生成一个管子,两个进程一端一个,一段发送,另一端接受,至于哪个程序在那一段无所谓,都可以!!当然子线程发送给,父进程接受一样也是可以的!!

2.3Manager代理者

from multiprocessing import Manager,Process
import os
def f(f_dict,f_list):
    f_dict[1] = 'a'
    f_dict[2] = 'b'
    f_dict[3] = 'c'
    f_dict[os.getpid()] = os.getpid()
    print('f_list:',f_list)
    print('f_dict:',f_dict)

if __name__ == '__main__':
    manager = Manager()
    m_dict = manager.dict()
    m_list = manager.list(range(5))
    p_list = []
    for i in range(10):
        p = Process(target=f,args=(m_dict,m_list))
        p.start()
        p_list.append(p)
    for p in p_list:
        p.join()

其实Manager和Queue效果是一样的,只不过Queue只是生成一种队列,也就是一种类型的池子,而Manager可以生成多种不同类型的池子,可以在池子里放的类型更多了。道理是一样的!!

3、进程同步

from multiprocessing import Process,Lock,Pipe
import os,time
def f(lock,an,n):
    lock.acquire()
    print('The recv data {n}:'.format(n = n),an.recv())
    lock.release()
if __name__ == '__main__':
    lock = Lock()
    one, another = Pipe()

    for i in range(10):
        one.send('伟大的祖国!!')
        p = Process(target=f,args=(lock,another,i))
        p.start()

虽然不同进程之间的数据是相互独立的,但是比如不同进程在想屏幕上输出数据是,他们是公用的一块屏幕啊,这就有可能第一个进程在向屏幕输出数据还没输出完毕时,第二个进程就开始向屏幕输出数据了,这时就有可能是进程在向屏幕输出数据是出现乱了。

4、进程池

进程池需要导入multiprocessing中的Pool

进程池有两种方法一种是apply也就是串行,另一种是apply_async,这种是并行,例如运行100个进程,但是pool的数据为5,那意思就是5个进程并行运行,运行结束后和下边的5个并行进程串行运行。

from multiprocessing import Process,Pool
import os,time
def f(a):
    time.sleep(2)
    print('The current data:',a)
    return a + 100

def back(ar):
    print('The main func has runned,the back func is running!!',ar)

if __name__ == '__main__':
    pool = Pool(3)
    for i in range(10):
        # pool.apply(func = f,args = (i,))
        pool.apply_async(func=f, args=(i,), callback=back)  //callback回调函数的参数为func调用函数的返回值,如果没有返回值,则程序出错

    print('End!!')
    pool.close()   //这里要注意pool先关闭,然后再等待进程运行,否则进程直接关闭。
    pool.join()

运行结果如下:

End!!
The current data: 0
The main func has runned,the back func is running!! 100         //callback函数的参数为func函数的返回值
The current data: 1
The current data: 2
The main func has runned,the back func is running!! 101
The main func has runned,the back func is running!! 102
The current data: 3
The main func has runned,the back func is running!! 103
The current data: 5
The current data: 4
The main func has runned,the back func is running!! 105
The main func has runned,the back func is running!! 104
The current data: 6
The main func has runned,the back func is running!! 106
The current data: 7
The current data: 8
The main func has runned,the back func is running!! 108
The main func has runned,the back func is running!! 107
The current data: 9
The main func has runned,the back func is running!! 109

Process finished with exit code 0

原文地址:https://www.cnblogs.com/zhouzhe-blog/p/9435548.html