python多进程和进程池

image description

python多进程和进程池

写在最前面:

linux下可使用 fork 函数

通常使用 multiprocessing更常见

我们分别使用单进程和多进程处理run函数

import time,os
from multiprocessing import Pool


def run(n):
  time.sleep(1)
  print('Run child process %s (%s)...' % (n*n, os.getpid()))


if __name__ == "__main__":
  testFL = [1,2,3,4,5,6]
  print('顺序执行:')    # 顺序执行(也就是串行执行,单进程)
  starttime = time.time()
  for n in testFL:
    run(n)
  t = time.time()
  print("顺序执行时间:", int(t - starttime))

  print ('concurrent:') # 创建多个进程,并行执行
  pool = Pool(10)       # 创建拥有10个进程数量的进程池
  pool.map(run, testFL)
  pool.close()          # 关闭进程池,不再接受新的进程
  pool.join()           # 主进程阻塞等待子进程的退出
  endtime = time.time()
  print("并行执行时间:", int(endtime-t))

执行结果如下:

顺序执行:

Run child process 1 (32669)...
Run child process 4 (32669)...
Run child process 9 (32669)...
Run child process 16 (32669)...
Run child process 25 (32669)...
Run child process 36 (32669)...
顺序执行时间: 6
concurrent:
Run child process 4 (32671)...
Run child process 1 (32670)...
Run child process 16 (32673)...
Run child process 9 (32672)...
Run child process 25 (32674)...
Run child process 36 (32675)...
并行执行时间: 1

Process finished with exit code 0

如果不用map,一般采用apply_async(func[, args[, kwds[, callback]]]),这里是非阻塞的且支持结果返回后进行回调,

而apply用于传递不定参数,同python中的apply函数一致(不过内置的apply函数从2.3以后就不建议使用了),主进程会阻塞于函数。

import time,os
from multiprocessing import Pool


def run(n):
  time.sleep(1)
  print('Run child process %s (%s)...' % (n*n, os.getpid()))


if __name__ == "__main__":
  testFL = [1,2,3,4,5,6]
  print('顺序执行:')    # 顺序执行(也就是串行执行,单进程)
  starttime = time.time()
  for n in testFL:
    run(n)
  t = time.time()
  print("顺序执行时间:", int(t - starttime))

  print ('concurrent:') # 创建多个进程,并行执行
  pool = Pool(10)       # 创建拥有10个进程数量的进程池
  for i in range(1,7):
      pool.apply_async(run,(i,))
  pool.close()          # 关闭进程池,不再接受新的进程
  pool.join()           # 主进程阻塞等待子进程的退出
  endtime = time.time()
  print("并行执行时间:", int(endtime-t))

执行结果如下:

顺序执行:

Run child process 1 (32880)...
Run child process 4 (32880)...
Run child process 9 (32880)...
Run child process 16 (32880)...
Run child process 25 (32880)...
Run child process 36 (32880)...
顺序执行时间: 6
concurrent:
Run child process 9 (32885)...
Run child process 1 (32883)...
Run child process 25 (32887)...
Run child process 16 (32886)...
Run child process 4 (32884)...
Run child process 36 (32888)...
并行执行时间: 1

Process finished with exit code 0
    ArithmeticJia         0         1399         Python         15    

David Ramon

ArithmeticJia

www.guanacossj.com

Life is Short,You need Python

Related Posts

You may like these post too

image description

python中的GIL锁

为什么我们说python中无法实现真正的多线程呢,这是因为在C语言写的python解释器中存在全局解释器锁,由于全局解释器锁的存在,在同一时间内,python解释器只能运行一个线程的代码,这大大影响了python多线程的性能。而这个解释器锁由于历史原因,现在几乎无法消除。 python GIL

image description

leetcode【867】Transpose

##写在最前面: 很常规的一道二叉树的基本操作,最偷懒的办法是递归,不需要想那么多。 给定一个二叉树,返回它的中序 遍历 示例: ``` 输入: [1,null,2,3] 1 \ 2 / 3 输出: [1,3,2] ```

Comments on this post

0 comments

Leave a comment

it’s easy to post a comment

image description
image description
image description
image description
image description
image description
image description
image description
image description

Copyright © 2019.Company name All rights reserved.苏ICP备19007197号