多进程在CPU密集型任务中性能优于多线程,因GIL限制多线程并行;而多线程在IO密集型任务中表现良好,适合高并发等待场景。
在Python中,多线程和多进程是实现并发编程的两种常见方式。但由于GIL(全局解释器锁)的存在,多线程在CPU密集型任务中表现不佳,而多进程则能真正利用多核优势。下面通过实际测试对比两者的性能差异。
为了公平比较,我们设定两个典型任务:
分别用单线程、多线程、多进程执行,记录耗时。
代码示例:
import threading import multiprocessing import timedef cpu_task(n): return sum(i * i for i in range(n))
def single_threadcpu(n, loops): for in range(loops): cpu_task(n)
def multi_threadcpu(n, loops, threads=4): def worker(): for in range(loops // threads): cpu_task(n) threadslist = [threading.Thread(target=worker) for in range(threads)] for t in threads_list: t.start() for t in threads_list: t.join()
def multi_process_cpu(n, loops, processes=4): with multiprocessing.Pool(processes) as pool: pool.map(cpu_task, [n] * loops)
测试参数
n = 10000 loops = 20
单线程
start = time.time() single_thread_cpu(n, loops) print(f"单线程耗时: {time.time() - start:.2f}s")
多线程
start = time.time() multi_thread_cpu(n, loops) print(f"多线程耗时: {time.time() - start:.2f}s")
多进程
start = time.time() multi_process_cpu(n,
loops) print(f"多进程耗时: {time.time() - start:.2f}s")
结果分析:
模拟IO操作(如网络请求):
import time import threading import multiprocessingdef io_task(seconds): time.sleep(seconds)
def single_threadio(loops, sec=0.1): for in range(loops): io_task(sec)
def multi_threadio(loops, sec=0.1, threads=4): def worker(): for in range(loops // threads): io_task(sec) threadslist = [threading.Thread(target=worker) for in range(threads)] for t in threads_list: t.start() for t in threads_list: t.join()
def multi_process_io(loops, sec=0.1, processes=4): with multiprocessing.Pool(processes) as pool: pool.map(io_task, [sec] * loops)
测试参数
loops = 40 sec = 0.1
单线程
start = time.time() single_thread_io(loops, sec) print(f"IO-单线程耗时: {time.time() - start:.2f}s")
多线程
start = time.time() multi_thread_io(loops, sec) print(f"IO-多线程耗时: {time.time() - start:.2f}s")
多进程
start = time.time() multi_process_io(loops, sec) print(f"IO-多进程耗时: {time.time() - start:.2f}s")
结果分析:
根据测试结果得出以下结论:
基本上就这些。选择哪种方式,关键看任务类型。理解GIL的影响,才能写出高效的Python并发程序。