New submission from mrqianjinsi <mrqianji...@gmail.com>:
Hi guys.
I'm using multiprocessing module to accelerate my program. and I want to do
some cleanup work when sub-process exit.
but I found that sub-process would be terminated when registered finalizer are
working.
Is this behavior designed intentionally?
code example:
import multiprocessing as mp
from multiprocessing.util import Finalize
import os
import time
def finalizer():
time.sleep(0.2) # some time consuming work
print('do cleanup work: {}'.format(os.getpid()))
def worker(_):
print('do some work: {}'.format(os.getpid()))
def initializer():
# ref:
https://github.com/python/cpython/blob/master/Lib/multiprocessing/util.py#L147
Finalize(None, finalizer, exitpriority=1)
# atexit module don't work along with multiprocessing module
# because sub-process exit via os._exit()
# ref: https://docs.python.org/3/library/atexit.html
# atexit.register(finalizer) # don't work
print('main process ID: {}'.format(os.getpid()))
with mp.Pool(4, initializer=initializer) as executor:
executor.map(worker, range(20))
gist link: https://gist.github.com/MrQianJinSi/2daf5b6a9ef08b00facdfbea5200dd28
----------
components: Library (Lib)
messages: 345310
nosy: mrqianjinsi
priority: normal
severity: normal
status: open
title: sub-process would be terminated when registered finalizer are working
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue37242>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
