New submission from Chaitanya <chaitanya.t...@gmail.com>:
This is similar to https://bugs.python.org/issue22393 but the child is killed in the Pool.terminate() using SIGTERM i.e., proper way. Run the attached script that runs an indefinite method using multiprocessing.apply_async, and in the signal handler for SIGTERM does the pool clean up (close, terminate and join), on issuing a SIGTERM (`kill <parent PID>`), the cleanup works fine, but the parent process is stuck in get() as during termination the event is not set. Able to repro with all 3.6/3.9/3.10. See logs below ``` [DEBUG/MainProcess] created semlock with handle 140035884298240 [DEBUG/MainProcess] created semlock with handle 140035884294144 [DEBUG/MainProcess] created semlock with handle 140035884290048 [DEBUG/MainProcess] created semlock with handle 140035884285952 [DEBUG/MainProcess] created semlock with handle 140035884281856 [DEBUG/MainProcess] created semlock with handle 140035884277760 [DEBUG/MainProcess] added worker [INFO/ForkPoolWorker-1] child process calling self.run() [INFO/MainProcess] <function close_thread_pools at 0x7f5ca3a20b80> [INFO/MainProcess] <function close_thread_pools at 0x7f5ca3a20b80> [INFO/MainProcess] <function close_thread_pools at 0x7f5ca3a20b80> [INFO/ForkPoolWorker-1] Signals for 28462 [INFO/ForkPoolWorker-1] Handlers.SIG_DFL [INFO/ForkPoolWorker-1] <built-in function default_int_handler> [INFO/ForkPoolWorker-1] Handlers.SIG_DFL [INFO/MainProcess] 28461: Caught signal 15 [INFO/MainProcess] 28461: Closing pools [INFO/MainProcess] 28461: Terminating pools [DEBUG/MainProcess] terminating pool [DEBUG/MainProcess] finalizing pool [DEBUG/MainProcess] helping task handler/workers to finish [DEBUG/MainProcess] removing tasks from inqueue until task handler finished [DEBUG/MainProcess] joining worker handler [DEBUG/MainProcess] worker handler exiting [DEBUG/MainProcess] result handler found thread._state=TERMINATE [DEBUG/MainProcess] ensuring that outqueue is not full [DEBUG/MainProcess] result handler exiting: len(cache)=1, thread._state=TERMINATE [DEBUG/MainProcess] task handler got sentinel [DEBUG/MainProcess] task handler sending sentinel to result handler [DEBUG/MainProcess] task handler sending sentinel to workers [DEBUG/MainProcess] task handler exiting [DEBUG/MainProcess] terminating workers [DEBUG/MainProcess] joining task handler [DEBUG/MainProcess] joining result handler [DEBUG/MainProcess] joining pool workers [DEBUG/MainProcess] cleaning up worker 28462 [INFO/MainProcess] 28461: Joining pools [DEBUG/MainProcess] joining pool [INFO/MainProcess] 28461: Closed pools ``` ``` (gdb) py-bt Traceback (most recent call first): File "/usr/lib/python3.9/threading.py", line 312, in wait waiter.acquire() File "/usr/lib/python3.9/threading.py", line 574, in wait signaled = self._cond.wait(timeout) File "/usr/lib/python3.9/multiprocessing/pool.py", line 762, in wait self._event.wait(timeout) File "/usr/lib/python3.9/multiprocessing/pool.py", line 765, in get self.wait(timeout) File "/home/tatac/workspaces/gerrit/bwt-device-health-manager/mp_test.py", line 66, in <module> a.get() (gdb) ``` ---------- components: Library (Lib) files: mp_test.py messages: 396151 nosy: tkc17 priority: normal severity: normal status: open title: multiprocessing.get hangs if the pool is closed in the signal handler type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50121/mp_test.py _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue44462> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com