New submission from Daniel Grunwald <dan...@danielgrunwald.de>:
Running the attached script deadlocks. Uncommenting the `time.sleep(1)` in the script makes the deadlock disappear. For context: our application uses multiple child processes (multiprocessing.Process) and uses pipes (multiprocessing.Pipe) to communicate with them. If one process fails with an error, the main process will kill all other child processes running concurrently. We noticed that sometimes (non-deterministically), when an error occurs soon after startup, the main process ends up hanging. We expect that when we pass the writing half of a connection to a child process and close the connection in the main process, that we will receive EOFError if the child process terminates unexpectedly. But sometimes the EOFError never comes and our application hangs. I've reduced the problem to the script attached. With the reduced script, the deadlock happens reliably for me. I've debugged this a bit, and I think this is happening because passing a connection to the process being started involves reduce_pipe_connection() which creates a copy of the handle within the main process. When the pickled data is unpickled in the child process, it uses DUPLICATE_CLOSE_SOURCE to close the copy in the main process. But if the pickled data is never unpickled by the child process, the handle ends up being leaked. Thus the main process itself holds the writing half of the connection open, causing the recv() call on the reading half to block forever. ---------- components: Windows files: deadlock.py messages: 385283 nosy: dgrunwald, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: multiprocessing handle leak on Windows when child process is killed during startup/unpickling type: behavior versions: Python 3.7, Python 3.8, Python 3.9 Added file: https://bugs.python.org/file49751/deadlock.py _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue42968> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
