[issue10332] Multiprocessing maxtasksperchild results in hang

Charles-François Natali added the comment: James, thanks for the report! -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker ___

[issue10332] Multiprocessing maxtasksperchild results in hang

Roundup Robot added the comment: New changeset 3465a9b2d25c by Charles-François Natali in branch '2.7': Issue #10332: multiprocessing: fix a race condition when a Pool is closed http://hg.python.org/cpython/rev/3465a9b2d25c New changeset 52c98a729a71 by Charles-François Natali in branch '3.2':

[issue10332] Multiprocessing maxtasksperchild results in hang

Antoine Pitrou added the comment: The patch looks good to me, thanks. -- versions: +Python 3.3 -Python 3.1 ___ Python tracker ___ ___

[issue10332] Multiprocessing maxtasksperchild results in hang

Changes by Charles-François Natali : Removed file: http://bugs.python.org/file21644/pool_lifetime_close.diff ___ Python tracker ___ ___ Python

[issue10332] Multiprocessing maxtasksperchild results in hang

Charles-François Natali added the comment: Here's an updated patch. I'll open a separate issue for the thread-safety. -- keywords: +needs review nosy: +pitrou stage: -> patch review Added file: http://bugs.python.org/file23489/pool_lifetime_close-1.diff ___

[issue10332] Multiprocessing maxtasksperchild results in hang

Jesse Noller added the comment: > Note: I noticed that there are some thread-unsafe operations (the cache that > can be modified from different threads, and thread states are modified also > from different threads). While this isn't an issue with the current cPython > implementation (GIL), I

[issue10332] Multiprocessing maxtasksperchild results in hang

Charles-Francois Natali added the comment: This problem arises because the pool's close method is called before all the tasks have completed. Putting a sleep(1) before pool.close() won't exhibit this lockup. The root cause is that close makes the workers handler thread exit: since the maxtask

[issue10332] Multiprocessing maxtasksperchild results in hang

Jesse Keating added the comment: I can duplicate this using python-2.7-8.fc14.1 on Fedora 14, and using map_async with the pool. -- nosy: +jkeating ___ Python tracker ___ _

[issue10332] Multiprocessing maxtasksperchild results in hang

Changes by Antoine Pitrou : -- nosy: +asksol, jnoller type: -> behavior versions: +Python 2.7, Python 3.1 ___ Python tracker ___ ___

[issue10332] Multiprocessing maxtasksperchild results in hang

New submission from James Hutchison : v.3.2a3 If the maxtasksperchild argument is used, the program will just hang after whatever that value is rather than working as expected. Tested in Windows XP 32-bit test code: import multiprocessing def f(x): return 0; if __name__ == '__main__':