On Fri, Aug 26, 2011 at 3:18 AM, Nir Aides <n...@winpdb.org> wrote: > Another face of the discussion is about whether to deprecate the mixing of > the threading and processing modules and what to do about the > multiprocessing module which is implemented with worker threads.
There's a bug open - http://bugs.python.org/issue8713 which would offer non windows users the ability to avoid using fork() entirely, which would sidestep the problem outlined in the atfork() bug. Under windows, which has no fork() mechanism, we create a subprocess and then use pipes for intercommunication: nothing is inherited from the parent process except the state passed into the child. I think that "deprecating" the use of threads w/ multiprocessing - or at least crippling it is the wrong answer. Multiprocessing needs the helper threads it uses internally to manage queues, etc. Removing that ability would require a near-total rewrite, which is just a non-starter. I'd rather examine bug 8713 more closely, and offer this option for all users in 3.x and document the existing issues outlined in http://bugs.python.org/issue6721 for 2.x - the proposals in that bug are IMHO, out of bounds for a 2.x release. In essence; the issue here is multiprocessing's use of fork on unix without the following exec - which is what the windows implementation essentially does using subprocess. Adding the option to *not* fork changes the fundamental behavior on unix systems - but I fundamentally feel that it's a saner, and more consistent behavior for the module as a whole. So, I'd ask that we not talk about tearing out the ability to use MP and threads, or threads with MP - that would be crippling, and there's existing code in the wild (including multiprocessing itself) that uses this mix without issue - it's stripping out functionality for what is a surprising and painful edge case that rarely directly affects users. I would focus on the atfork() patch more directly, ignoring multiprocessing in the discussion, and focusing on the merits of gps' initial proposal and patch. jesse _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
