[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Roundup Robot added the comment: New changeset 751371dd4d1c by Richard Oudkerk in branch '2.7': Issue #14548: Make multiprocessing finalizers check pid before http://hg.python.org/cpython/rev/751371dd4d1c -- ___ Python tracker

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Changes by Richard Oudkerk : -- resolution: -> fixed stage: commit review -> committed/rejected status: open -> closed ___ Python tracker ___ ___

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Roundup Robot added the comment: New changeset 59567c117b0e by Richard Oudkerk in branch 'default': Issue #14548: Make multiprocessing finalizers check pid before running http://hg.python.org/cpython/rev/59567c117b0e -- nosy: +python-dev ___ Python t

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Charles-François Natali added the comment: > Alternative patch which records pid when Finalize object is created. Looks good to me. Note that it could eventually be rewritten to use an atfork() module, when we finally merge your patch for this other issue :-) -- stage: -> commit revi

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

sbt added the comment: > But what if Finalize is used to cleanup a resource that gets > duplicated in children, like a file descriptor? > See e.g. forking.py, line 137 (in Popen.__init__()) > or heap.py, line 244 (BufferWrapper.__init__()). This was how Finalize objects already acted (or were

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Antoine Pitrou added the comment: But what if Finalize is used to cleanup a resource that gets duplicated in children, like a file descriptor? See e.g. forking.py, line 137 (in Popen.__init__()) or heap.py, line 244 (BufferWrapper.__init__()). -- __

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

sbt added the comment: Alternative patch which records pid when Finalize object is created. The callback does nothing if recorded pid does not match os.getpid(). -- Added file: http://bugs.python.org/file25195/mp_finalize_pid.patch ___ Python track

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Charles-François Natali added the comment: >> That's a problem indeed. Perhaps we need a global "fork lock" shared >> between subprocess and multiprocessing? > > I did an atfork patch which included a (recursive) fork lock.  See > >    http://bugs.python.org/review/6721/show > > The patch includ

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

sbt added the comment: > That's a problem indeed. Perhaps we need a global "fork lock" shared > between subprocess and multiprocessing? I did an atfork patch which included a (recursive) fork lock. See http://bugs.python.org/review/6721/show The patch included changes to multiprocessing

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Antoine Pitrou added the comment: > Hmm... > I don't really like disabling GC, because it has a process-wide side > effect, and hence isn't thread-safe: if another thread forks() or > creates a subprocess right at the wrong time, it could end up with the > GC disabled for good... That's a probl

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Charles-François Natali added the comment: Hmm... I don't really like disabling GC, because it has a process-wide side effect, and hence isn't thread-safe: if another thread forks() or creates a subprocess right at the wrong time, it could end up with the GC disabled for good... -- ___

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Antoine Pitrou added the comment: Shouldn't there be a try..finally, in case os.fork() fails? -- ___ Python tracker ___ ___ Python-bu

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

sbt added the comment: Patch to disable gc. -- keywords: +patch Added file: http://bugs.python.org/file25180/mp_disable_gc.patch ___ Python tracker ___ _

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

Antoine Pitrou added the comment: > Disabling gc during fork seems to prevent the errors. Sounds reasonable to me. -- components: +Library (Lib) nosy: +neologix, pitrou type: -> behavior versions: +Python 2.7, Python 3.2, Python 3.3 ___ Python trac

[issue14548] garbage collection just after multiprocessing's fork causes exceptions

New submission from sbt : When running test_multiprocessing on Linux I occasionally see a stream of errors caused by ignored weakref callbacks: Exception AssertionError: AssertionError() in ignored These do not cause the unittests to fail. Finalizers from the parent process are supposed to