[issue24484] multiprocessing cleanup occasionally throws exception

Antoine Pitrou added the comment: This should be fixed now. Please reopen if not. -- resolution: -> fixed stage: commit review -> resolved status: open -> closed ___ Python tracker

[issue24484] multiprocessing cleanup occasionally throws exception

Antoine Pitrou added the comment: New changeset a0ecaab006849ade08fbcf8413fe1e92d9c8d71c by Antoine Pitrou in branch '3.5': [3.5] bpo-24484: Avoid race condition in multiprocessing cleanup (GH-2159) (#2167) https://github.com/python/cpython/commit/a0ecaab006849ade08fbcf8413fe1e92d9c8d71c

[issue24484] multiprocessing cleanup occasionally throws exception

Antoine Pitrou added the comment: New changeset d09f1674d70dd84c5be37b5fe11bca4913146c6b by Antoine Pitrou in branch '2.7': [2.7] bpo-24484: Avoid race condition in multiprocessing cleanup (GH-2159) (#2168) https://github.com/python/cpython/commit/d09f1674d70dd84c5be37b5fe11bca4913146c6b

[issue24484] multiprocessing cleanup occasionally throws exception

Antoine Pitrou added the comment: New changeset 6fd03459957ee53941183212457bba19f977679f by Antoine Pitrou in branch '3.6': [3.6] bpo-24484: Avoid race condition in multiprocessing cleanup (GH-2159) (#2166) https://github.com/python/cpython/commit/6fd03459957ee53941183212457bba19f977679f

[issue24484] multiprocessing cleanup occasionally throws exception

Antoine Pitrou added the comment: Python 2.7 doesn't suffer from this bug, so no actual fix is needed. Still, I'll backport the additional test. -- stage: needs patch -> commit review versions: -Python 2.7 ___ Python tracker

[issue24484] multiprocessing cleanup occasionally throws exception

Changes by Antoine Pitrou : -- pull_requests: +2219 ___ Python tracker ___ ___

[issue24484] multiprocessing cleanup occasionally throws exception

Changes by Antoine Pitrou : -- pull_requests: +2218 ___ Python tracker ___ ___

[issue24484] multiprocessing cleanup occasionally throws exception

Changes by Antoine Pitrou : -- pull_requests: +2217 ___ Python tracker ___ ___

[issue24484] multiprocessing cleanup occasionally throws exception

Antoine Pitrou added the comment: New changeset 1eb6c0074d17f4fd425cacfdda893d65f5f77f0a by Antoine Pitrou in branch 'master': bpo-24484: Avoid race condition in multiprocessing cleanup (#2159) https://github.com/python/cpython/commit/1eb6c0074d17f4fd425cacfdda893d65f5f77f0a --

[issue24484] multiprocessing cleanup occasionally throws exception

Changes by Antoine Pitrou : -- pull_requests: +2210 ___ Python tracker ___ ___

[issue24484] multiprocessing cleanup occasionally throws exception

Antoine Pitrou added the comment: Ok, I got to write a reproducer, which fails reliably here. -- Added file: http://bugs.python.org/file46946/finalizestress.py ___ Python tracker

[issue24484] multiprocessing cleanup occasionally throws exception

Antoine Pitrou added the comment: Marcin, Jorge, Ned, could one of you apply the following simple patch and see if that seems to solve the issue? -- keywords: +patch Added file: http://bugs.python.org/file46944/run_finalizers.patch ___ Python

[issue24484] multiprocessing cleanup occasionally throws exception

Antoine Pitrou added the comment: The simple answer here is _run_finalizers() is not thread-safe, but finalizers can be registered or cancelled from any thread. I could not write a simple synthetic script to reproduce the issue, but it should nevertheless be fixable. -- nosy: +pitrou

[issue24484] multiprocessing cleanup occasionally throws exception

Marcin Słowik added the comment: I can confirm, I have managed to accidentally reproduce this issue on a custom build of 3.6.0 on RHEL. sys.version: '3.6.0 (default, Jan 18 2017, 11:23:11) [GCC 4.4.7 20120313 (Red Hat 4.4.7-16)]' The issue popped out of nowhere, on a code that was working

[issue24484] multiprocessing cleanup occasionally throws exception

Jorge Herskovic added the comment: Ok, I *think* I tracked it down to Connections getting cleaned up in a non-orderly fashion, sometimes while _run_finalizers is executing. The following patch to lib/python3.4/multiprocessing/connection.py cures the problem on my machine. @@ -124,10 +124,10

[issue24484] multiprocessing cleanup occasionally throws exception

Jorge Herskovic added the comment: 20,000+ iterations of the test suite with my homebuilt 3.4.3 throw no exceptions. Weirder and weirder. Any suggestions? Without any actual knowledge (unencumbered by the thought process), a concurrency issue in the interpreter itself is my guess.

[issue24484] multiprocessing cleanup occasionally throws exception

Jorge Herskovic added the comment: Finally able to repro on my old Mac Pro at work (Universal 64/32bit 3.4.3 from python.org, OS X 10.10.3). For some reason I can't quite fathom, there were two exceptions this time. Error in atexit._run_exitfuncs: Traceback (most recent call last): File

[issue24484] multiprocessing cleanup occasionally throws exception

Jorge Herskovic added the comment: Ok, sorry, one more important thing I omitted in the previous comment. ^^^ The second exception above seems to be related to a new test I added, one that includes a daemonic process which itself relies on daemonic threads. The cleanup of that seems to be

[issue24484] multiprocessing cleanup occasionally throws exception

Jorge Herskovic added the comment: More data: I've been unable to replicate this behavior on Ubuntu 15.04 on Python 3.4.3 over ~20 thousand test runs. (Same machine, running in a VM) An overnight repeated run on the original machine on OS X, 3.4.3, official distribution gave an actual

[issue24484] multiprocessing cleanup occasionally throws exception

Ned Deily added the comment: Exactly how are you building the Python 3.4 that does not fail for you? FWIW, I am able to reproduce the failure easily on a 10.10.3 system using python.org 64-bit/32-bit 3.4.3, using a current MacPorts source-built 3.4.3 (which is built with clang), and

[issue24484] multiprocessing cleanup occasionally throws exception

Jorge Herskovic added the comment: The failing Python: 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] The non-failing Python: 3.4.3 (default, Jun 23 2015, 06:33:02) [GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] The non-failing python,

[issue24484] multiprocessing cleanup occasionally throws exception

Ned Deily added the comment: FWIW, building 3.4.3 from source as you described results in failures about 30% of the time. With some of the earlier tests, it failed even more frequently. This is running on a 2.2GHz I7. -- ___ Python tracker

[issue24484] multiprocessing cleanup occasionally throws exception

Ned Deily added the comment: Can you show the version info from the Python 3.4 where you see the failures? /usr/local/bin/python3.4 -c 'import sys;print(sys.version)' -- nosy: +ned.deily ___ Python tracker rep...@bugs.python.org

[issue24484] multiprocessing cleanup occasionally throws exception

Jorge Herskovic added the comment: This happens reliably between 1-5% of the time on my home Mac (a 4.0 GHz i7). My work Mac Pro, a lot slower, doesn't exhibit this behavior on the same 3.4.3 interpreter. Could it be related to a concurrency issue in the interpreter? We're attempting to

[issue24484] multiprocessing cleanup occasionally throws exception

New submission from Jorge Herskovic: I'm writing a library on top of multiprocessing. As part of the test suite, I create and destroy dozens of processes repeatedly. About once in 50 runs, the tests complete successfully but the program crashes with: Error in atexit._run_exitfuncs: Traceback

[issue24484] multiprocessing cleanup occasionally throws exception

Changes by Ned Deily n...@acm.org: -- nosy: +davin, sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24484 ___ ___ Python-bugs-list mailing list