[issue1856] shutdown (exit) can hang or segfault with daemon threads running

STINNER Victor added the comment: bpo-1193099 was marked as a duplicate of this issue. -- ___ Python tracker ___ ___

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Roundup Robot added the comment: New changeset 4ceca79d1c63 by Antoine Pitrou in branch '2.7': Issue #21963: backout issue #1856 patch (avoid crashes and lockups when https://hg.python.org/cpython/rev/4ceca79d1c63 -- ___ Python tracker

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Matthias Klose added the comment: http://tracker.ceph.com/issues/8797 reports that the backport to 2.7 causes a regression in ceph. -- nosy: +benjamin.peterson, doko status: closed - open ___ Python tracker rep...@bugs.python.org

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Ned Deily added the comment: I've opened Issue21963 to track the 2.7.8 regression. Please continue any discussion there. -- nosy: +ned.deily status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1856

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Roundup Robot added the comment: New changeset 7741d0dd66ca by Benjamin Peterson in branch '2.7': avoid crashes and lockups from daemon threads during interpreter shutdown (#1856) http://hg.python.org/cpython/rev/7741d0dd66ca -- ___ Python tracker

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Thomas Guettler added the comment: There are some examples to work around this for Python2: http://stackoverflow.com/questions/18098475/detect-interpreter-shut-down-in-daemon-thread -- nosy: +guettli ___ Python tracker rep...@bugs.python.org

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Roundup Robot devnull@devnull added the comment: New changeset 2a19d09b08f8 by Antoine Pitrou in branch '3.2': Issue #1856: Avoid crashes and lockups when daemon threads run while the http://hg.python.org/cpython/rev/2a19d09b08f8 New changeset c892b0321d23 by Antoine Pitrou in branch 'default':

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Antoine Pitrou pit...@free.fr added the comment: Should be fixed in 3.2 and 3.3 now. I don't really want to bother with 2.7 and 3.1 (the GIL implementation is different), but someone can backport the patch if they want to :) -- resolution: - fixed stage: patch review -

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Antoine Pitrou pit...@free.fr added the comment: Victor pointed out that Py_Finalize() is not necessarily called in the main Python thread. This new patch records the thread state of the finalizing thread, and also includes a test case. -- Added file:

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Antoine Pitrou pit...@free.fr added the comment: With Python 3.3, thread_exit.py still crashes more or less randomly: $ ./python thread_exit.py [49205 refs] python: Python/_warnings.c:501: setup_context: Assertion `((PyObject*)(*filename))-ob_type))-tp_flags ((1L28))) != 0)' failed.

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Antoine Pitrou pit...@free.fr added the comment: A patch that seems to work under Linux and Windows (for 3.2/3.3). -- nosy: +haypo stage: - patch review Added file: http://bugs.python.org/file21862/finalizing.patch ___ Python tracker

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by Andreas Stührk andy-pyt...@hammerhartes.de: -- nosy: +Trundle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1856 ___ ___

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Antoine Pitrou pit...@free.fr added the comment: See #9901 for a variation on the same global issue (running threads can access interpreter structures - the GIL - while the main thread is trying to destroy them). -- ___ Python tracker

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Antoine Pitrou pit...@free.fr added the comment: It is certainly still a problem with 3.x, but I don't find a way to exhibit it here. -- nosy: +pitrou versions: +Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by Guido van Rossum gu...@python.org: -- nosy: -gvanrossum ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1856 ___ ___ Python-bugs-list

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Terry J. Reedy tjre...@udel.edu added the comment: Is this still an issue in 2.7 or 3.x? Or should it be closed? -- nosy: +tjreedy versions: +Python 2.7 -Python 2.5, Python 2.6 ___ Python tracker rep...@bugs.python.org

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by Ernesto Menéndez pya...@gmail.com: -- nosy: +Netto ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1856 ___ ___ Python-bugs-list mailing

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by Terry J. Reedy tjre...@udel.edu: -- versions: -Python 2.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1856 ___ ___ Python-bugs-list

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by James Cooper james.coo...@solidodesign.com: -- nosy: +jamescooper ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1856 ___ ___

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by Floris Bruynooghe floris.bruynoo...@gmail.com: -- nosy: +flub ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1856 ___ ___

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Gabriel Genellina gagsl-...@yahoo.com.ar added the comment: I think applying Rhamphoryncus' patch in #1722344 fixes this too (that is, move WaitForThreadShutdown from the end of PyMain into Py_Finalize, so it is always called) But it should be tested on several platforms. -- nosy:

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by Forest Wilkinson for...@users.sourceforge.net: -- nosy: +forest ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1856 ___ ___

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: The threads don't have to be daemons: sys.exit() calls Py_Finalize() without waiting for non-daemons threads. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1856

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Georg Brandl [EMAIL PROTECTED] added the comment: Closed #2077 as a duplicate. -- nosy: +georg.brandl __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1856 __ ___

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Adam Olsen [EMAIL PROTECTED] added the comment: Cleaned up version of Amaury's patch. I stop releasing the GIL after sys.exitfunc is called, which protects threads from the ensuing teardown. I also grab the import lock (and never release it). This should prevent the nasty issue with daemon

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by Lars Immisch [EMAIL PROTECTED]: -- nosy: +larsimmisch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1856 __ ___ Python-bugs-list mailing list Unsubscribe:

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by Kurt B. Kaiser: -- nosy: +kbk __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1856 __ ___ Python-bugs-list mailing list Unsubscribe:

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Changes by Christian Heimes: -- priority: - high __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1856 __ ___ Python-bugs-list mailing list Unsubscribe:

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Amaury Forgeot d'Arc added the comment: I uploaded a script for a similar issue: http://bugs.python.org/issue1596321 (thread_crash.py) which sometimes segfaults on Windows XP (trunk version, debug build), with the same call stacks as printed by Gregory, on both threads. I explain it this way:

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Gregory P. Smith added the comment: agreed, during shutdown the other threads should be stopped. anything to do this complicates acquiring and releasing the GIL by adding another check to see if we're shutting down. brainstorm: I haven't looked at the existing BEGIN_ALLOW_THREADS and

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Amaury Forgeot d'Arc added the comment: Adam, your patch cover one case of the thread releasing the GIL (Py_ALLOW_THREADS), but there are more calls to PyThread_acquire_lock (the checkinterval for example). I have a competing patch: it makes the main thread never release the GIL after some

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Amaury Forgeot d'Arc added the comment: PyGILState_Ensure WOULD block forever if it acquired the GIL before doing anything else. Is it possible at all? PyThread_acquire_lock is not reentrant. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1856

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Amaury Forgeot d'Arc added the comment: That doesn't matter. PyGILState_Ensure needs to remain valid *forever*. Only once the process is completely gone can we be sure it won't be called. We could apply the same idea: when exiting, PyGILState_Ensure() blocks forever, except for the main

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Adam Olsen added the comment: Hrm. It seems you're right. Python needs thread-local data to determine if the GIL is held by the current thread. Thus, autoTLSkey and all that need to never be torn down. (The check could be done much more directly than the current PyThreadState_IsCurrent

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Adam Olsen added the comment: PyGILState_Ensure WOULD block forever if it acquired the GIL before doing anything else. The only way to make Py_Initialize callable after Py_Finalize is to make various bits of the finalization into no-ops. For instance, it's currently impossible to unload C

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Adam Olsen added the comment: Adam, did you notice the change on revision 59231 ? the PyGILState_Ensure stuff should now remain valid during the PyInterpreterState_Clear() call. That doesn't matter. PyGILState_Ensure needs to remain valid *forever*. Only once the process is completely gone

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Adam Olsen added the comment: I'm not sure I understand you, Gregory. Are arguing in favour of adding extra logic to the GIL code, or against it? I'm attaching a patch that has non-main thread exit, and it seems to fix the test case. It doesn't fix the PyGILState_Ensure problems though. Also

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Amaury Forgeot d'Arc added the comment: PyGILState_Ensure would still be broken. It touches various things that get torn down (autoInterpreterState, autoTLSkey, and the HEAD_LOCK macros) before it grabs the GIL. Reordering should be possible of course. Adam, did you notice the change on

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

New submission from Gregory P. Smith: This probably applies to 3.0 as well but i have not tested it there. Here are some sample failures: === A == Exception in thread Thread-01 (most likely raised during interpreter shutdown):Exception in thread Thread-03

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Guido van Rossum added the comment: So can we definitely rule out that this could be caused by the recent changes to threading.py (r57216)? -- nosy: +gvanrossum __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1856

[issue1856] shutdown (exit) can hang or segfault with daemon threads running

Gregory P. Smith added the comment: Yes i believe it is unrelated to any recent change. I can reproduce both behaviors on my OS X 10.4 dual core mac using Python 2.5.1. Python 2.3 on the mac appears to get stuck in a loop when run stand alone but gets a memory access fault when run under gdb.