[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-05-09 Thread STINNER Victor
STINNER Victor added the comment: > Note that even with POSIX TLS in use, it isn't entirely safe. I wrote faulthandler to collect debug data just before dying, when something already gone very bad, like a deadlock or a memory corruption. I didn't design faulthandler for correctness. If you ca

[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-05-09 Thread Gregory P. Smith
Gregory P. Smith added the comment: Follow up note: Note that even with POSIX TLS in use, it isn't entirely safe. pthread_getspecific() as used by the posix thread code is not required to be async signal safe by the POSIX standard. :( The Linux glibc implementation thankfully does not use l

[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-04-27 Thread STINNER Victor
STINNER Victor added the comment: Since CPython 3 is not affected, I close this issue and move back the discussion on the bug tracker of the CPython 2.7 backport: https://github.com/haypo/faulthandler/issues/29 -- resolution: -> works for me stage: needs patch -> resolved status: open

[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-04-26 Thread STINNER Victor
STINNER Victor added the comment: Oh sorry, you reported the bug on my 2.7 backport but I ask you to report the bug here, and now the bug is specific to 2.7 :-) Yes, faulthandler implementation is different on Python 2.7. If you look in depth, it is even very different especially "dump later" (2

[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-04-26 Thread Gregory P. Smith
Gregory P. Smith added the comment: So... it looks like this could and does happen only on the 2.7 backport of faulthandler. 2.7 is missing the following commit to use pthreads native TLS instead of our lock filled non-async signal safe implementation: https://github.com/python/cpython/commi

[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-04-26 Thread Gregory P. Smith
Gregory P. Smith added the comment: This report is based on manual code inspection in CPython head after we encountered a deadlock using pytracemalloc on Python 2.7.12 where it _appeared_ to be the scenario I've described. I see now that I missed noticing the "#ifndef Py_HAVE_NATIVE_TLS" withi

[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-04-26 Thread STINNER Victor
STINNER Victor added the comment: > This sometimes leads to a deadlock when the process is crashing, handled via > faulthandler, instead of a crash with stacktrace information printed. The > opposite of the happy debugging experience it is intended to provide. Is it an hypothetical analysis b

[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-04-26 Thread STINNER Victor
STINNER Victor added the comment: > I _believe_ we always want to use _PyThreadState_UncheckedGet() faulthandler_dump_traceback() uses PyGILState_GetThisThreadState(). There is a comment to explain why: /* SIGSEGV, SIGFPE, SIGABRT, SIGBUS and SIGILL are synchronous signals and are t

[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-04-26 Thread STINNER Victor
STINNER Victor added the comment: > but faulthandler_dump_traceback calls PyGILState_GetThisThreadState() which > ultimately calls thread.c's find_key() which acquires a lock: Hum, Python 3 now uses native TLS, not this fallback implementation using a lock. At least on Linux and Windows. I don

[issue30165] faulthandler acquires lock from signal handler, can deadlock while crashing

2017-04-25 Thread Gregory P. Smith
New submission from Gregory P. Smith: https://github.com/python/cpython/blob/master/Modules/faulthandler.c#L240 faulthandler_dump_traceback() is called from the signal handler faulthandler_fatal_error() which needs to be async signal safe and only call async signal safe things... but faulthand