STINNER Victor <vstin...@redhat.com> added the comment:
> That's okay, Victor. Thanks for jumping on this. I'll take a look when I > get a chance. >From what I saw, your first commit was enough to reproduce the crash. If I recall correctly, Antoine Pitrou modified the GIL so threads exit immediately when Py_Finalize() is called. I'm thinking at: void PyEval_RestoreThread(PyThreadState *tstate) { ... take_gil(tstate); /* _Py_Finalizing is protected by the GIL */ if (_Py_IsFinalizing() && !_Py_CURRENTLY_FINALIZING(tstate)) { drop_gil(tstate); PyThread_exit_thread(); Py_UNREACHABLE(); } ... PyThreadState_Swap(tstate); } Problem: this code uses tstate, whereas the crash occurred because tstate pointed to freed memory: """ Thread 1 got the crash (gdb) p *tstate $3 = { prev = 0xdbdbdbdbdbdbdbdb, next = 0xdbdbdbdbdbdbdbdb, interp = 0xdbdbdbdbdbdbdbdb, ... } ... Thread 1 (LWP 100696): #0 0x0000000000368210 in take_gil (tstate=0x8027e2050) at Python/ceval_gil.h:216 #1 0x0000000000368a94 in PyEval_RestoreThread (tstate=0x8027e2050) at Python/ceval.c:281 ... """ https://bugs.python.org/issue36114#msg337090 When this crash occurred, Py_Finalize() already completed in the main thread! """ void _Py_NO_RETURN Py_Exit(int sts) { if (Py_FinalizeEx() < 0) { /* <==== DONE! */ sts = 120; } exit(sts); /* <=============== Crash occurred here! */ } """ Py_Finalize() is supposed to wait for threads before deleting Python thread states: """ int Py_FinalizeEx(void) { ... /* The interpreter is still entirely intact at this point, and the * exit funcs may be relying on that. In particular, if some thread * or exit func is still waiting to do an import, the import machinery * expects Py_IsInitialized() to return true. So don't say the * interpreter is uninitialized until after the exit funcs have run. * Note that Threading.py uses an exit func to do a join on all the * threads created thru it, so this also protects pending imports in * the threads created via Threading. */ call_py_exitfuncs(interp); ... /* Remaining threads (e.g. daemon threads) will automatically exit after taking the GIL (in PyEval_RestoreThread()). */ _PyRuntime.finalizing = tstate; _PyRuntime.initialized = 0; _PyRuntime.core_initialized = 0; ... /* Delete current thread. After this, many C API calls become crashy. */ PyThreadState_Swap(NULL); PyInterpreterState_Delete(interp); ... } """ The real problem for years are *deamon threads* which... BY DESIGN... remain alive after Py_Finalize() exit! But as I explained, they must exit as soon at they attempt to get GIL. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33608> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com