[issue36476] Runtime finalization assumes all other threads have exited.

Change by STINNER Victor : -- components: +Subinterpreters ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36476] Runtime finalization assumes all other threads have exited.

Nick Coghlan added the comment: Thinking about that idea further, I don't think that change would help much, since the relevant operations should already be checking for thread termination when they attempt to reacquire the GIL. That means what we're missing is: 1. When daemon threads

[issue36476] Runtime finalization assumes all other threads have exited.

Nick Coghlan added the comment: Perhaps we need a threading.throw() API, similar to the one we have for generators and coroutines? If we had that, then Py_FinalizeEx() could gain a few new features: * throw SystemExit into all daemon threads and then give them a chance to terminate before

[issue36476] Runtime finalization assumes all other threads have exited.

Antoine Pitrou added the comment: > 1. Python daemon threads I think the answer is to document a bit more clearly that they can pose all kinds of problems. Perhaps we could even display a visible warning when people create daemon threads. > 2. Python threads created in atexit handlers We

[issue36476] Runtime finalization assumes all other threads have exited.

Eric Snow added the comment: So I see 3 things to address here: 1. Python daemon threads 2. Python threads created in atexit handlers 3. non-Python threads accessing the C-API Possible solutions (starting point for discussion): 1. stop them at the point we stop waiting for non-daemon

[issue36476] Runtime finalization assumes all other threads have exited.

Eric Snow added the comment: To put it another way: (from issue33608#msg358748) > The docs [1] aren't super clear about it, but there are some fundamental > assumptions we make about runtime finalization: > > * no use of the C-API while Py_FinalizeEx() is executing (except for a > few

[issue36476] Runtime finalization assumes all other threads have exited.

Eric Snow added the comment: Analysis by @pconnell: * https://bugs.python.org/issue33608#msg357169 * https://bugs.python.org/issue33608#msg357170 * https://bugs.python.org/issue33608#msg357179 tl;dr daemon threads and external C-API access during/after runtime finalization are causing

[issue36476] Runtime finalization assumes all other threads have exited.

Change by Eric Snow : -- nosy: +pconnell ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36476] Runtime finalization assumes all other threads have exited.

Eric Snow added the comment: Problems with lingering threads during/after runtime finalization continue to be a problem. I'm going to use this issue as the focal point for efforts to resolve this. Related issues: * #36479 "Exit threads when interpreter is finalizing rather than runtime."

[issue36476] Runtime finalization assumes all other threads have exited.

Change by Eric Snow : -- stage: -> needs patch versions: +Python 3.9 -Python 3.7 ___ Python tracker ___ ___ Python-bugs-list

[issue36476] Runtime finalization assumes all other threads have exited.

Eric Snow added the comment: Adding to the list: * any OS threads created by an extension module or embedding application -- ___ Python tracker ___

[issue36476] Runtime finalization assumes all other threads have exited.

Eric Snow added the comment: FYI, I've opened issue36477 to deal with the subinterpreters case. -- ___ Python tracker ___ ___

[issue36476] Runtime finalization assumes all other threads have exited.

New submission from Eric Snow : Among the first 3 things that happen in Py_FinalizeEx() are, in order: 1. wait for all non-daemon threads (of the main interpreter) to finish 2. call registered atexit funcs 3. mark the runtime as finalizing At that point the only remaining Python threads are: