On 09Jan2020 1659, Victor Stinner wrote:
Le jeu. 9 janv. 2020 à 19:33, Steve Dower <steve.do...@python.org> a écrit :
Requiring an _active_ Python thread (~GIL held) to only run on a single
OS thread is fine, but tying it permanently to a single OS thread makes
things very painful. (Of course, this isn't the only thing that makes it
painful, but if we're going to make this a deliberate design of CPython
then we are essentially excluding entire interesting classes of embedding.)
Do you have an use case where you reuse the same Python thread state
in different native threads?
I do, but I can't share exactly what it is (yet) :)
But essentially, the main application has its own threading setup where
callbacks happen on its own threadpool. Some of those callbacks
sometimes have to raise Python events, but all of those events should be
serialized (i.e. *no* Python code should run in parallel, even though
the app's callbacks can).
Note that this is not arbitrary CPython code. It is a very restricted
context (only first-party modules, no threading/ctypes/etc.).
Ideally, we'd just lock the Python thread and run the Python code and
get the result back directly. Instead, we've had to set up a parallel
Python thread that is constantly running, and build a complicated
cross-thread marshalling setup in order to simulate synchronous calls.
(And also argue with the application architects who are very against
mixing threading models in their app, unsurprisingly.)
Currently, the Python runtime expects to have a different Python
thread state per native thread. For example, PyGILState_Ensure()
creates one if there is no Python thread state associated to the
current thread.
And ultimately, everything above is just because of this assumption.
Turns out that simply transferring the thread state and setting it in
TLS is *nearly* okay most of the time, but not quite. Breaking down the
assumption that each CPython thread is the only code running on its
particular OS thread would help a lot here - making the CPython thread
state a parameter rather than ambient state is the right direction.
Cheers,
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/Q7VIABF346X4GQRXDIQIHD4I74GDNF54/
Code of Conduct: http://python.org/psf/codeofconduct/
