Le jeu. 9 janv. 2020 à 13:35, Mark Shannon <m...@hotpy.org> a écrit :
> Passing the thread state explicitly creates a new class of errors that
> was not there before.
> What happens if the tstate argument is NULL,
You will likely get a crash.
> or points to a different thread?
No idea what will happen. If it's a different subinterpreter, bad
things can happen I guess :-)
Well, I don't see this class of bug as a blocker issuer. If you pass
invalid data, you get a crash. I'm not surprised by that.
In my previous email, I proposed to continue to pass implicitly tstate
when you call a function using the public C API. Only Python internals
would pass explicitly tstate and so only internals should be carefully
reviewed.
> There is a one-to-one correspondence between Python threads and O/S
> threads.
I'm not convinced that this :-) So far, it's unclear to me how the
transition from one interpreter to another happens with Python thread
states.
Let's say that the main thead spawns a thread A. The main thread and
the thread A are running in the main interpreter. Then thread A calls
_testcapi.run_in_subinterp() or _xxsubinterpreter.run_string() (or
something else to run code in a subinterpreter). A subinterpreter is
created and a new and different Python thread state is "attached" to
thread A. The thread A gets 2 Python thread states, one per
interpreter. It gets one or the other depending which in which
interpreter the thread is running...
We have to be careful to pass the proper thread state to internal C
functions while doing this dance between two interpreters in the same
thread. PyThreadState_Swap(tstate) must be called to set the current
Python thread state.
Full implementation of _testcapi.run_in_subinterp(), notice the
PyThreadState_Swap() dance:
/* To run some code in a sub-interpreter. */
static PyObject *
run_in_subinterp(PyObject *self, PyObject *args)
{
const char *code;
int r;
PyThreadState *substate, *mainstate;
if (!PyArg_ParseTuple(args, "s:run_in_subinterp",
&code))
return NULL;
mainstate = PyThreadState_Get();
PyThreadState_Swap(NULL);
substate = Py_NewInterpreter();
if (substate == NULL) {
/* Since no new thread state was created, there is no exception to
propagate; raise a fresh one after swapping in the old thread
state. */
PyThreadState_Swap(mainstate);
PyErr_SetString(PyExc_RuntimeError, "sub-interpreter creation failed");
return NULL;
}
r = PyRun_SimpleString(code);
Py_EndInterpreter(substate);
PyThreadState_Swap(mainstate);
return PyLong_FromLong(r);
}
> So the threadstate can, and should, be stored in a thread local
> variable.
> Accessing thread local storage is fast. x86/64 uses the fs register to
> point to it, whereas ARM dedicates R15 (I think).
*If* we managed to design subinterpreters in a way that a native
thread is always associated to the same Python interpreter, moving to
a thread local variable can make Python more efficient :-) It is
likely to be more efficient than the current atomic variable design.
> > I started to move more and more things from "globals" to "per
> > interpreter". For example, the garbage collector is now "per
> > interpreter" (lives in PyThreadState). Small integer singletons are
> > now also "per singleton": int("1") are now different objects in each
> > interpreter, whereas they were shared previously. Later, even "None"
> > singleton (and all other singletons) should be made "per interpreter".
> > Getting a "per interpreter" object requires to state from the Python
> > thread state: call _PyThreadState_GET(). Avoiding _PyThreadState_GET()
> > calls reduces any risk of making Python slower with incoming
> > subinterpreter changes.
>
> Thread locals are not "global". Each sub-interpreter will have its own
> pool of threads. Each threadstate object should contain a pointer to its
> sub-interpreter.
Getting the interpreter from a Python thread state is trivial: interp
= tstate->interp.
The problem is how to get the current Python thread state.
*Currently*, it's an atomic variable. But tomorrow with multiple
interpreters running in parallel, I expect that it's going to me more
expensive like first having the current the interpreter running the
current native thread, and then get the Python thread state of this
interpreter. Or something like that. We may get more and more
indirections...
Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/HMSRW57KNZOOWRWHSXVGYR6BYLT3NPPN/
Code of Conduct: http://python.org/psf/codeofconduct/
