On 6/17/2020 12:07 PM, Jeff Allen wrote:
On 12/06/2020 12:55, Eric V. Smith wrote:
On 6/11/2020 6:59 AM, Mark Shannon wrote:
Different interpreters need to operate in their own isolated address
space, or there will be horrible race conditions.
Regardless of whether that separation is done in software or hardware,
it has to be done.
I realize this is true now, but why must it always be true? Can't we
fix this? At least one solution has been proposed: passing around a
pointer to the current interpreter. I realize there issues here, like
callbacks and signals that will need to be worked out. But I don't
think it's axiomatically true that we'll always have race conditions
with multiple interpreters in the same address space.
Eric
Axiomatically? No, but let me rise to the challenge.
If (1) interpreters manage the life-cycle of objects, and (2) a race
condition arises when the life-cycle or state of an object is accessed
by the interpreter that did not create it, and (3) an object will
sometimes be passed to an interpreter that did not create it, and (4)
an interpreter with a reference to an object will sometimes access its
life-cycle or state, then (5) a race condition will sometimes arise.
This seems to be true (as a deduction) if all the premises hold.
(1) and (2) are true in CPython as we know it. (3) is prevented
(completely?) by the Python API, but not at all by the C API. (4) is
implicit in an interpreter having access to an object, the way CPython
and its extensions are written, so (5) follows in the case that the C
API is used. You could change (1) and/or (2), maybe (4).
I'm assuming that passing an object between interpreters would not be
supported. It would require that the object somehow be marshalled
between interpreters, so that no object would be operated on outside the
interpreter that created it. So 2-5 couldn't happen in valid code.
"Passing around a pointer to the current interpreter" sounds like an
attempt to break (2) or maybe (4). But I don't understand "current".
What you need at any time is the interpreter (state and life-cycle
manager) for the object you're about to handle, so that the receiving
interpreter can delegate the action, instead of crashing ahead itself.
This suggests a reference to the interpreter must be embedded in each
object, but it could be implicit in the memory address.
Sorry for being loose with terms. If I want to create an interpreter and
execute it, then I'd allocate and initialize an interpreter state
object, then call it, passing the interpreter state object in to
whatever Python functions I want to call. They would in turn pass that
pointer to whatever they call, or access the state through it directly.
That pointer is the "current interpreter".
There is then still an issue that the owning interpreter has to be
thread-safe (if there are threads) in the sense that it can serialise
access to object state or life-cycle. If serialisation is by a GIL,
the receiving interpreter must take the GIL of the owning interpreter,
and we are somewhat back where we started. Note that the "current
interpreter" is not a function of the current thread (or vice-versa).
The current thread is running in both interpreters, and by hypothesis,
so are the competing threads.
Agreed that an interpreter shouldn't belong to a thread, but since an
interpreter couldn't access objects of another interpreter, there'd be
no need for cross-intepreter locking. There would be a GIL per
interpreter, protecting access to that interpreter's state.
Can I just point out that, while most of this argument concerns a
particular implementation, we have a reason in Python (the language)
for an interpreter construct: it holds the current module context, so
that whenever code is executing, we can give definite meaning to the
'import' statement. Here "current interpreter" does have a meaning,
and I suggest it needs to be made a property of every function object
as it is defined, and picked up when the execution frame is created.
This *may* help with the other, internal, use of interpreter, for
life-cycle and state management, because it provides a recognisable
point (function call) where one may police object ownership, but that
isn't why you need it.
There's a lot of state per interpreter, including the module state. See
"struct _is" in Include/internal/pycore_interp.h.
Eric
Jeff Allen
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/GACVQJNCZLT4P3YX5IISRBOQTXXTJVMB/
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/NFSF5MC6WMGPDL77AEALKOCYN4MPDWV4/
Code of Conduct: http://python.org/psf/codeofconduct/