[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
On 09. 03. 22 4:38, Eric Snow wrote: I'd really appreciate feedback on this new PEP about making the GIL per-interpreter. Yay! Thank you! The PEP targets 3.11, but we'll see if that is too close. I don't mind waiting one more release, though I'd prefer 3.11 (obviously). Regardless, I have no intention of rushing this through at the expense of cutting corners. Hence, we'll see how it goes. How mature is the implementation? If it ends up in 3.12, I'd consider asking the release manager for an extra alpha release so people can start playing with the feature early. (With my Fedora hat on: I'd love to test it with thousands of packages!) The PEP text is included inline below. Thanks! -eric === PEP: 684 Title: A Per-Interpreter GIL Author: Eric Snow Discussions-To: python-dev@python.org Status: Draft Type: Standards Track Content-Type: text/x-rst This iteration of the PEP should also have `Requires: 683` (Immortal Objects). Created: 08-Mar-2022 Python-Version: 3.11 Post-History: 08-Mar-2022 Resolution: Abstract Since Python 1.5 (1997), CPython users can run multiple interpreters in the same process. However, interpreters in the same process have always shared a significant amount of global state. This is a source of bugs, with a growing impact as more and more people use the feature. Furthermore, sufficient isolation would facilitate true multi-core parallelism, where interpreters no longer share the GIL. The changes outlined in this proposal will result in that level of interpreter isolation. High-Level Summary == At a high level, this proposal changes CPython in the following ways: * stops sharing the GIL between interpreters, given sufficient isolation * adds several new interpreter config options for isolation settings * adds some public C-API for fine-grained control when creating interpreters * keeps incompatible extensions from causing problems The GIL --- The GIL protects concurrent access to most of CPython's runtime state. So all that GIL-protected global state must move to each interpreter before the GIL can. (In a handful of cases, other mechanisms can be used to ensure thread-safe sharing instead, such as locks or "immortal" objects.) CPython Runtime State - Properly isolating interpreters requires that most of CPython's runtime state be stored in the ``PyInterpreterState`` struct. Currently, only a portion of it is; the rest is found either in global variables or in ``_PyRuntimeState``. Most of that will have to be moved. This directly coincides with an ongoing effort (of many years) to greatly reduce internal use of C global variables and consolidate the runtime state into ``_PyRuntimeState`` and ``PyInterpreterState``. (See `Consolidating Runtime Global State`_ below.) That project has `significant merit on its own `_ and has faced little controversy. So, while a per-interpreter GIL relies on the completion of that effort, that project should not be considered a part of this proposal--only a dependency. Other Isolation Considerations -- CPython's interpreters must be strictly isolated from each other, with few exceptions. To a large extent they already are. Each interpreter has its own copy of all modules, classes, functions, and variables. The CPython C-API docs `explain further `_. .. _caveats: https://docs.python.org/3/c-api/init.html#bugs-and-caveats However, aside from what has already been mentioned (e.g. the GIL), there are a couple of ways in which interpreters still share some state. First of all, some process-global resources (e.g. memory, file descriptors, environment variables) are shared. There are no plans to change this. Second, some isolation is faulty due to bugs or implementations that did not take multiple interpreters into account. This includes CPython's runtime and the stdlib, as well as extension modules that rely on global variables. Bugs should be opened in these cases, as some already have been. Depending on Immortal Objects - :pep:`683` introduces immortal objects as a CPython-internal feature. With immortal objects, we can share any otherwise immutable global objects between all interpreters. Consequently, this PEP does not need to address how to deal with the various objects `exposed in the public C-API `_. It also simplifies the question of what to do about the builtin static types. (See `Global Objects`_ below.) Both issues have alternate solutions, but everything is simpler with immortal objects. If PEP 683 is not accepted then this one will be updated with the alternatives. This lets us reduce noise in this proposal. Motivation == The fundamental problem we're solving here is a lack of true multi-core parallelism (for Python code) in the CPython runtime. The GIL is the cause. While it usually isn't a problem in practice, at the v
[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
Oops, I hit Send by mistake! Please disregard the previous message (I often draft questions I later find answered, so I delete them.) On Wed, Mar 9, 2022 at 5:53 PM Petr Viktorin wrote: > > On 09. 03. 22 4:38, Eric Snow wrote: > > I'd really appreciate feedback on this new PEP about making the GIL > > per-interpreter. > > Yay! Thank you! > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RLBJEE2MLXMJNN2R444AFZDN54JDRWI7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
On 09. 03. 22 4:38, Eric Snow wrote:
I'd really appreciate feedback on this new PEP about making the GIL
per-interpreter.
Yay! Thank you!
This PEP definitely makes per-interpreter GIL sound possible :)
The PEP targets 3.11, but we'll see if that is too close. I don't
mind waiting one more
release, though I'd prefer 3.11 (obviously). Regardless, I have no
intention of rushing
this through at the expense of cutting corners. Hence, we'll see how it goes.
> The PEP text is included inline below. Thanks!
-eric
===
PEP: 684
Title: A Per-Interpreter GIL
Author: Eric Snow
Discussions-To: python-dev@python.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
This iteration of the PEP should also have `Requires: 683` (Immortal
Objects).
[...]
Motivation
==
The fundamental problem we're solving here is a lack of true multi-core
parallelism (for Python code) in the CPython runtime. The GIL is the
cause. While it usually isn't a problem in practice, at the very least
it makes Python's multi-core story murky, which makes the GIL
a consistent distraction.
Isolated interpreters are also an effective mechanism to support
certain concurrency models. :pep:`554` discusses this in more detail.
Indirect Benefits
-
Most of the effort needed for a per-interpreter GIL has benefits that
make those tasks worth doing anyway:
* makes multiple-interpreter behavior more reliable
* has led to fixes for long-standing runtime bugs that otherwise
hadn't been prioritized > * has been exposing (and inspiring fixes for) previously unknown
runtime bugs
* has driven cleaner runtime initialization (:pep:`432`, :pep:`587`)
* has driven cleaner and more complete runtime finalization
* led to structural layering of the C-API (e.g. ``Include/internal``)
* also see `Benefits to Consolidation`_ below
Do you want to dig up some bpo examples, to make these more convincing
to the casual reader?
Furthermore, much of that work benefits other CPython-related projects:
* performance improvements ("faster-cpython")
* pre-fork application deployment (e.g. Instagram)
Maybe say “e.g. with Instagram's Cinder” – both the household name and
the project you can link to?
* extension module isolation (see :pep:`630`, etc.)
* embedding CPython
A lot of these points are duplicated in "Benefits to Consolidation" list
below, maybe there'd be, ehm, benefits to consolidating them?
[...]
PEP 554
---
Please spell out "PEP 554 (Multiple Interpreters in the Stdlib)", for
people who don't remember the magic numbers but want to skim the table
of contents.
:pep:`554` is strictly about providing a minimal stdlib module
to give users access to multiple interpreters from Python code.
In fact, it specifically avoids proposing any changes related to
the GIL. Consider, however, that users of that module would benefit
from a per-interpreter GIL, which makes PEP 554 more appealing.
Rationale
=
During initial investigations in 2014, a variety of possible solutions
for multi-core Python were explored, but each had its drawbacks
without simple solutions:
* the existing practice of releasing the GIL in extension modules
* doesn't help with Python code
* other Python implementations (e.g. Jython, IronPython)
* CPython dominates the community
* remove the GIL (e.g. gilectomy, "no-gil")
* too much technical risk (at the time)
* Trent Nelson's "PyParallel" project
* incomplete; Windows-only at the time
* ``multiprocessing``
* too much work to make it effective enough;
high penalties in some situations (at large scale, Windows)
* other parallelism tools (e.g. dask, ray, MPI)
* not a fit for the stdlib
* give up on multi-core (e.g. async, do nothing)
* this can only end in tears
This list doesn't render correctly in ReST, you need blank lines everywhere.
There are more cases like this below.
[...]> Per-Interpreter State
-
The following runtime state will be moved to ``PyInterpreterState``:
* all global objects that are not safely shareable (fully immutable)
* the GIL
* mutable, currently protected by the GIL
Spelling out “mutable state” in these lists would make this clearer,
since “state” isn't elided from all the points.
* mutable, currently protected by some other per-interpreter lock
* mutable, may be used independently in different interpreters
This includes extension modules (with multi-phase init), right?
* all other mutable (or effectively mutable) state
not otherwise excluded below
Furthermore, a number of parts of the global state have already been
moved to the interpreter, such as GC, warnings, and atexit hooks.
The following state will not be moved:
* global objects that are safely shareable, if any
* immutable, often ``const``
* treated as immutable
Do you have an example for this?
* related to CPython's ``main()`` execution
* related to the REPL
Woul
[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
Thanks for the feedback, Petr! Responses inline below.
-eric
On Wed, Mar 9, 2022 at 10:58 AM Petr Viktorin wrote:
> This PEP definitely makes per-interpreter GIL sound possible :)
Oh good. :)
> > PEP: 684
> > Title: A Per-Interpreter GIL
> > Author: Eric Snow
> > Discussions-To: python-dev@python.org
> > Status: Draft
> > Type: Standards Track
> > Content-Type: text/x-rst
>
> This iteration of the PEP should also have `Requires: 683` (Immortal
> Objects).
+1
> > Most of the effort needed for a per-interpreter GIL has benefits that
> > make those tasks worth doing anyway:
> >
> > * makes multiple-interpreter behavior more reliable
> > * has led to fixes for long-standing runtime bugs that otherwise
> >hadn't been prioritized > * has been exposing (and inspiring fixes for)
> > previously unknown
> runtime bugs
> > * has driven cleaner runtime initialization (:pep:`432`, :pep:`587`)
> > * has driven cleaner and more complete runtime finalization
> > * led to structural layering of the C-API (e.g. ``Include/internal``)
> > * also see `Benefits to Consolidation`_ below
>
> Do you want to dig up some bpo examples, to make these more convincing
> to the casual reader?
Heh, the casual reader isn't really my target audience. :) I actually
have a stockpile of links but left them all out until they were
needed. Would the decision-makers benefit from the links? I'm trying
to avoid adding to the already sizeable clutter in this PEP. :) I'll
add some links in if you think it matters.
> > Furthermore, much of that work benefits other CPython-related projects:
> >
> > * performance improvements ("faster-cpython")
> > * pre-fork application deployment (e.g. Instagram)
>
> Maybe say “e.g. with Instagram's Cinder” – both the household name and
> the project you can link to?
+1
Note that Instagram isn't exactly using Cinder. I'll have to check if
Cinder uses the pre-fork model.
> > * extension module isolation (see :pep:`630`, etc.)
> > * embedding CPython
>
> A lot of these points are duplicated in "Benefits to Consolidation" list
> below, maybe there'd be, ehm, benefits to consolidating them?
There shouldn't be any direct overlap.
FWIW, the whole "Extra Context" section is essentially a separate PEP
that I inlined (with the caveat that it really isn't worth its own
PEP). I'm still considering yanking it, so the above list should
stand on its own.
> > PEP 554
> > ---
>
> Please spell out "PEP 554 (Multiple Interpreters in the Stdlib)", for
> people who don't remember the magic numbers but want to skim the table
> of contents.
+1
> This list doesn't render correctly in ReST, you need blank lines everywhere.
> There are more cases like this below.
Hmm, I had blank lines and the PEP editor told me I needed to remove them.
> [...]> Per-Interpreter State
> > -
> >
> > The following runtime state will be moved to ``PyInterpreterState``:
> >
> > * all global objects that are not safely shareable (fully immutable)
> > * the GIL
> > * mutable, currently protected by the GIL
>
> Spelling out “mutable state” in these lists would make this clearer,
> since “state” isn't elided from all the points.
+1
> > * mutable, currently protected by some other per-interpreter lock
> > * mutable, may be used independently in different interpreters
>
> This includes extension modules (with multi-phase init), right?
Yep.
> > The following state will not be moved:
> >
> > * global objects that are safely shareable, if any
> > * immutable, often ``const``
> > * treated as immutable
>
> Do you have an example for this?
Strings (PyUnicodeObject) actually cache some info, making them not
strictly immutable, but they are close enough to be treated as such.
I'll add a note to the PEP.
> > * related to CPython's ``main()`` execution
> > * related to the REPL
>
> Would “only used by” work instead of “related to”?
Sure.
> > * set during runtime init, then treated as immutable
>
> `main()`, REPL and runtime init look like special cases of functionality
> that only runs in one interpreter. If it's so, maybe generalize this?
+1
> > * ``_PyInterpreterConfig``
> > * ``_Py_NewInterpreter()`` (as ``Py_NewInterpreterEx()``)
>
> Since the API is not documented (and _PyInterpreterConfig is not even in
> main yet!), it would be good to sketch out the docs (intended behavior)
> here.
+1
> > The following fields will be added to ``PyInterpreterConfig``:
> >
> > * ``own_gil`` - (bool) create a new interpreter lock
> >(instead of sharing with the main interpreter)
>
> As a user of the API, what should I consider when setting this flag?
> Would the GIL be shared with the *parent* interpreter or the main one?
The GIL would be shared with the main interpreter. I state that there
but it looks like I wasn' clear enough.
> What are the restrictions/implications of this flag?
Good point. I'll add a brief explanation of why you would want to
keep sharing the GIL (e.g. the status quo) and what is different if
[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
Hi Eric, just one note: On Wed, Mar 9, 2022 at 7:13 PM Eric Snow wrote: > > Maybe say “e.g. with Instagram's Cinder” – both the household name and > > the project you can link to? > > +1 > > Note that Instagram isn't exactly using Cinder. This sounds like a misunderstanding somewhere. Instagram server is "exactly using Cinder" :) > I'll have to check if Cinder uses the pre-fork model. It doesn't really make sense to ask whether "Cinder uses the pre-fork model" -- Cinder is just a CPython variant, it can work with all the same execution models CPython can. Instagram server uses Cinder with a pre-fork execution model. Some other workloads use Cinder without pre-forking. Carl ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5A3E6VCEY5XZXEFPGHNGKPM3HXQEJRTX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
On Wed, Mar 9, 2022 at 7:37 PM Carl Meyer wrote: > > Note that Instagram isn't exactly using Cinder. > > This sounds like a misunderstanding somewhere. Instagram server is > "exactly using Cinder" :) :) Thanks for clarifying, Carl. > > I'll have to check if Cinder uses the pre-fork model. > > It doesn't really make sense to ask whether "Cinder uses the pre-fork > model" -- Cinder is just a CPython variant, it can work with all the > same execution models CPython can. Instagram server uses Cinder with a > pre-fork execution model. Some other workloads use Cinder without > pre-forking. +1 -eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZI6JXJJ2F6DCHTVYUVQFDNPCWEH76J6V/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
> Is ``allow_all_extensions`` the best name for the context manager? Nope. I'm pretty sure that "parallel processing via multiple simultaneous interpreters" won't be the only reason people ever want to exclude certain extensions. It might be easier to express that through package or module name, but importlib and util aren't specific enough. For an example of an extension that works with multiple interpreters but only if they share a single GIL ... why wouldn't that apply to any extension designed to work with a Singleton external resource? For example, the interpreters could all share a single database connection, and repurpose the GIL to ensure that there isn't a thread (or interpreter) switch mid-transaction. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RUDVIEDDCNFDRBIQVQU334GMPW77ZNOK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
That sounds like a horrible idea. The GIL should never be held during an I/O operation. On Fri, Mar 11, 2022 at 19:00 Jim J. Jewett wrote: > > Is ``allow_all_extensions`` the best name for the context manager? > > Nope. I'm pretty sure that "parallel processing via multiple simultaneous > interpreters" won't be the only reason people ever want to exclude certain > extensions. > > It might be easier to express that through package or module name, but > importlib and util aren't specific enough. > > For an example of an extension that works with multiple interpreters but > only if they share a single GIL ... why wouldn't that apply to any > extension designed to work with a Singleton external resource? For > example, the interpreters could all share a single database connection, and > repurpose the GIL to ensure that there isn't a thread (or interpreter) > switch mid-transaction. > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/RUDVIEDDCNFDRBIQVQU334GMPW77ZNOK/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido (mobile) ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VKFNIFQRQZ5EVZ4G4TQIR2NGMO7BAMWY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
On Thu, 10 Mar 2022, 12:12 pm Eric Snow, wrote: > > On Wed, Mar 9, 2022 at 10:58 AM Petr Viktorin wrote: > > This PEP definitely makes per-interpreter GIL sound possible :) > > > > > > “Main interpreter” should be defined. > > +1 > > > (Or maybe the term should be > > avoided instead -- always having to spell out “interpreter started by > > Py_Initialize rather than Py_NewInterpreter” might push us toward > > finding ways to avoid the special case...) > > We (me, Nick, Victor, others) have considered this in the past and > have concluded that having a distinct main interpreter is valuable. > That topic is a bit out of scope for this PEP though. > The PEP can mostly link to https://docs.python.org/3/c-api/init.html#sub-interpreter-support for the explanation of the main interpreter, and just include a partial paraphrase of those docs to give the gist of the idea. We added that info the last time we considered whether the main interpreter's "first among equals" status was necessary and decided it was. For example, something based on the first and third sentences out of the docs explanation: The “main” interpreter is the first one created when the runtime initializes. It continues to manage unique process-global responsibilities like signal handling even when other subinterpreters have been started. Cheers, Nick. > > > > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/M4IWPEJVKS6VRKLZGAFYZSXMFK6CMBD6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 684: A Per-Interpreter GIL
> That sounds like a horrible idea. The GIL should never be held during an > I/O operation. For a greenfield design, I agree that it would be perverse. But I thought we were talking about affordances for transitions from code that was written without consideration of multiple interpreters. In those cases, the GIL can be a way of saying "OK, this is the part where I haven't thought things through yet." Using a more fine-grained lock would be better, but would take a lot more work and be more error-prone. For a legacy system, I'm seen plenty of situations where a blunt (but simple) hammer like "Grab the GIL" would still be a huge improvement from the status quo. And those situations tend to occur with the sort of clients where "Brutally inefficient, but it does work because the fragile parts are guaranteed by an external tool" is the right tradeoff. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AAWSCUNVS2NUXRHVATO736KM6I5M6RK5/ Code of Conduct: http://python.org/psf/codeofconduct/
