On 19. 02. 22 8:46, Eric Snow wrote:
Thanks to all those that provided feedback. I've worked to
substantially update the PEP in response. The text is included below.
Further feedback is appreciated.
Thank you! This version is much clearer. I like the PEP more and more!
I've sent a PR with a some typo fixes:
https://github.com/python/peps/pull/2348
and I have a few comments:
[...]
Public Refcount Details
[...]
As part of this proposal, we must make sure that users can clearly
understand on which parts of the refcount behavior they can rely and
which are considered implementation details. Specifically, they should
use the existing public refcount-related API and the only refcount value
with any meaning is 0. All other values are considered "not 0".
Should we care about hacks/optimizations that rely on having the only
reference (or all references), e.g. mutating a tuple if it has refcount
1? Immortal objects shouldn't break them (the special case simply won't
apply), but this wording would make them illegal.
AFAIK CPython uses this internally, but I don't know how
prevalent/useful it is in third-party code.
[...]
_Py_IMMORTAL_REFCNT
-------------------
We will add two internal constants::
#define _Py_IMMORTAL_BIT (1LL << (8 * sizeof(Py_ssize_t) - 4))
#define _Py_IMMORTAL_REFCNT (_Py_IMMORTAL_BIT + (_Py_IMMORTAL_BIT / 2))
As a nitpick: could you say this in prose?
* ``_Py_IMMORTAL_BIT`` has the third top-most bit set.
* ``_Py_IMMORTAL_REFCNT`` has the third and fourth top-most bits set.
[...]
Immortal Global Objects
-----------------------
All objects that we expect to be shared globally (between interpreters)
will be made immortal. That includes the following:
* singletons (``None``, ``True``, ``False``, ``Ellipsis``, ``NotImplemented``)
* all static types (e.g. ``PyLong_Type``, ``PyExc_Exception``)
* all static objects in ``_PyRuntimeState.global_objects`` (e.g. identifiers,
small ints)
All such objects will be immutable. In the case of the static types,
they will be effectively immutable. ``PyTypeObject`` has some mutable
start (``tp_dict`` and ``tp_subclasses``), but we can work around this
by storing that state on ``PyInterpreterState`` instead of on the
respective static type object. Then the ``__dict__``, etc. getter
will do a lookup on the current interpreter, if appropriate, instead
of using ``tp_dict``.
But tp_dict is also public C-API. How will that be handled?
Perhaps naively, I thought static types' dicts could be treated as
(deeply) immutable, and shared?
Perhaps it would be best to leave it out here and say say "The details
of sharing ``PyTypeObject`` across interpreters are left to another PEP"?
Even so, I'd love to know the plan. (And even if these are internals,
changes to them should be mentioned in What's New, for the sake of
people who need to maintain old extensions.)
Object Cleanup
--------------
In order to clean up all immortal objects during runtime finalization,
we must keep track of them.
For GC objects ("containers") we'll leverage the GC's permanent
generation by pushing all immortalized containers there. During
runtime shutdown, the strategy will be to first let the runtime try
to do its best effort of deallocating these instances normally. Most
of the module deallocation will now be handled by
``pylifecycle.c:finalize_modules()`` which cleans up the remaining
modules as best as we can. It will change which modules are available
during __del__ but that's already defined as undefined behavior by the
docs. Optionally, we could do some topological disorder to guarantee
that user modules will be deallocated first before the stdlib modules.
Finally, anything leftover (if any) can be found through the permanent
generation gc list which we can clear after finalize_modules().
For non-container objects, the tracking approach will vary on a
case-by-case basis. In nearly every case, each such object is directly
accessible on the runtime state, e.g. in a ``_PyRuntimeState`` or
``PyInterpreterState`` field. We may need to add a tracking mechanism
to the runtime state for a small number of objects.
Out of curiosity: How does this extra work affect in the performance? Is
it part of the 4% slowdown?
And from the other thread:
On 17. 02. 22 18:23, Eric Snow wrote:
> On Thu, Feb 17, 2022 at 3:42 AM Petr Viktorin <encu...@gmail.com> wrote:
>>>> Weren't you planning a PEP on subinterpreter GIL as well? Do you
want to
>>>> submit them together?
>>>
>>> I'd have to think about that. The other PEP I'm writing for
>>> per-interpreter GIL doesn't require immortal objects. They just
>>> simplify a number of things. That's my motivation for writing this
>>> PEP, in fact. :)
>>
>> Please think about it.
>> If you removed the benefits for per-interpreter GIL, the motivation
>> section would be reduced to is memory savings for fork/CoW. (And lots of
>> performance improvements that are great in theory but sum up to a 4%
loss.)
>
> Sounds good. Would this involve more than a note at the top of the PEP?
No, a note would work great. If you read the motivation carefully, it's
(IMO) clear that it's rather weak without the other PEP. But that
realization shouldn't come as a surprise to the reader.
> And just to be clear, I don't think the fate of a per-interpreter GIL
> PEP should not depend on this one.
I think that's clear.
It's other way around - the fate of this PEP will probably depend on the
per-interpreter GIL one.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/EFTWUNDK7PDELCZSDU6NMKA4W4VJ6BNT/
Code of Conduct: http://python.org/psf/codeofconduct/
