[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

On 28. 05. 21 6:55, Tim Peters wrote: [Pablo] Tim, check this out: import re, gc x = re.compile("x") gc.get_referents(x.__class__)[-1] Cool! So presumably this constructs a cycle involving a pattern object: import re p = re.compile("ab*c") import _sre _sre.WOWZA = p Indeed, under the

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

On Fri, May 28, 2021 at 6:55 AM Tim Peters wrote: > I suppose I could ask why heap types were fiddled to point to their > module objects too - but that's really got nothing to do with getting > the release done, so I won't :-) PyHeapTypeObject.ht_module was added by the PEP 573 "Module State

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

[Pablo] > Tim, check this out: > > >>> import re, gc > >>> x = re.compile("x") > >>> gc.get_referents(x.__class__)[-1] > Cool! So presumably this constructs a cycle involving a pattern object: import re p = re.compile("ab*c") import _sre _sre.WOWZA = p Indeed, under the current main branch:

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

{Tim] > So, on what principled basis do we exempt, say, ints from > participating in cyclic GC too? {Pablo] > In this case, the int object doesn't have a reference to its type because is > not a heap type > so that's fine. That baffled me at first, because _every_ object contains a pointer to

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

> So, on what principled basis do we exempt, say, ints from participating in cyclic GC too? Do we have to "just know" that a cycle can't be reached from an int's type object? If so, is that even true? Or just convenient to pretend to believe to avoid adding 16 more bytes to each int object and

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

;Victor Stinner ] > ... > For a more concrete example, read the "_thread lock traverse" section > of my article on these problems: > https://vstinner.github.io/subinterpreter-leaks.html > > There were two reference cycles, and both were "connected" with a lock > object in the middle (look at my

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

Tim, check this out: >>> import re, gc >>> x = re.compile("x") >>> gc.get_referents(x.__class__)[-1] That seems due to: https://github.com/python/cpython/blob/e90e0422182f4ca7faefd19c629f84aebb34e2ee/Objects/typeobject.c#L4241 On Thu, 27 May 2021 at 20:15, Tim Peters wrote: > [Tim] > >>

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

On Thu, May 27, 2021 at 8:24 PM Pablo Galindo Salgado wrote: > > And if a type pointer is the only thing being visited, then there's no > > point unless the object can itself be reachable from the type object. > > But that could happen easily for heap types as they are mutable by default. > For

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

[Tim] > Can you flesh this out for what stumbled into being my running example? That is, how could a regexp pattern object be part of a cycle? Found the problem that we saw regarding a cycle involving the type. That comes from this comment:

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

> Can you flesh this out for what stumbled into being my running example? That is, how could a regexp pattern object be part of a cycle? Let me try to remember when we saw this problem in the past, but on first sigh, it seems that indeed that cannot happen in the regular case. > And, in general,

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

[Tim] >> And if a type pointer is the only thing being visited, then there's no point >> unless the object can itself be reachable from the type object. {Pablo] > But that could happen easily for heap types as they are mutable by > default. For instance, you set the instance in a global: > > type

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

Sorry, the last link should have been: https://bugs.python.org/issue43908 On Thu, 27 May 2021 at 19:41, Pablo Galindo Salgado wrote: > > Modules dicts are cleared during interpreter shutdown to break > such cycles. > > That is precisely what's not working because of the cycles, check the >

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

> Modules dicts are cleared during interpreter shutdown to break such cycles. That is precisely what's not working because of the cycles, check the first message in the issue: https://bugs.python.org/msg385297 > Also: Why are types mutable ? AFAIK, they are only meant to be mutable at C level

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

On 27.05.2021 20:20, Pablo Galindo Salgado wrote: >> And if a type pointer is the only thing being visited, then there's > no point unless the object can itself be reachable from the type object. > > But that could happen easily for heap types as they are mutable by default. > For > instance,

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

> "Why?" is baffling to me: how could they possibly participate in a cycle? If the type object is a heap type (by default mutable), someone could just add a reference directly to it that makes it being in a cycle with the instance. Even if that's not the case, IIRC, as the type refers to the

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

> So all those instances have an increase in memory footprint compared to Python 3.7 ? I am afraid that's the case. This is one of the costs of making types not being heap types. On Thu, 27 May 2021, 19:04 Marc-Andre Lemburg, wrote: > On 27.05.2021 19:40, Tim Peters wrote: > > [Pablo Galindo

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

[Tim] >> But I think "waste of time" is the worst of it. Participating in >> cyclic gc does nothing to delay refcounting from recycling objects >> ASAP. gc only reclaims objects that are reachable only from dead >> cycles; everything else in CPython is reclaimed the instant its >> refcount falls

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

> And if a type pointer is the only thing being visited, then there's no point unless the object can itself be reachable from the type object. But that could happen easily for heap types as they are mutable by default. For instance, you set the instance in a global: type -> module -> globals ->

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

[Tim Peters ] > ... > This is, I believe, akin to what Marc-Andre is bringing up: if X > can't be reached _from_ X's type object, there's no need for X's > tp_traverse to visit X's type object. It _can_ be visited, but it > would be a waste of time. Ya, I need to retract that :-) If X's type

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

On 27.05.2021 19:40, Tim Peters wrote: > [Pablo Galindo Salgado ] >> Hi Marc, >> >> Yes, check out this from the 3.9 what's new document: >> >> https://docs.python.org/3/whatsnew/3.9.html#changes-in-the-c-api >> >> Instances of heap-allocated types (such as those created with >> PyType_FromSpec()

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

[Pablo Galindo Salgado ] > Hi Marc, > > Yes, check out this from the 3.9 what's new document: > > https://docs.python.org/3/whatsnew/3.9.html#changes-in-the-c-api > > Instances of heap-allocated types (such as those created with > PyType_FromSpec() > and similar APIs) hold a reference to their

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

Hi Marc, Yes, check out this from the 3.9 what's new document: https://docs.python.org/3/whatsnew/3.9.html#changes-in-the-c-api Instances of heap-allocated types (such as those created with PyType_FromSpec() and similar APIs) hold a reference to their type object since Python 3.8. As indicated

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

Hi Pablo, could you or Erlend please explain why types which don't reference any other objects need to participate in GC for deallocation ? Many PRs or checked in patches only do this: +static int +ucd_traverse(PreviousDBVersion *self, visitproc visit, void *arg) +{ +

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

The friendly reminder is so we don't forget that we are blocked by the issue. Also, a small and friendly ping the the core devs that made the changes in the first place that require this fix to get involved in the issue, as the process is blocked by changes they authored or reviewed and merged.

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

On 5/26/21 7:21 AM, Pablo Galindo Salgado wrote: Hi, Friendly reminder that the Python3.10 beta 2 is still blocked on: https://bugs.python.org/issue42972 Thanks for your help, Regards from stormy London, Pablo Galindo Salgado I took a quick look at

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

Hi, Friendly reminder that the Python3.10 beta 2 is still blocked on: https://bugs.python.org/issue42972 Thanks for your help, Regards from stormy London, Pablo Galindo Salgado On Mon, 24 May 2021 at 23:54, Pablo Galindo Salgado wrote: > Small correction: > >

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

On 25/05/2021 00.45, Pablo Galindo Salgado wrote: > Hi, > > Tomorrow is the scheduled release of Python 3.10 beta 2 but > unfortunately we have several release blockers: > > https://bugs.python.org/issue44043 : > 3.10 b1 armhf Bus Error in hashlib test:

[python-committers] Re: IMPORTANT: Python 3.10b2 release blockers

Small correction: https://bugs.python.org/issue40222: "Zero cost" exception handling and the segfaults on these buildbots: https://buildbot.python.org/all/#/builders/582/builds/165/steps/5/logs/stdio https://buildbot.python.org/all/#/builders/543/builds/190 are 3-11 (main branch) only, but