[issue40312] Weakref callbacks running before finalizers in GC collection

Tim Peters added the comment: A simple (finalizer-only) example of what an SCC-based DAG topsort ordering would accomplish: import gc class C: def __init__(self, val): self.val = val def __del__(self): print("finalizing", self.val) c, b,

[issue40312] Weakref callbacks running before finalizers in GC collection

Tim Peters added the comment: Well, the refcounts on _everything_ cyclic gc sees are greater than 0. Because if an object's refcount ever falls to 0 in CPython, reference counting deals with it immediately, and so it doesn't survive to participate in cyclic gc. IOW, absolutely everything

[issue40312] Weakref callbacks running before finalizers in GC collection

Allan Feldman added the comment: Yup agree with all the points above, just wanted to point out that I think self.value is strongly referenced (even though it's just internal references and will be collected by the gc) during Foo.__del__ execution (annotated code below), yet the

[issue40312] Weakref callbacks running before finalizers in GC collection

Pablo Galindo Salgado added the comment: > The result is that behavioral consistency becomes more difficult in > application code when using language provided structures such as > WeakValueDictionary. Well, you are already in tricky territory using finalizers. Note this sentence from the

[issue40312] Weakref callbacks running before finalizers in GC collection

Allan Feldman added the comment: I definitely understand the possibility that some code is relying on the current gc behavior of weakref callbacks being invoked after finalizers. That being said, the behavior is currently inconsistent between gc and reference counted paths. The language

[issue40312] Weakref callbacks running before finalizers in GC collection

Pablo Galindo Salgado added the comment: > Pablo, as above, I'm inclined to leave things alone unless we can "prove" no > current code could possibly be relying (even by accident) on that gc > currently runs callbacks before finalizers. Which may be the case! I don't > know ;-) I very

[issue40312] Weakref callbacks running before finalizers in GC collection

Tim Peters added the comment: Allan, we don't (at least not knowingly) write tests that rely on order of end-of-life actions, because the _language_ defines nothing about the order. So you can permute the order and it's unlikely any standard tests would fail. The only reason your example

[issue40312] Weakref callbacks running before finalizers in GC collection

Pablo Galindo Salgado added the comment: One thing we could do is call the weakref callbacks *after* we call `finalize_garbage` and only on the "final_unreachable" set (the objects that do not resurrect). Notice that doing this still has one difference: the callback will be executed AFTER

[issue40312] Weakref callbacks running before finalizers in GC collection

Pablo Galindo Salgado added the comment: I think this is a good summary of what you are referring to: >>> import gc, weakref >>> class Lazarus: ...def __del__(self): ... global x ... x = self ... >>> def callback(*args): ... print("DEAD") ... # No gc dead: >>> x = None

[issue40312] Weakref callbacks running before finalizers in GC collection

Allan Feldman added the comment: Also I just noticed this statement: > In current CPython, for your ForeverObject(False), `del o` does not make the > object trash "for real". __del__ runs immediately (due to deterministic, > synchronous reference counting) and resurrects it. That cuts off

[issue40312] Weakref callbacks running before finalizers in GC collection

Allan Feldman added the comment: Thanks for the explanation! I agree that "about to be finalized" is unclear in the docs :) I still believe that having different behavior for the ordering of finalizers versus weakref callbacks depending on whether the path is through gc versus reference

[issue40312] Weakref callbacks running before finalizers in GC collection

Tim Peters added the comment: Things get complicated here because in older versions of Python an instance of ForeverObject(True) could "leak" forever: if an object in a trash cycle had a __del__ method, that method would never be called, and the object would never be collected. Starting

[issue40312] Weakref callbacks running before finalizers in GC collection

Allan Feldman added the comment: Reading more carefully, I may have jumped to conclusions here :) Looking at the weakref docs, I see the following description of the callback functionality: > If callback is provided and not None, and the returned weakref object is > still alive, the

[issue40312] Weakref callbacks running before finalizers in GC collection

Tim Peters added the comment: Ah, I missed that `cache` is global. So it will hold reachable strong refs to the weakrefs stored for the dict's values. So gc will run the callbacks associated with weakrefs' trash referents. I think you're out of luck. Nothing is defined about the order in

[issue40312] Weakref callbacks running before finalizers in GC collection

Allan Feldman added the comment: Thanks for the response! > if a weakref W refers to object O, and W and O are _both_ in cyclic trash I believe that in the examples W is not in cyclic trash, but remains referenced as a global in the frame. Only O is in cyclic trash (O references itself). I

[issue40312] Weakref callbacks running before finalizers in GC collection

Tim Peters added the comment: Offhand I'm surprised because of this: if a weakref W refers to object O, and W and O are _both_ in cyclic trash, gc does not want to invoke W's callback (if any). In fact, it works hard not to call it. So I'm surprised that the callback is invoked at all,

[issue40312] Weakref callbacks running before finalizers in GC collection

Change by Ned Deily : -- Removed message: https://bugs.python.org/msg366681 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue40312] Weakref callbacks running before finalizers in GC collection

Change by Ned Deily : -- nosy: -ned.deily ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40312] Weakref callbacks running before finalizers in GC collection

Change by Ned Deily : -- Removed message: https://bugs.python.org/msg366682 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue40312] Weakref callbacks running before finalizers in GC collection

Change by Ned Deily : -- Removed message: https://bugs.python.org/msg366680 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue40312] Weakref callbacks running before finalizers in GC collection

Ned Deily added the comment: Going going gone -- nosy: +ned.deily ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue40312] Weakref callbacks running before finalizers in GC collection

deekay added the comment: Wrong thread, wrong thread, abort! Sorry, I meant to submit this as a separate issue. I don't see a delete option. Maybe a mod can remove this please? -- ___ Python tracker

[issue40312] Weakref callbacks running before finalizers in GC collection

deekay added the comment: I'm baffled by the performance difference of the following two semantically equivalent(?) programs. Python: #test.py import time starttime=time.time() import tensorflow print(f"Import took: {time.time() - starttime}") C using CPython

[issue40312] Weakref callbacks running before finalizers in GC collection

New submission from Allan Feldman : Our team is making use of a weakref.WeakValueDictionary() that is accessed through the finalizer of a class. We observed that in Python 3 occasionally values that are directly referenced by an object being finalized were missing from the