[Python-ideas] Re: weakref.link: Keep A alive while B is alive (and/or update WeakKeyDictionary)

2021-07-19 Thread Sebastian Berg
On Mon, 2021-07-19 at 22:24 +, Mark Gordon wrote:
> Proposal:
> 
> Have a weakref.link (not at all attached to the naming) primitive
> that allows one to keep object A alive while object B is alive. This
> would be functionally similar to adding A as an attribute of B from
> the GC's perspective without making any assumptions about what
> attributes are available in B. Here's a really course example of how
> this might be used:
> 

Probably, the "stake holders" on this list are well aware.  But, as far
as I understand, this is (or requires) an "Ephemeron":

https://en.wikipedia.org/wiki/Ephemeron

While I might be tempted to try/use it if Python had it (and was
available from C), I doubt that it actually matters in practice, for my
"use case".


The "use case", why I was curious about it is for multiple-dispatching
(to allow users to extend NumPy).
If you dispatch based on input types and the input type can be deleted,
the same problem arises:  The (type specific) implementation is
expected to – and sometimes must – hold a strong reference to the input
types.  Thus creating an unbreakable circle.

(You can avoid this sometimes, but I don't think you can in general)

An Ephemeron should be able to describe this correctly. I expect even a
situation like:

func(A, B) -> (A, B, C)

which can be deleted if either `A` or `B` is "almost collectable", but
must keep alive `C` if both `A` and `B` are alive.

Cheers,

Sebastian


> # Associate "extra_data" with "obj" where "extra_data" will lose its
> reference when "obj" is no longer reachable through strong refs.
> my_link = weakref.link(obj, extra_data)
> 
> ...
> 
> obj, extra_data = my_link()
> if obj is not None:
>   do_something(obj, extra_data)
> 
> 
> This would also allow us to fix or create a new version of
> WeakKeyDictionary where values can have strong references back to
> their key without preventing their key from being reclaimed. This has
> been an issue for tensorflow in the past (see 
> https://github.com/tensorflow/tensorflow/issues/42022) and is likely
> to be a subtle issue in many more libraries. Even the suggested use
> case in the docs "This can be especially useful with objects that
> override attribute accesses." is likely to be broken because of this
> issue.  While I'd be weary of changing the semantics of an existing
> data structure it seems highly unlikely anything depends on this
> behavior since it makes WeakKeyDictionary degrade to behaving like a
> normal dictionary for that key.
> 
> 
> Notes:
> * Not attached to this specific approach or API
> * Mostly interesting in updating/creating new WeakKeyDictionary as
> mentioned
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/4KBLRV7QYS36NVJNKAJIDCFTH4DF5YSY/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 



signature.asc
Description: This is a digitally signed message part
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7C7EVG5ALBP266T6FROR6L7IK4GULPIS/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] weakref.link: Keep A alive while B is alive (and/or update WeakKeyDictionary)

2021-07-19 Thread Mark Gordon
Proposal:

Have a weakref.link (not at all attached to the naming) primitive that allows 
one to keep object A alive while object B is alive. This would be functionally 
similar to adding A as an attribute of B from the GC's perspective without 
making any assumptions about what attributes are available in B. Here's a 
really course example of how this might be used:

# Associate "extra_data" with "obj" where "extra_data" will lose its reference 
when "obj" is no longer reachable through strong refs.
my_link = weakref.link(obj, extra_data)

...

obj, extra_data = my_link()
if obj is not None:
  do_something(obj, extra_data)


This would also allow us to fix or create a new version of WeakKeyDictionary 
where values can have strong references back to their key without preventing 
their key from being reclaimed. This has been an issue for tensorflow in the 
past (see https://github.com/tensorflow/tensorflow/issues/42022) and is likely 
to be a subtle issue in many more libraries. Even the suggested use case in the 
docs "This can be especially useful with objects that override attribute 
accesses." is likely to be broken because of this issue.  While I'd be weary of 
changing the semantics of an existing data structure it seems highly unlikely 
anything depends on this behavior since it makes WeakKeyDictionary degrade to 
behaving like a normal dictionary for that key.


Notes:
* Not attached to this specific approach or API
* Mostly interesting in updating/creating new WeakKeyDictionary as mentioned
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4KBLRV7QYS36NVJNKAJIDCFTH4DF5YSY/
Code of Conduct: http://python.org/psf/codeofconduct/