[Python-ideas] Re: weakref.link: Keep A alive while B is alive (and/or update WeakKeyDictionary)
Looks like a separate issue to me. That issue can be fixed by updating the Python implementation of WeakKeyDictioary. The fix for the WeakKeyDictionary issue I mentioned and the Ephemeron primitive (as Sebastian identified it) are equivalant (you can implement one from the other) and cannot be reduced to any existing primitive as far as I can tell. Such a change would need to happen within the Python engine itself. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KJUFOKVTU2YY32CS6TCRAN3P3XLBCJ42/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: weakref.link: Keep A alive while B is alive (and/or update WeakKeyDictionary)
Would fixing this help? https://bugs.python.org/issue44140 On Tue, 20 Jul 2021, 02:16 Sebastian Berg, wrote: > On Mon, 2021-07-19 at 22:24 +, Mark Gordon wrote: > > Proposal: > > > > Have a weakref.link (not at all attached to the naming) primitive > > that allows one to keep object A alive while object B is alive. This > > would be functionally similar to adding A as an attribute of B from > > the GC's perspective without making any assumptions about what > > attributes are available in B. Here's a really course example of how > > this might be used: > > > > Probably, the "stake holders" on this list are well aware. But, as far > as I understand, this is (or requires) an "Ephemeron": > > https://en.wikipedia.org/wiki/Ephemeron > > While I might be tempted to try/use it if Python had it (and was > available from C), I doubt that it actually matters in practice, for my > "use case". > > > The "use case", why I was curious about it is for multiple-dispatching > (to allow users to extend NumPy). > If you dispatch based on input types and the input type can be deleted, > the same problem arises: The (type specific) implementation is > expected to – and sometimes must – hold a strong reference to the input > types. Thus creating an unbreakable circle. > > (You can avoid this sometimes, but I don't think you can in general) > > An Ephemeron should be able to describe this correctly. I expect even a > situation like: > > func(A, B) -> (A, B, C) > > which can be deleted if either `A` or `B` is "almost collectable", but > must keep alive `C` if both `A` and `B` are alive. > > Cheers, > > Sebastian > > > > # Associate "extra_data" with "obj" where "extra_data" will lose its > > reference when "obj" is no longer reachable through strong refs. > > my_link = weakref.link(obj, extra_data) > > > > ... > > > > obj, extra_data = my_link() > > if obj is not None: > > do_something(obj, extra_data) > > > > > > This would also allow us to fix or create a new version of > > WeakKeyDictionary where values can have strong references back to > > their key without preventing their key from being reclaimed. This has > > been an issue for tensorflow in the past (see > > https://github.com/tensorflow/tensorflow/issues/42022) and is likely > > to be a subtle issue in many more libraries. Even the suggested use > > case in the docs "This can be especially useful with objects that > > override attribute accesses." is likely to be broken because of this > > issue. While I'd be weary of changing the semantics of an existing > > data structure it seems highly unlikely anything depends on this > > behavior since it makes WeakKeyDictionary degrade to behaving like a > > normal dictionary for that key. > > > > > > Notes: > > * Not attached to this specific approach or API > > * Mostly interesting in updating/creating new WeakKeyDictionary as > > mentioned > > ___ > > Python-ideas mailing list -- python-ideas@python.org > > To unsubscribe send an email to python-ideas-le...@python.org > > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > > Message archived at > > > https://mail.python.org/archives/list/python-ideas@python.org/message/4KBLRV7QYS36NVJNKAJIDCFTH4DF5YSY/ > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/7C7EVG5ALBP266T6FROR6L7IK4GULPIS/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AOFYS2KTGVEZJ2WFY7BOEUZIK2QGMJKY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: weakref.link: Keep A alive while B is alive (and/or update WeakKeyDictionary)
On Mon, 2021-07-19 at 22:24 +, Mark Gordon wrote: > Proposal: > > Have a weakref.link (not at all attached to the naming) primitive > that allows one to keep object A alive while object B is alive. This > would be functionally similar to adding A as an attribute of B from > the GC's perspective without making any assumptions about what > attributes are available in B. Here's a really course example of how > this might be used: > Probably, the "stake holders" on this list are well aware. But, as far as I understand, this is (or requires) an "Ephemeron": https://en.wikipedia.org/wiki/Ephemeron While I might be tempted to try/use it if Python had it (and was available from C), I doubt that it actually matters in practice, for my "use case". The "use case", why I was curious about it is for multiple-dispatching (to allow users to extend NumPy). If you dispatch based on input types and the input type can be deleted, the same problem arises: The (type specific) implementation is expected to – and sometimes must – hold a strong reference to the input types. Thus creating an unbreakable circle. (You can avoid this sometimes, but I don't think you can in general) An Ephemeron should be able to describe this correctly. I expect even a situation like: func(A, B) -> (A, B, C) which can be deleted if either `A` or `B` is "almost collectable", but must keep alive `C` if both `A` and `B` are alive. Cheers, Sebastian > # Associate "extra_data" with "obj" where "extra_data" will lose its > reference when "obj" is no longer reachable through strong refs. > my_link = weakref.link(obj, extra_data) > > ... > > obj, extra_data = my_link() > if obj is not None: > do_something(obj, extra_data) > > > This would also allow us to fix or create a new version of > WeakKeyDictionary where values can have strong references back to > their key without preventing their key from being reclaimed. This has > been an issue for tensorflow in the past (see > https://github.com/tensorflow/tensorflow/issues/42022) and is likely > to be a subtle issue in many more libraries. Even the suggested use > case in the docs "This can be especially useful with objects that > override attribute accesses." is likely to be broken because of this > issue. While I'd be weary of changing the semantics of an existing > data structure it seems highly unlikely anything depends on this > behavior since it makes WeakKeyDictionary degrade to behaving like a > normal dictionary for that key. > > > Notes: > * Not attached to this specific approach or API > * Mostly interesting in updating/creating new WeakKeyDictionary as > mentioned > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/4KBLRV7QYS36NVJNKAJIDCFTH4DF5YSY/ > Code of Conduct: http://python.org/psf/codeofconduct/ > signature.asc Description: This is a digitally signed message part ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7C7EVG5ALBP266T6FROR6L7IK4GULPIS/ Code of Conduct: http://python.org/psf/codeofconduct/