Re: [Python-Dev] Making proxy types easier to write and maintain
On Thu, Mar 20, 2014 at 11:56 AM, Larry Hastings la...@hastings.org wrote: On 03/20/2014 12:49 AM, Nick Coghlan wrote: So long as Graham's willing to go along with it, he doesn't have to to be the one to write the PEP. PEP? Why does it need a PEP? I didn't think it'd even be a new top-level library module, it'd be an addition / additions to functools. Would it make sense to have a new module named classtools rather than add more class-related stuff to functools. For instance, total_ordering() is the sort of thing I would expect to find in classtools rather than functools. -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On Wed, Mar 19, 2014 at 12:46 PM, Antoine Pitrou solip...@pitrou.net wrote: In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a proxy protocol (__proxy__ / tp_proxy) that would be used as a fallback by _PyObject_LookupSpecial to fetch the lookup target, i.e.: def _PyObject_LookupSpecial(obj, name): tp = type(obj) try: return getattr(tp, name) except AttributeError: return getattr(tp.tp_proxy(), name) What do you think? Regardless of the attr lookup in the interpreter, it would be nice to have a standardized explicit __proxied__ attribute on proxy objects (similar to __wrapped__). -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On 03/19/2014 08:49 PM, Nick Coghlan wrote: I had vague plans to bug Graham (yay, spelling!) about proposing wrapt for inclusion in 3.5 (possibly split between functools and types rather than as a completely new module, though). I pinged him about it for 3.4. He didn't have the time for it. I hope we can talk him into it for 3.5, wrapt is pretty sweet! //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On 20 March 2014 17:17, Larry Hastings la...@hastings.org wrote: On 03/19/2014 08:49 PM, Nick Coghlan wrote: I had vague plans to bug Graham (yay, spelling!) about proposing wrapt for inclusion in 3.5 (possibly split between functools and types rather than as a completely new module, though). I pinged him about it for 3.4. He didn't have the time for it. I hope we can talk him into it for 3.5, wrapt is pretty sweet! So long as Graham's willing to go along with it, he doesn't have to to be the one to write the PEP. Just needs someone that understands the problem it's designed to solve and is willing to write it up as a PEP, and look at the question of whether it makes more sense to drop it in wholesale as a new module, or to break it up amongst existing modules. (and python-ideas discussion can help with that). One particularly thorny question to investigate: would it be possible to *implicitly* switch functools.wraps() over to using it? I suspect the backwards compatibility implications wouldn't be acceptable, but the idea at least needs to be looked into to identify the specific problems with the concept. Cheers, Nick. /arry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On 03/20/2014 12:49 AM, Nick Coghlan wrote: So long as Graham's willing to go along with it, he doesn't have to to be the one to write the PEP. PEP? Why does it need a PEP? I didn't think it'd even be a new top-level library module, it'd be an addition / additions to functools. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
pypy's transparent proxy feature: http://pypy.readthedocs.org/en/latest/objspace-proxies.html#transparent-proxies On Thu, Mar 20, 2014 at 1:56 PM, Larry Hastings la...@hastings.org wrote: On 03/20/2014 12:49 AM, Nick Coghlan wrote: So long as Graham's willing to go along with it, he doesn't have to to be the one to write the PEP. PEP? Why does it need a PEP? I didn't think it'd even be a new top-level library module, it'd be an addition / additions to functools. /arry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Making proxy types easier to write and maintain
Hello, It is known to be cumbersome to write a proxy type that will correctly proxy all special methods (this has to be done manually on the type, since special methods are not looked up on the instance: a __getattr__ method would not work). Recently we've had reports of two stdlib types that forgot to implement some special methods: - weakref.proxy doesn't implement __reversed__: http://bugs.python.org/issue19359 - mock.MagicMock doesn't implement __truediv__: http://bugs.python.org/issue20968 In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a proxy protocol (__proxy__ / tp_proxy) that would be used as a fallback by _PyObject_LookupSpecial to fetch the lookup target, i.e.: def _PyObject_LookupSpecial(obj, name): tp = type(obj) try: return getattr(tp, name) except AttributeError: return getattr(tp.tp_proxy(), name) What do you think? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Making proxy types easier to write and maintain
On Wed Mar 19 2014 at 2:46:48 PM, Antoine Pitrou solip...@pitrou.net wrote: Hello, It is known to be cumbersome to write a proxy type that will correctly proxy all special methods (this has to be done manually on the type, since special methods are not looked up on the instance: a __getattr__ method would not work). Recently we've had reports of two stdlib types that forgot to implement some special methods: - weakref.proxy doesn't implement __reversed__: http://bugs.python.org/issue19359 - mock.MagicMock doesn't implement __truediv__: http://bugs.python.org/issue20968 In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a proxy protocol (__proxy__ / tp_proxy) that would be used as a fallback by _PyObject_LookupSpecial to fetch the lookup target, i.e.: def _PyObject_LookupSpecial(obj, name): tp = type(obj) try: return getattr(tp, name) except AttributeError: return getattr(tp.tp_proxy(), name) What do you think? Without having the code in front of me, would this only be for magic methods and attributes, or all attributes? IOW would this mean that if I assign an object to __proxy__ it would take care of the uses of __getattr__ for proxying? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On 19 March 2014 18:46, Antoine Pitrou solip...@pitrou.net wrote: In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a proxy protocol (__proxy__ / tp_proxy) that would be used as a fallback by _PyObject_LookupSpecial to fetch the lookup target, i.e.: def _PyObject_LookupSpecial(obj, name): tp = type(obj) try: return getattr(tp, name) except AttributeError: return getattr(tp.tp_proxy(), name) What do you think? Would that increase the size of type objects? Would that matter? (There's a similar question that came up in the thread about adding the @ operator over on python-ideas, which is what made me think of it...) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On Wed, 19 Mar 2014 19:32:50 + Brett Cannon bcan...@gmail.com wrote: In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a proxy protocol (__proxy__ / tp_proxy) that would be used as a fallback by _PyObject_LookupSpecial to fetch the lookup target, i.e.: def _PyObject_LookupSpecial(obj, name): tp = type(obj) try: return getattr(tp, name) except AttributeError: return getattr(tp.tp_proxy(), name) What do you think? Without having the code in front of me, would this only be for magic methods and attributes, or all attributes? IOW would this mean that if I assign an object to __proxy__ it would take care of the uses of __getattr__ for proxying? In a first approach it would be only for magic methods and attributes. It if proves successful, perhaps it can be later expanded to also be used for regular lookups. By the way, Benjamin pointed me to a prior discussion: http://bugs.python.org/issue643841 Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On Wed, 19 Mar 2014 20:15:15 + Paul Moore p.f.mo...@gmail.com wrote: On 19 March 2014 18:46, Antoine Pitrou solip...@pitrou.net wrote: In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a proxy protocol (__proxy__ / tp_proxy) that would be used as a fallback by _PyObject_LookupSpecial to fetch the lookup target, i.e.: def _PyObject_LookupSpecial(obj, name): tp = type(obj) try: return getattr(tp, name) except AttributeError: return getattr(tp.tp_proxy(), name) What do you think? Would that increase the size of type objects? Would that matter? (There's a similar question that came up in the thread about adding the @ operator over on python-ideas, which is what made me think of it...) One additional slot is one additional pointer field, which is mostly trivial in a type object. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On 20 Mar 2014 06:24, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 19 Mar 2014 19:32:50 + Brett Cannon bcan...@gmail.com wrote: In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a proxy protocol (__proxy__ / tp_proxy) that would be used as a fallback by _PyObject_LookupSpecial to fetch the lookup target, i.e.: def _PyObject_LookupSpecial(obj, name): tp = type(obj) try: return getattr(tp, name) except AttributeError: return getattr(tp.tp_proxy(), name) What do you think? Without having the code in front of me, would this only be for magic methods and attributes, or all attributes? IOW would this mean that if I assign an object to __proxy__ it would take care of the uses of __getattr__ for proxying? In a first approach it would be only for magic methods and attributes. It if proves successful, perhaps it can be later expanded to also be used for regular lookups. By the way, Benjamin pointed me to a prior discussion: http://bugs.python.org/issue643841 Graeme Dumpleton has also subsequently written a library to handle easier creation of correct proxy types: https://pypi.python.org/pypi/wrapt It isn't as simple as changing one lookup function though - abstract.c reads the type slots directly, and those are already hairy performance critical code paths (especially the binary operator type dispatch). Cheers, Nick. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On Thu, 20 Mar 2014 07:54:39 +1000 Nick Coghlan ncogh...@gmail.com wrote: Graeme Dumpleton has also subsequently written a library to handle easier creation of correct proxy types: https://pypi.python.org/pypi/wrapt (is Graeme another spelling for Graham?) It isn't as simple as changing one lookup function though - abstract.c reads the type slots directly, and those are already hairy performance critical code paths (especially the binary operator type dispatch). Oh indeed, you're right. I think my proposal is taking water. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On Wed, Mar 19, 2014 at 11:01:39PM +0100, Antoine Pitrou wrote: On Thu, 20 Mar 2014 07:54:39 +1000 Nick Coghlan ncogh...@gmail.com wrote: Graeme Dumpleton has also subsequently written a library to handle easier creation of correct proxy types: https://pypi.python.org/pypi/wrapt (is Graeme another spelling for Graham?) Yes. It isn't as simple as changing one lookup function though - abstract.c reads the type slots directly, and those are already hairy performance critical code paths (especially the binary operator type dispatch). Oh indeed, you're right. I think my proposal is taking water. Nevertheless, I would like to see a standard way to proxy dunder methods. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On 20 Mar 2014 10:34, Steven D'Aprano st...@pearwood.info wrote: On Wed, Mar 19, 2014 at 11:01:39PM +0100, Antoine Pitrou wrote: On Thu, 20 Mar 2014 07:54:39 +1000 Nick Coghlan ncogh...@gmail.com wrote: Graeme Dumpleton has also subsequently written a library to handle easier creation of correct proxy types: https://pypi.python.org/pypi/wrapt (is Graeme another spelling for Graham?) Yes. It isn't as simple as changing one lookup function though - abstract.c reads the type slots directly, and those are already hairy performance critical code paths (especially the binary operator type dispatch). Oh indeed, you're right. I think my proposal is taking water. Nevertheless, I would like to see a standard way to proxy dunder methods. I had vague plans to bug Graham (yay, spelling!) about proposing wrapt for inclusion in 3.5 (possibly split between functools and types rather than as a completely new module, though). functools.wraps was always intended as a near term workaround that, like most sufficiently effective workarounds, ended lasting years before its limitations irritated anyone enough for them to go back and do it right :) Cheers, Nick. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
