On Sat, 30 Oct 2021, Erik Demaine wrote:
Functions are already a form of deferred evaluation. PEP 671 is an
embellishment to this mechanism for some of the code in the function
signature to actually get executed within the body scope, *just like the body
of the function*.
I was thinking about what other forms of deferred evaluation Python has, and
ran into descriptors [https://docs.python.org/3/howto/descriptor.html].
Classes support this mechanism for calling arbitrary code when accessing the
attribute, instead of when calling the class:
```
class CallMeLater:
'''Descriptor for calling a specified function with no arguments.'''
def __init__(self, func):
self.func = func
def __get__(self, obj, objtype=None):
return self.func()
class Foo:
early_list = []
late_list = CallMeLater(lambda: [])
foo1 = Foo()
foo2 = Foo()
foo1.early_list == foo2.early_list == foo1.late_list == foo2.late_list
foo1.early_list is foo2.early_list # the same []
foo1.late_list is not foo2.late_list # two different []s
```
Written this way, it feels quite a bit like early and late arguments to me.
So this got me thinking:
What if parameter defaults supported descriptors? Specifically, something
like the following:
If a parameter (passed or defaulted) has a __get__ method, call it with
one argument (beyond self), namely, the function scope's locals().
Parameters are so processed in order from left to right.
(PEPs 549 and 649 are somewhat related in that they also propose extending
descriptors.)
This would enable the following hand-rolled late-bound defaults (using two
early-bound defaults):
```
def foo(early_list = [], late_list = CallMeLater(lambda: [])):
...
```
Or we could write a decorator to make this somewhat cleaner:
```
def late_defaults(func):
'''Convert callable defaults into late-bound defaults'''
func.__defaults__ = tuple(
CallMeLater(default) if callable(default) else default
for default in func.__defaults__
)
return func
@late_defaults
def foo(early_list = [], late_list = lambda: []):
...
```
It's also possible, but difficult, to write `end := len(a)` defaults:
```
class LateLength:
'''Descriptor for calling len(specified name)'''
def __init__(self, name):
self.name = name
def __get__(self, locals):
return len(locals[self.name])
def __repr__(self):
# This is bad form for repr, but it makes help(bisect)
# output the "right" thing: end=len(a)
return f'len({self.name})'
def bisect(a, start=0, end=LateLength('a')):
...
```
One feature/bug of this approach is that someone calling the function could
pass in a descriptor, and its __get__ method will get called by the function
(immediately at the start of the call). Personally I find this dangerous, but
those excited about general deferreds might like it? At least it's still
executing the function in its natural scope; it's "just" the locals() dict
that gets exposed, as an argument.
Alternatively, we could forbid this (at least for now): perhaps a __get__
method only gets checked and called on a parameter when that parameter has its
default value (e.g. `end is bisect.__defaults__[1]`). In addition to
feeling safer (to me), this would enable a lot of optimization:
* Parameters without defaults don't need any __get__ checking.
* Default values could be checked for the presence of a __get__ method at
function definition time (or when setting func.__defaults__), and that flag
could get checked at function call time, and __get__ semantics occur only when
that flag is set. (I'm not sure whether this would actually save time,
though. Maybe if it were a global flag for the function, "any
late-bound arguments here?". If not, old behavior and performance.)
This proposal could be compatible with PEP 671. What I find nice about this
proposal is that it's valid Python syntax today, just an extension of the data
model. But I wouldn't necessarily want to use the ugly incantations above,
and rather use some syntactic sugar on top of it -- and that's where PEP 671
could come in. What this proposal might offer is a *meaning* for that
syntactic sugar, which is more general and perhaps more Pythonic (building on
the existing Python data model). It provides another way to think about what
the notation in PEP 671 means, and suggests a (different) mechanism to
implement it.
Some nice features:
* __defaults__ naturally generalizes here; no need for auxiliary structures
or different signatures for __defaults__. A tool looking at __defaults__
could either be aware of descriptors in this context or not. All other
introspection should be the same.
* It becomes possible to skip a positional argument again: pass in the value
in __defaults__ and it will behave as if that argument wasn't passed.
* The syntactic sugar could build a __repr__ (or some new dunder like
__help__) that makes help() output the right thing, as in the example above.
The use of locals() (as an argument to __get__) is rather ugly, and probably
prevents name lookup optimization. Perhaps there's a better way, at least
with the syntactic sugar. For eaxmple, in CPython, late-bound defaults using
the syntactic sugar could compile the function to include some bytecode that
sets the __get__ function's frame to be the function's frame before it gets
called. Hmm, but then the function needs to know whether it's the default or
something else that got passed in...
What do people think? I'm still thinking about possible repurcussions, but it
seems like a promising direction to explore...
Erik
--
Erik Demaine | edema...@mit.edu | http://erikdemaine.org/
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/JQNSZFFYPVJBUEOX4ATSAPIQJNI53ACZ/
Code of Conduct: http://python.org/psf/codeofconduct/
