[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
Brendan Barnwell writes: > The ability to write something in the function signature that we > can already write in the body, and that quite naturally belongs in > the body, because it is executed when the function is called, not > when it is defined. I'm basically in sympathy with your conclusion, but I don't think it's useful to prejudice the argument by saying it *naturally belongs* in the body. Some languages quite naturally support thunks/blocks (Ruby) or even go so far as to equate code to data (Lisp), and execute that code in lieu of what appear to be variable references. But maybe it's *Pythonic* to necessarily place the source code in the body? I can't say that. > > I *really* don't like the idea that some types of object will be > > executed instead of being used, just because they have a flag > > set. >From a syntactic point of view, that's how Ruby blocks work. Closer to home, that's how properties work. And in the end, *all* objects are accessed by executing code. This is a distinction without a difference, except in our heads. I wouldn't want to be asked to explain the dividing line between objects that were "just used" and objects that were "produced by code that was executed instead of being just used". > I *really* don't like the idea that some types of argument will be > inlined into the function body instead of being stored as first-class > values like other `__defaults__`, just because there happens to be this > one extra character next to the equals sign in the function signature. > That strikes me as the sort of thing that should be incredibly > scary. Properties have *no* visible syntax if they're imported from a module. Properties are extremely useful, and we all use them all the time without noticing or caring. I see no reason in principle why the same kind of feature wouldn't be useful and just as invisible and just as "natural" for local or global variables -- or callable parameters, as long as properly restricted. Chris's proposal is nothing if not restricted! :-) My issues with Chris's proposal are described elsewhere, but I don't really see a problem in principle. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GSMTALVKO5SMP7JJFBQDG2CD352GRYDF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On 2021-12-01 at 17:16:34 +1100, Chris Angelico wrote: > *PEP 671: Syntax for late-bound function argument defaults* > > Questions, for you all: > > 1) If this feature existed in Python 3.11 exactly as described, would > you use it? No. I understand the arguments (pun intended) for the proposal, but I find none of them compelling. > 2) Independently: Is the syntactic distinction between "=" and "=>" a > cognitive burden? No. The biggest cognitive burden I have with either is the lack of white space around the = or =>, but that's a different problem. > (It's absolutely valid to say "yes" and "yes", and feel free to say > which of those pulls is the stronger one.) > > 3) If "yes" to question 1, would you use it for any/all of (a) mutable > defaults, (b) referencing things that might have changed, (c) > referencing other arguments, (d) something else? That depends on what you mean by "use." I wouldn't *write* code that uses it (I can't find many (if any) cases of (a), (b), or (c) in my code), but I would have to *read* other people's code that does. FWIW, the PEP doesn't mention mutability or mutable values at all. Also FWIW, I still think that if you're doing (b) or (c), then you're *not* doing default values anymore, you're moving pieces of the logic or the design into the wrong place. One example of (b) goes something like this: def write_to_log(event, time=>current_time()): actually_write_to_log(event, time) IOW, default to the current time, but allow the caller to specify a some other time instead. Maybe I'm old school, or overly pedantic, but IMO, those are two different use cases, and there should be two separate functions (potentially with separate authorization and/or notations in the log, or maybe I've spent too much time deciphering badly designed logs and log entries). *Maybe* a better example would be something like this: def write_to_log(event, id=>generate_appropriate_uuid()): actually_write_to_log(event, id) but I would still personally rather (for testability and maintainability reasons) write two functions, even (or perhaps especially) if they both called a common lower-level function to do the actual work. > 4) If "no" to question 1, is there some other spelling or other small > change that WOULD mean you would use it? (Some examples in the PEP.) No. > 5) Do you know how to compile CPython from source, and would you be > willing to try this out? Please? :) Yes, and no. (Seriously: Apparently, I don't create APIs, in any language, that would/could/might benefit from late binding default values. What would I be trying?) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JH6JSSIAQ7TIQJXO5C7ZAGASDV4CV6A2/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 review of default arguments evaluation in other languages
On Sun, Dec 5, 2021 at 3:39 PM David Mertz, Ph.D. wrote:
>
> On Sat, Dec 4, 2021 at 11:25 PM Chris Angelico wrote:
>>
>> > def add(a, b):
>> > return a+b
>> > How could you write that differently with your PEP
>>
>> I wouldn't. There are no default arguments, and nothing needs to be changed.
>
>
> I do recognize that I *could* call that with named arguments. I also
> recognize that the long post I wrote in the bath from my tablet is rife with
> embarrassing typos :-).
>
> Technically, I'd need `def add(a, b, /)` to be positional-only. But in
> practice, almost everyone who writes or calls a function like that passes by
> position. I'm not sure that I've *ever* actually used the explicit
> positional-only `/` other than to try it out. If I have, it was rare enough
> that I had to look it up then, as I did just now.
>
No problem. Doesn't really matter. In any case, argument defaults have
always been orthogonal with parameter passing styles (named or
positional), and that's not changing.
>> Actually PEP 671 applies identically to arguments passed by name or
>> position, and identically to keyword-only, positional-or-keyword, and
>> positional-only parameters.
>>
>> >>> def f(a=>[], /, b=>{}, *, c=>len(a)+len(b)):
>> ... print(a, b, c)
>
>
> Wow! That's an even bigger teaching nightmare than I envisioned in my prior
> post. Nine (3x3) different kinds of parameters is already too big of a
> cognitive burden. Doubling that to 18 kinds makes me shudder. I admit I sort
> of blocked out the positional-only defaults thing.
>
> I understand that it's needed to emulate some of the builtin or standard
> library functions, but I would avoid allowing that in code review...
> specifically because of the burden on future readers of the code.
>
What you're missing here is that it's not 3x3 becoming 3x3x2.
Everything is completely orthogonal. For instance, we don't consider
string arguments to be a fundamentally different thing from integer
arguments:
def f(text, times): ...
f("spam", 5)
They're just... arguments! And it's not massively more cognitive load
to be able to pass string arguments by name or position AND to be able
to pass integer arguments by name or position.
f(text="spam", times=5)
f("spam", times=5)
That's not a problem, because the matrix is absolutely complete: there
is no way in which these interact whatsoever.
It's the same with positional and named parameters: the way the
language assigns argument values to parameters is independent of the
types of those objects, whether they have early-bound defaults,
whether they have late-bound defaults, whether they have annotations,
and whether the function returns "Hello world". It's not quadratic
cognitive load to comprehend this. It is linear. The only thing you
need to understand is the one thing you're looking at right now.
With function defaults, it is currently the case that any parameter
can have a default, so long as there are no positional parameters to
its right which lack defaults. That's the only restriction on default
arguments. And that restriction is not changing at all by my proposal:
it is neither weakened nor strengthened by the fact that the default
might be an expression rather than a precomputed value.
(By the way, if I ever get the words "argument" and "parameter" wrong,
my apologies; but truth be told, everyone does that. The Python
grammar specifies that a def statement has a block of params, which is
of type arguments_ty. So that's a thing.)
CPython currently states that a function's parameters consist of three
groups (or five, kinda):
def func(pos_only, /, pos_or_kwd, *, kwd_only): ...
def func(pos_only, /, pos_or_kwd, *args, kwd_only, **kwargs): ...
Inside each group (not counting the collectors *args and **kwargs if
present), legality is defined by... well, I'll just quote the grammar
file:
# There are three styles:
# - No default
# - With default
# - Maybe with default
#
# There are two alternative forms of each, to deal with type comments:
# - Ends in a comma followed by an optional type comment
# - No comma, optional type comment, must be followed by close paren
# The latter form is for a final parameter without trailing comma.
And if you've never thought about type comments, don't worry, neither
had I till I was tinkering with the grammar, and we can for the most
part ignore them. :)
Either the parameter has a default, or it doesn't. A "maybe with
default" either looks like a "with default" or a "no default" (and
grammatically, it's there to handle that one restriction of "def
f(a=1, b):" being invalid, but everything else is fine). I'm not
changing any of that. All I'm changing is the default itself:
default[default_ty]:
| '=' a=expression { _PyPegen_arg_default(p, a, DfltValue) }
| '=' '>' a=expression { _PyPegen_arg_default(p, a, DfltExpr) }
When a parameter has a default, two options: either it's a default
value, or it's a default expression. This change doesn't care what
[Python-ideas] Re: PEP 671 review of default arguments evaluation in other languages
On Sat, Dec 4, 2021 at 11:25 PM Chris Angelico wrote:
> > def add(a, b):
> > return a+b
> > How could you write that differently with your PEP
>
> I wouldn't. There are no default arguments, and nothing needs to be
> changed.
>
I do recognize that I *could* call that with named arguments. I also
recognize that the long post I wrote in the bath from my tablet is rife
with embarrassing typos :-).
Technically, I'd need `def add(a, b, /)` to be positional-only. But in
practice, almost everyone who writes or calls a function like that passes
by position. I'm not sure that I've *ever* actually used the explicit
positional-only `/` other than to try it out. If I have, it was rare
enough that I had to look it up then, as I did just now.
Actually PEP 671 applies identically to arguments passed by name or
> position, and identically to keyword-only, positional-or-keyword, and
> positional-only parameters.
>
> >>> def f(a=>[], /, b=>{}, *, c=>len(a)+len(b)):
> ... print(a, b, c)
>
Wow! That's an even bigger teaching nightmare than I envisioned in my
prior post. Nine (3x3) different kinds of parameters is already too big of
a cognitive burden. Doubling that to 18 kinds makes me shudder. I admit I
sort of blocked out the positional-only defaults thing.
I understand that it's needed to emulate some of the builtin or standard
library functions, but I would avoid allowing that in code review...
specifically because of the burden on future readers of the code.
--
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons. Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/GH7TDMDRW3KNAXY5S4DOAXHCJPIL6EEM/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 review of default arguments evaluation in other languages
On Sun, Dec 5, 2021 at 3:17 PM David Mertz, Ph.D. wrote:
>
> On Sat, Dec 4, 2021, 11:13 PM Chris Angelico
>>
>> Not sure I'm understanding you correctly; in what way are named parameters
>> relevant here?
>
>
> def add(a, b):
> return a+b
>
> How could you write that differently with your PEP
I wouldn't. There are no default arguments, and nothing needs to be changed.
> (which only pertains to named parameters, not positional)?
Actually PEP 671 applies identically to arguments passed by name or
position, and identically to keyword-only, positional-or-keyword, and
positional-only parameters.
>>> def f(a=>[], /, b=>{}, *, c=>len(a)+len(b)):
... print(a, b, c)
...
>>> f()
[] {} 0
>>> f([1,2,3])
[1, 2, 3] {} 3
>>> f(b={"a":1})
[] {'a': 1} 1
>>> f(c=42)
[] {} 42
>>>
You can put early-bound or late-bound defaults on all three types of
parameter, and you can either provide or omit both kinds of argument;
the entire matrix has always been possible [1], and the entire matrix
will continue to be possible.
ChrisA
[1] For values of "always" that go back as far as PEP 570, at least,
since that's when pos-only params came in. Before that, it was a 2x2
matrix.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/NRU4C7V2LKLA6WON6UC5IYS3A62D5H2Z/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 review of default arguments evaluation in other languages
On Sat, Dec 4, 2021, 11:13 PM Chris Angelico > Not sure I'm understanding you correctly; in what way are named parameters > relevant here? > def add(a, b): return a+b How could you write that differently with your PEP (which only pertains to named parameters, not positional)? > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QFBTXHFPQRFKKGDJODYPII2T7S2OCURJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sun, Dec 5, 2021 at 3:08 PM Stephen J. Turnbull wrote: > > Barry Scott writes: > > > There are many possible implementation of the late bound idea that > > could create an object/default expression. > > But is it reasonable to bother with that added > > complexity/maintenance burden for a first implementation. > > Yes. If you don't do it, you'll have backward compatibility issues or > technical debt. > > I'm not saying that's a compelling argument here, except that one of > the main alleged problems is that users don't understand mutable > defaults. So adding more and more layers of support for default > arguments is making matters worse, I suspect. (Remember, they're > going to be reading "arg=None" and "@arg=[]" for a long long time.) > > This one is Worth Doing Right the first time, I think. And IMO David > Mertz is right: doing it right means a more general deferred-evaluation > object (not to be confused with Deferreds that need to be queried > about their value). If you think that deferred evaluation objects are the right way to do it, then write up a proposal to compete with PEP 671. In my opinion, it is a completely independent idea, which has its own merit, and which is not a complete replacement for late-bound defaults; the two could coexist in Python simultaneously, or either one could be accepted without the other, or we could continue to have neither. Yes, there's some overlap in the problems they solve, just as there's overlap between PEP 671 and PEP 661 on named sentinels; but there's also overlap between plenty of other language features, and we don't deem try/finally or context managers to be useless because of the other. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KWGOMJRXOLLZXNQH33IV7OFVM7TBQ7LZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 review of default arguments evaluation in other languages
On Sun, Dec 5, 2021 at 3:03 PM David Mertz, Ph.D. wrote: > Probably fewer than half of functions I've written use named parameters at > all. > Not sure I'm understanding you correctly; in what way are named parameters relevant here? ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VI2WUA6IGHMXNU3RRXDKEYHNR56DVUZG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
Barry Scott writes: > There are many possible implementation of the late bound idea that > could create an object/default expression. > But is it reasonable to bother with that added > complexity/maintenance burden for a first implementation. Yes. If you don't do it, you'll have backward compatibility issues or technical debt. I'm not saying that's a compelling argument here, except that one of the main alleged problems is that users don't understand mutable defaults. So adding more and more layers of support for default arguments is making matters worse, I suspect. (Remember, they're going to be reading "arg=None" and "@arg=[]" for a long long time.) This one is Worth Doing Right the first time, I think. And IMO David Mertz is right: doing it right means a more general deferred-evaluation object (not to be confused with Deferreds that need to be queried about their value). > And maybe no one will care enough to ever implement the ability to > modify the code of a late bound variables expression as a separate > object later. Hear! Hear! That's exactly how I feel about *this* proposal! With all due respect to Chris and Steve who have done great work advocating, implementing, and clarifying the proposal, IAGNU (I am gonna not use). Too much muscle memory, and more important, existing code whose style I want to be consistent and don't wanna mess with because it works, around "arg=None". ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OL3ZZPOFLBPSY32TDH5IOFDUJ5FKMJCA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 review of default arguments evaluation in other languages
On Sat, Dec 4, 2021, 10:14 PM Rob Cliffe via Python-ideas > The designers of 12 languages have chosen to provide late binding; those > of 3 or 4 have provided early binding. > I think this is at least tenuous evidence in favour of my belief that late > binding is more useful than early binding. > As the person probably most vociferous in opposing this PEP, I absolutely agree that late-binding is more useful. If I were creating a new programming language today, I would certainly make arguments be evaluated on call, not on definition. There are perfectly good ways to "fake" either one if you only have the other. Probably more work is needed to simulate early binding, but there are ways to achieve the same effect. However, that language would not be Python. That ship sailed in 1991. What's being discussed here isn't changing the behavior of binding in `def f(foo=bar)`. Instead, it's a discussion of adding ADDITIONAL syntax for late-binding behavior. I think the proposed syntax is the worst of all the options discussed. But the real issue is that the cases where it is relevant are vanishingly rate, and the extra cognitive, teaching, and maintenance burden is significant. In 90%+ of the functions I've written, default arguments are non-sentinel immutable values. If those were late bound, nothing whatsoever would change. Yes, maybe slightly different bytecodes would exist, but at the Python level, everything works work the same. So this issue only issue is only remotely relevant to <10% of functions with default arguments. However, of those <10%, 98% work perfectly fine with None as a sentinel. Probably fewer than half of functions I've written use named parameters at all. In other words, for somewhere fewer than one in a thousand functions, this new syntax might serve any purpose at all. That purpose is predominantly "avoid using a custom sentinel." A custom sentinel is a SMALL lift. I agree that a custom sentinel, while rare, is a slight wart in a program. I also believe that in this 1/1000 case, there could be a slightly prettier automatic docstrings. But not prettier than writing an explicit docstring in any case. The cost here is that EVERY SINGLE student learning Python needs to add this new construct to their mental load. EVERY book and tutorial needs to be updated. EVERY experienced developer has to spend extra effort understanding and writing code. The COST of implementing this PEP is *quite literally* tens of millions of person days. The benefit is a rare savings of two lines of function body code or a docstring line. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EHLOIHUL4B56TDI7PRYYJXMXVXYORSCQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 review of default arguments evaluation in other languages
On Sun, Dec 5, 2021 at 2:14 PM Rob Cliffe via Python-ideas wrote: > > Thank you for doing this research, Steven. > The designers of 12 languages have chosen to provide late binding; those > of 3 or 4 have provided early binding. > I think this is at least tenuous evidence in favour of my belief that > late binding is more useful than early binding. Perhaps, but more importantly, it provides strong evidence that late-binding of argument defaults is a real, viable concept and not a hack. (I also find it notable that quite a few of those blog posts, and even the JavaScript language reference on MDN, call out Python as having surprising behaviour. To people coming from those languages, Python's current behaviour is a gotcha, which would become a much smaller one if late-binding were a language-supported feature.) ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DFMINR3AO753VEG6HS7I44W6YIVZ62OY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 review of default arguments evaluation in other languages
Thank you for doing this research, Steven.
The designers of 12 languages have chosen to provide late binding; those
of 3 or 4 have provided early binding.
I think this is at least tenuous evidence in favour of my belief that
late binding is more useful than early binding.
Best wishes
Rob Cliffe
On 03/12/2021 21:05, Steven D'Aprano wrote:
A woefully incomplete review of default argument evaluation in other
languages. Updates and corrections are welcome.
Out of 22 languages apart from Python:
- 3 use early binding (default is evaluated at compile or function
definition time);
- 12 use late binding (default is evaluated at call time);
- 1 simulates late binding with a standard idiom;
- and 6 do not support default arguments.
Note that R's model for defaults in particularly interesting.
Early binding
-
PHP:
function f($arg = const) {body}
PHP default arguments must be constant expressions, not variables or
function calls. I infer from this that they are evaluated at function
definition time (compile time?).
https://www.php.net/manual/en/functions.arguments.php#functions.arguments.default
Dart:
f({arg=const}) {body}
Dart default values appear to be restricted to constants, by which I
infer that they are evaluated at compile-time.
Visual Basic:
Sub F(Optional arg As Type = constant)
body
End Sub
VB default values are restricted to constants, by which I infer that
they are evaluated at compile time.
https://docs.microsoft.com/en-us/dotnet/visual-basic/language-reference/modifiers/optional
Late binding
Javascript:
function f(arg=expression) {
body
}
ECMAScript 2015 (ES6) introduced default arguments to Javascript.
Javascript default arguments are evaluated when the function is called
(late binding).
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Default_parameters
https://dev.to/kenbellows/javascript-vs-python-default-function-parameter-values-7dc
CoffeeScript:
f = (arg = expression) ->
body
which compiles to JavaScript:
f = function(arg) {
if (arg == null) {
arg = expression;
}
body;
};
CoffeeScript default arguments are evaluated when the function is called.
https://stackoverflow.com/questions/23763825/coffeescript-default-arguments
C++:
void f(type arg = expression); {body}
C++ default arguments are evaluated when the function is called (late
binding).
The rules for C++ default arguments are complicated, for example local
variables *usually* cannot be used in the default expression.
https://en.cppreference.com/w/cpp/language/default_arguments
https://edux.pjwstk.edu.pl/mat/260/lec/PRG2CPP_files/node61.html
Ruby:
def f(arg = expression)
body
end
Ruby default values are evaluated when the function is called.
https://asquera.de/blog/2012-06-29/2-detect-default-argument-evaluation/
Kotlin:
fun f(arg: Type = expression) {body}
Kotlin default arguments are evaluated when the function is called.
https://kotlinlang.org/spec/expressions.html#function-calls-and-property-access
Elixir:
def f(arg \\ expression) do
body
end
Elixir default arguments are evaluated when the function is called.
https://til.mirego.com/2021-07-14-elixir-and-default-argument-evaluation
https://hexdocs.pm/elixir/Kernel.html#def/2-default-arguments
Scala:
def f(arg: Type = expression) : Type = {body}
Scala default arguments are evaluated when the function is called.
https://docs.scala-lang.org/sips/named-and-default-arguments.html#default-arguments
D:
void f(Type arg = value) {body}
It is not entirely clear to me when D evaluates default arguments, but
I think it is when the function is called.
https://dlang.org/spec/function.html#function-default-args
Julia:
function f(arg::Type=expression)
body
end
Julia default arguments are evaluated when the function is called.
https://docs.julialang.org/en/v1/manual/functions/
Swift:
func f(arg: Type = expression) {body}
Swift default arguments are evaluated when the function is called.
https://docs.swift.org/swift-book/ReferenceManual/Declarations.html#//apple_ref/doc/uid/TP40014097-CH34-ID472
https://stackoverflow.com/questions/38464715/when-are-swift-function-default-parameter-values-evaluated/38464716
Raku (Perl 6):
sub f($arg = expression) {body}
It is unclear to me when Raku evaluates default arguments, but I think
it is when the function is called.
https://raku.guide/#_default_and_optional_parameters
https://perl6advent.wordpress.com/2009/12/09/day-9-having-beautiful-arguments-and-parameters/
R:
f <- function(arg=expression) {body}
Default arguments in R are evaluated at need, at call time (lazy late
binding). Because they are not evaluated until the argument is needed,
they can refer to local variables defined in the body of the
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sun, Dec 5, 2021 at 11:34 AM Steven D'Aprano wrote:
>
> On Sat, Dec 04, 2021 at 10:50:14PM +1100, Chris Angelico wrote:
>
> > > syntactic sugar for this:
> > >
> > > def f(b, x=lambda b: a+b): ...
> > >
> > > except that the lambda has the LB flag set.
> >
> > Okay. So the references to 'a' and 'b' here are one more level of
> > function inside the actual function we're defining, which means you're
> > paying the price of nonlocals just to be able to late-evaluate
> > defaults. Not a deal-breaker, but that is a notable cost (every
> > reference to them inside the function will be slower).
>
> How much slower? By my tests:
>
> - access to globals is 25% more expensive than access to locals;
> - access to globals is 19% more expensive than nonlocals;
> - and nonlocals are 6% more expensive than locals.
>
> Or if you do the calculation the other way (the percentages don't match
> because the denominators are different):
>
> - locals are 20% faster than globals;
> - and 5% faster than nonlocals;
> - nonlocals are 16% faster than globals.
>
> Premature optimization is the root of all evil.
>
> We would be better off spending effort making nonlocals faster for
> everyone than throwing out desirable features and a cleaner design just
> to save 5% on a microbenchmark.
Fair, but the desirable feature can be achieved without this cost, and
IMO your design isn't cleaner than the one I'm already using, and 5%
is a lot for no benefit.
> [...]
> > What this does mean, though, is that there are "magic objects" that
> > cannot be used like other objects.
>
> NotImplemented says hello :-)
Good point. Still, I don't think we want more magic like that.
> And if you still think that we should care, we can come up with a more
> complex trigger condition:
>
> - the parameter was flagged as using a late-default;
> - AND the default is a LB function.
>
> Problem solved. Now you can use LB functions as early-bound defaults,
> and all it costs is to record and check a flag for each parameter. Is it
> worth it? Dunno.
Uhh so. the parameter has to be flagged AND the value has to
be flagged? My current proposal just flags the parameter. So I ask
again: what are you gaining by this change? You've come right back to
where you started, and added extra costs and requirements, all for
what?
> [...]
> > > The default expression is just a function (with the new LB flag set). So
> > > we can inspect its name, its arguments, its cell variables, etc:
> > >
> > > >>> default_expression.__closure__
> > > (,)
> > >
> > > We can do anything that we could do with any other other function object.
> >
> > Yup. As long as it doesn't include any assignment expressions, or
> > anything else that would behave differently.
>
> I don't get what you mean here. Functions with the walrus operator are
> still just functions that we can introspect:
>
> >>> f = lambda a, b: (len(w:=str(a))+b)*w
> >>> f('spam', 2)
> 'spamspamspamspamspamspam'
> >>> f.__code__
> at 0x7fc945e07c00, file "", line 1>
>
> What sort of "behave differently" do you think would prevent us from
> introspecting the function object? "Differently" from what?
Wrapping it in a function means the walrus would assign in that
function's context, not the outer function. I think it'd be surprising
if this works:
def f(x=>(a:=1)+a): # default is 2
but this doesn't:
def g(x=>(a:=1), y=>a): # default is UnboundLocalError
It's not a showstopper, but it is most definitely surprising.
The obvious solution is to say that, in this context, a is a nonlocal.
But this raises a new problem: The function object, when created, MUST
know its context. A code object says "this is a nonlocal", and a
function object says "when I'm called, this is my context". Which
means you can't have a function object that gets called externally,
because it's the code, not the function, that is what you need here.
And that means it's not directly executable, but it needs a context.
So, once again, we come right back around to what I have already: code
that you can't lift out and call externally. The difference is that,
by your proposal, there's a lot more overhead, for the benefit of
maybe under some very specific circumstances being able to synthesize
the result.
> > We also need to
> > have these special functions that get stored as separate code objects.
>
> That's not a cost, that's a feature. Seriously. We're doing that so that
> we can introspect them individually, not just as the source string, but
> as actual callable objects that can be:
>
> - introspected;
>
> - tested;
>
> - monkey-patched and modified in place (to the degree that any function
> can be modified, which is not a lot);
>
> - copied or replaced with a new function.
>
> Testing is probably the big one. Test frameworks will soon develop a way
> to let you write tests to confirm that your late bound defaults do what
> you expect them to do.
>
> That's trivial for `arg=[]` expressions, but for complex expressions in
>
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sat, Dec 04, 2021 at 10:50:14PM +1100, Chris Angelico wrote:
> > syntactic sugar for this:
> >
> > def f(b, x=lambda b: a+b): ...
> >
> > except that the lambda has the LB flag set.
>
> Okay. So the references to 'a' and 'b' here are one more level of
> function inside the actual function we're defining, which means you're
> paying the price of nonlocals just to be able to late-evaluate
> defaults. Not a deal-breaker, but that is a notable cost (every
> reference to them inside the function will be slower).
How much slower? By my tests:
- access to globals is 25% more expensive than access to locals;
- access to globals is 19% more expensive than nonlocals;
- and nonlocals are 6% more expensive than locals.
Or if you do the calculation the other way (the percentages don't match
because the denominators are different):
- locals are 20% faster than globals;
- and 5% faster than nonlocals;
- nonlocals are 16% faster than globals.
Premature optimization is the root of all evil.
We would be better off spending effort making nonlocals faster for
everyone than throwing out desirable features and a cleaner design just
to save 5% on a microbenchmark.
[...]
> What this does mean, though, is that there are "magic objects" that
> cannot be used like other objects.
NotImplemented says hello :-)
You are correct that one cannot use a LB function as a standard, early
bound default without triggering the "evaluate this at call time"
behaviour. If we're happy with this behaviour, it would need to be
documented for people to ignore *wink*
There's precedence though. You cannot overload an operator method to
return NotImplemented without triggering the special "your object
doesn't support this operator" behaviour.
And there are two obvious workarounds:
1. Just pass the LB function in as an explicit argument. The trigger
only operates when looking up a default, not on every access to a
function.
2. Or you can wrap the LB function you actually want to be the default
in a late-bound expression that returns that function.
And if you still think that we should care, we can come up with a more
complex trigger condition:
- the parameter was flagged as using a late-default;
- AND the default is a LB function.
Problem solved. Now you can use LB functions as early-bound defaults,
and all it costs is to record and check a flag for each parameter. Is it
worth it? Dunno.
[...]
> > The default expression is just a function (with the new LB flag set). So
> > we can inspect its name, its arguments, its cell variables, etc:
> >
> > >>> default_expression.__closure__
> > (,)
> >
> > We can do anything that we could do with any other other function object.
>
> Yup. As long as it doesn't include any assignment expressions, or
> anything else that would behave differently.
I don't get what you mean here. Functions with the walrus operator are
still just functions that we can introspect:
>>> f = lambda a, b: (len(w:=str(a))+b)*w
>>> f('spam', 2)
'spamspamspamspamspamspam'
>>> f.__code__
at 0x7fc945e07c00, file "", line 1>
What sort of "behave differently" do you think would prevent us from
introspecting the function object? "Differently" from what?
> Great. So now we have some magnificently magical behaviour in the
> language, which will have some nice sharp edge cases, but which nobody
> will ever notice. Totally. I'm sure.
NotImplemented. Document it and move on.
There are two work-arounds for those who care. And if you still think it
matters, you can record a flag for each parameter recording whether it
actually used a late-bound default or not.
> Plus, we pay a performance price
> in any function that makes use of argument references, not just for
> the late-bound default, but in the rest of the code.
Using a late-bound default doesn't turn every local variable in your
function into a cell variable. For any function that does a meaningful
amount of work, the cost of making one or two parameters into cell
variables instead of local variables is negligible.
At worst, if you do *no other work at all*, it's a cost of about 5% on
two-fifths of bugger-all. But if your function does a lot of real work,
the difference between using cell variables instead of locals is going
to be insignificant compared to ~~the power of the Force~~ the rest of
the work done in the function.
And if you have some unbelievably critical function that you need to
optimize up the wahzoo?
def func(a, b=None):
if b is None:
# Look ma, no cell variables!
b = expression
Python trades off convenience for speed and safety all the time.
This will just be another such example. You want the convenience of a
late-bound default? Use this feature. You want it to be 3ns faster?
Use the old "if arg is None" idiom.
Or write your code in C, and make it 50ns faster.
> We also need to
> have these special functions that get stored as separate code objects.
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sun, Dec 5, 2021 at 5:29 AM Barry Scott wrote: > > > > > On 1 Dec 2021, at 06:16, Chris Angelico wrote: > > > > I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/ > > with some additional information about the reference implementation, > > and some clarifications elsewhere. > > (I suspect that there was a reply that I should be replying to but, cannot > find one appropriate) > > I have a lot of code that exploits the fact that passing an explicit None > will cause the early bound default idiom to set the default for me. > > def inner(timestamp=None): > if timestamp is None: > timestamp = time.time() > do_stuff... > > def outer(timestamp=None): > inner(timestamp=timestamp) > > outer can in an idiomatic way have inner default timestamp and not have to > know what that means. If you need outer() to be able to have a value that means "use the default", then there are three options: 1) Don't pass timestamp at all. In simple cases where it will only and always specify the default, this is fine. 2) Define a sentinel that is indeed part of your API. 3) Use *args or **kwargs to choose whether to pass it or not (best if there are multiple of them). You can continue to use the existing system of "if none, do this", or you can flip it around and have the sentinel as a special token within your code: def inner(timestamp=>time.time()): if timestamp is None: timestamp = time.time() Depends on how important this feature is outside of your own helper functions. (I would probably not do this for None specifically - if it's purely internal, I'm more likely to use a dedicated local sentinel object.) But as soon as there are two or three arguments that "might have to be passed, might not", it's far more readable to use kwargs to pass just the ones you want. def outer(**kwargs): inner(**kwargs) That way, if something changes in inner(), you don't have to worry about breaking your caller's API. > With late bound I cannot do this without more complex pattern of building an > arg list. > > What if passing None still worked? I know the argument that there are more > sentinels then None. > > def inner(timestamp=>time.time()) > do_stuff... > > def outer(timestamp=None): > inner(timestamp=timestamp) > > The code in inner that decides to when to allow the default could check for > timestamp being > missing or arg present and None. > > Would the lack of support for other sentinels out weight the simple way to > get the default applied? > None is most assuredly not going to trigger a late-bound default. Python is not JavaScript :) ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QWYXRITR56CKURYKE7CKQ7A4WVNTUVJL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sun, Dec 5, 2021 at 6:16 AM Brendan Barnwell wrote:
>
> On 2021-12-04 03:50, Chris Angelico wrote:
> > On Sat, Dec 4, 2021 at 8:48 PM Steven D'Aprano wrote:
> >> And third, when the interpreter fetches a default from
> >> func.__defaults__, if it is a LB function, it automatically calls that
> >> function with the parameters to the left of x (which in this case
> >> would be just b).
> >
> > Plausible. Okay.
> >
> > What this does mean, though, is that there are "magic objects" that
> > cannot be used like other objects. Consider:
>
> Your proposal also has the same problem, since it involves "magic
> functions" that do not have usable values for their argument defaults,
> instead having some kind of Ellipsis two-step. It's all a matter of
> what you consider magic.
My proposal allows any object to be used as a function default
argument. There's a minor technical difference that means that there's
a second lookup if you use Ellipsis, but you can still use Ellipsis
just fine.
>>> def f(x=...):
... print(type(x), x)
...
>>> f()
Ellipsis
>>> f(None)
None
>>> f("spam")
spam
There are no objects that will behave differently if used in this way.
EVERY object can be a function default argument. Steve's proposal has
some objects (functions with the LB flag set) actually behave
differently - they *will not behave correctly* if used in this way.
This is a restriction placed on the rest of the language.
> > Great. So now we have some magnificently magical behaviour in the
> > language, which will have some nice sharp edge cases, but which nobody
> > will ever notice. Totally. I'm sure. Plus, we pay a performance price
> > in any function that makes use of argument references, not just for
> > the late-bound default, but in the rest of the code. We also need to
> > have these special functions that get stored as separate code objects.
> >
> > All to buy what, exactly? The ability to manually synthesize an
> > equivalent parameter value, as long as there's no assignment
> > expressions, no mutation, no other interactions, etc, etc, etc? That's
> > an awful lot of magic for not a lot of benefit.
>
> I would consider most of what you say here an accurate description of
> your own proposal. :-)
That's highly unfair. No, I won't let that pass. Please retract or
justify that statement. You are quoting the conclusion of a lengthy
post in which I show significant magic in Steve's proposal,
contrasting it with mine which has much clearer behaviour, and you
then say that my proposal has the same magic. Frankly, that is not a
reasonable assertion, and I take offense.
> Now we have magnificently magical behavior in the language, which will
> take expressions in the function signature and behind the scenes
> "inline" them into the function body. We also need to have these
> special function arguments that do NOT get stored as separate objects,
> unlike ordinary function arguments. All to buy what, exactly? The
> ability to write something in the function signature that we can already
> write in the body, and that quite naturally belongs in the body, because
> it is executed when the function is called, not when it is defined.
You assert that it "belongs in the body", but only because Python
currently doesn't allow it to be anywhere else. Other languages have
this exact information in the function signature. This is a much
larger distinction than what Steve shows, which is the exact same
feature but with these magic callables.
> > I *really* don't like the idea that some types of object will be
> > executed instead of being used, just because they have a flag set.
> > That strikes me as the sort of thing that should be incredibly scary,
> > but since I can't think of any specific reasons, I just have to call
> > it "extremely off-putting".
>
> I *really* don't like the idea that some types of argument will be
> inlined into the function body instead of being stored as first-class
> values like other `__defaults__`, just because there happens to be this
> one extra character next to the equals sign in the function signature.
> That strikes me as the sort of thing that should be incredibly scary.
You're still being highly offensive here. There's a HUGE difference
between these two assertions. Steve's proposal makes some objects
*behave differently when used in existing features*. It would be like
creating a new type of string which, when printed out, would eval
itself. That proposal wouldn't fly, and it's why f-strings are most
assuredly NOT first-class objects.
Why is it such a big deal for these function default expressions to
not be first-class objects? None of these are first-class either:
print(f"An f-string's {x+y} subexpressions")
print(x/y if y else "An if/else expression's sides")
assign(x.y[42], "An assignment target")
We don't have a problem with these being unable to be externally
referenced, manipulated, etc, as first-class objects. Why is it a
problem to be
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On 2021-12-04 03:50, Chris Angelico wrote: On Sat, Dec 4, 2021 at 8:48 PM Steven D'Aprano wrote: And third, when the interpreter fetches a default from func.__defaults__, if it is a LB function, it automatically calls that function with the parameters to the left of x (which in this case would be just b). Plausible. Okay. What this does mean, though, is that there are "magic objects" that cannot be used like other objects. Consider: Your proposal also has the same problem, since it involves "magic functions" that do not have usable values for their argument defaults, instead having some kind of Ellipsis two-step. It's all a matter of what you consider magic. Great. So now we have some magnificently magical behaviour in the language, which will have some nice sharp edge cases, but which nobody will ever notice. Totally. I'm sure. Plus, we pay a performance price in any function that makes use of argument references, not just for the late-bound default, but in the rest of the code. We also need to have these special functions that get stored as separate code objects. All to buy what, exactly? The ability to manually synthesize an equivalent parameter value, as long as there's no assignment expressions, no mutation, no other interactions, etc, etc, etc? That's an awful lot of magic for not a lot of benefit. I would consider most of what you say here an accurate description of your own proposal. :-) Now we have magnificently magical behavior in the language, which will take expressions in the function signature and behind the scenes "inline" them into the function body. We also need to have these special function arguments that do NOT get stored as separate objects, unlike ordinary function arguments. All to buy what, exactly? The ability to write something in the function signature that we can already write in the body, and that quite naturally belongs in the body, because it is executed when the function is called, not when it is defined. I *really* don't like the idea that some types of object will be executed instead of being used, just because they have a flag set. That strikes me as the sort of thing that should be incredibly scary, but since I can't think of any specific reasons, I just have to call it "extremely off-putting". I *really* don't like the idea that some types of argument will be inlined into the function body instead of being stored as first-class values like other `__defaults__`, just because there happens to be this one extra character next to the equals sign in the function signature. That strikes me as the sort of thing that should be incredibly scary. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CKSNACEI2U2VHAMBLCRQRCSKJ52WYH33/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
> On 1 Dec 2021, at 06:16, Chris Angelico wrote: > > I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/ > with some additional information about the reference implementation, > and some clarifications elsewhere. (I suspect that there was a reply that I should be replying to but, cannot find one appropriate) I have a lot of code that exploits the fact that passing an explicit None will cause the early bound default idiom to set the default for me. def inner(timestamp=None): if timestamp is None: timestamp = time.time() do_stuff... def outer(timestamp=None): inner(timestamp=timestamp) outer can in an idiomatic way have inner default timestamp and not have to know what that means. With late bound I cannot do this without more complex pattern of building an arg list. What if passing None still worked? I know the argument that there are more sentinels then None. def inner(timestamp=>time.time()) do_stuff... def outer(timestamp=None): inner(timestamp=timestamp) The code in inner that decides to when to allow the default could check for timestamp being missing or arg present and None. Would the lack of support for other sentinels out weight the simple way to get the default applied? Barry ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/L7YDL225ETTLWAB64NJY5CJJUABUI4NB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
> On 4 Dec 2021, at 09:44, Steven D'Aprano wrote: > > On Sat, Dec 04, 2021 at 03:14:46PM +1100, Chris Angelico wrote: > >> Lots and lots and lots of potential problems. Consider: >> >> def f(): >>a = 1 >>def f(b, x=>a+b): >>def g(): return x, a, b >> >> Both a and b are closure variables - one because it comes from an >> outer scope, one because it's used in an inner scope. So to evaluate >> a+b, you have to look up an existing closure cell, AND construct a new >> closure cell. >> >> The only way to do that is for the compiled code of a+b to exist >> entirely within the context of f's code object. > > I dispute that is the only way. Let's do a thought experiment. There are many possible implementation of the late bound idea that could create an object/default expression. But is it reasonable to bother with that added complexity/maintenance burden for a first implementation. And maybe no one will care enough to ever implement the ability to modify the code of a late bound variables expression as a separate object later. I think I understand the argument as being along the lines of for early bound defaults they can be inspected and modified. Therefore being able to do the same for late bound defaults must be implemented. I'm not convinced that that is reasonable to require is implemented. If python had always had late bound defaults, as it is with most languages in the survey posted earlier in this thread, would that have been implemented as an object/expression? Maybe, but I doubt it. Summary: I agree it's not impossible, I do not agree that it's needed. Barry ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2GSTDHBKU55ZWUPVVGWRL44EYV54KFLV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sat, Dec 4, 2021 at 8:48 PM Steven D'Aprano wrote:
>
> On Sat, Dec 04, 2021 at 03:14:46PM +1100, Chris Angelico wrote:
>
> > Lots and lots and lots of potential problems. Consider:
> >
> > def f():
> > a = 1
> > def f(b, x=>a+b):
> > def g(): return x, a, b
> >
> > Both a and b are closure variables - one because it comes from an
> > outer scope, one because it's used in an inner scope. So to evaluate
> > a+b, you have to look up an existing closure cell, AND construct a new
> > closure cell.
> >
> > The only way to do that is for the compiled code of a+b to exist
> > entirely within the context of f's code object.
>
> I dispute that is the only way. Let's do a thought experiment.
>
> First, we add a new flag to the co_flags field on code objects. Call it
> the "LB" flag, for late-binding.
>
> Second, we make this:
>
> def f(b, x=>a+b): ...
>
> syntactic sugar for this:
>
> def f(b, x=lambda b: a+b): ...
>
> except that the lambda has the LB flag set.
Okay. So the references to 'a' and 'b' here are one more level of
function inside the actual function we're defining, which means you're
paying the price of nonlocals just to be able to late-evaluate
defaults. Not a deal-breaker, but that is a notable cost (every
reference to them inside the function will be slower).
> And third, when the interpreter fetches a default from
> func.__defaults__, if it is a LB function, it automatically calls that
> function with the parameters to the left of x (which in this case
> would be just b).
Plausible. Okay.
What this does mean, though, is that there are "magic objects" that
cannot be used like other objects. Consider:
def make_printer(dflt):
def func(x=dflt):
print("x is", x)
return func
Will make_printer behave the same way for all objects? Clearly the
expectation is that it will display the repr of whichever object is
passed to func, or if none is, whichever object is passed to
make_printer. But if you pass it a function with the magic LB flag
set, it will *execute* that function. I don't like the idea that some
objects will be invisibly different like that.
> Here's your function, with a couple of returns to make it actually do
> something:
>
> def f():
> a = 1
> def f(b, x=>a+b):
> def g(): return x, a, b
> return g
> return f
>
>
> We can test that right now (well, almost all of it) with this:
>
> def func(): # change of name to distinguish inner and outer f
> a = 1
> def f(b, x=lambda b: a+b):
> def g(): return x, a, b
> return g
> return f
>
>
> and just pretend that x is automatically evaluated by the interpreter.
> But as a proof of concept, it's enough that we can demonstrate that *we*
> can manually evaluate it, by calling the lambda.
Okay, sure. It's a bit hard to demo it (since it has to ONLY do that
magic if the arg was omitted), but sure, we can pretend.
> We can call func() to get the inner function f, and call f to get g:
>
> >>> f = func()
> >>> print(f)
> .f at 0x7fc945c41f30>
>
> >>> g = f(100)
> >>> print(g)
> .f..g at 0x7fc945e1f520>
>
> Calling g works:
>
> >>> print(g())
> (. at 0x7fc945c40f70>, 1, 100)
>
> with the understanding that the real implementation will have
> automatically called that lambda, so we would have got 101 instead of
> the lambda. That step requires interpreter support, so for now we just
> have to pretend that we get
>
> (101, 1, 100)
>
> instead of the lambda. But we can demonstrate that calling the lambda
> works, by manually calling it:
>
> >>> x = g()[0]
> >>> print(x)
> . at 0x7fc945c40f70>
> >>> print(x(100)) # the interpreter knows that b=100
> 101
>
>
> Now let's see if we can extract the default and play around with it:
>
> >>> default_expression = f.__defaults__[0]
> >>> print(default_expression)
> . at 0x7fc945c40f70>
>
> The default expression is just a function (with the new LB flag set). So
> we can inspect its name, its arguments, its cell variables, etc:
>
> >>> default_expression.__closure__
> (,)
>
> We can do anything that we could do with any other other function object.
Yup. As long as it doesn't include any assignment expressions, or
anything else that would behave differently.
> Can we evaluate it? Of course we can. And we can test it with any value
> we like, we're not limited to the value of b that we originally passed
> to func().
>
> >>> default_expression(3000)
> 3001
>
> Of course, if we are in a state of *maximal ignorance* we might have no
> clue what information is needed to evaluate that default expression:
>
> >>> default_expression()
> Traceback (most recent call last):
> File "", line 1, in
> TypeError: func..() missing 1 required positional
> argument: 'b'
>
> Oh look, we get a useful diagnostic message for free!
>
> What are we missing? The source code of the original
[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!
On Sat, Dec 04, 2021 at 03:14:46PM +1100, Chris Angelico wrote: > Lots and lots and lots of potential problems. Consider: > > def f(): > a = 1 > def f(b, x=>a+b): > def g(): return x, a, b > > Both a and b are closure variables - one because it comes from an > outer scope, one because it's used in an inner scope. So to evaluate > a+b, you have to look up an existing closure cell, AND construct a new > closure cell. > > The only way to do that is for the compiled code of a+b to exist > entirely within the context of f's code object. I dispute that is the only way. Let's do a thought experiment. First, we add a new flag to the co_flags field on code objects. Call it the "LB" flag, for late-binding. Second, we make this: def f(b, x=>a+b): ... syntactic sugar for this: def f(b, x=lambda b: a+b): ... except that the lambda has the LB flag set. And third, when the interpreter fetches a default from func.__defaults__, if it is a LB function, it automatically calls that function with the parameters to the left of x (which in this case would be just b). Here's your function, with a couple of returns to make it actually do something: def f(): a = 1 def f(b, x=>a+b): def g(): return x, a, b return g return f We can test that right now (well, almost all of it) with this: def func(): # change of name to distinguish inner and outer f a = 1 def f(b, x=lambda b: a+b): def g(): return x, a, b return g return f and just pretend that x is automatically evaluated by the interpreter. But as a proof of concept, it's enough that we can demonstrate that *we* can manually evaluate it, by calling the lambda. We can call func() to get the inner function f, and call f to get g: >>> f = func() >>> print(f) .f at 0x7fc945c41f30> >>> g = f(100) >>> print(g) .f..g at 0x7fc945e1f520> Calling g works: >>> print(g()) (. at 0x7fc945c40f70>, 1, 100) with the understanding that the real implementation will have automatically called that lambda, so we would have got 101 instead of the lambda. That step requires interpreter support, so for now we just have to pretend that we get (101, 1, 100) instead of the lambda. But we can demonstrate that calling the lambda works, by manually calling it: >>> x = g()[0] >>> print(x) . at 0x7fc945c40f70> >>> print(x(100)) # the interpreter knows that b=100 101 Now let's see if we can extract the default and play around with it: >>> default_expression = f.__defaults__[0] >>> print(default_expression) . at 0x7fc945c40f70> The default expression is just a function (with the new LB flag set). So we can inspect its name, its arguments, its cell variables, etc: >>> default_expression.__closure__ (,) We can do anything that we could do with any other other function object. Can we evaluate it? Of course we can. And we can test it with any value we like, we're not limited to the value of b that we originally passed to func(). >>> default_expression(3000) 3001 Of course, if we are in a state of *maximal ignorance* we might have no clue what information is needed to evaluate that default expression: >>> default_expression() Traceback (most recent call last): File "", line 1, in TypeError: func..() missing 1 required positional argument: 'b' Oh look, we get a useful diagnostic message for free! What are we missing? The source code of the original expression, as text. That's pretty easy too: the compiler knows the source, it can cram it into the default expression object: >>> default_expression.__expression__ = 'a+b' Introspection tools like help() can learn to look for that. What else are we missing? A cool repr. >>> print(default_expression) # Simulated. We can probably come up with a better repr, and a better name than "late bound default expression". We already have other co_flags that change the repr: 32 GENERATOR 128 COROUTINE 256 ITERABLE_COROUTINE so we need a name that is at least as cool as "generator" or "coroutine". Summary of changes: * add a new co_flag with a cool name better than "LB"; * add an `__expression__` dunder to hold the default expression; (possibly missing for regular functions -- we don't necessarily need *every* function to have this dunder) * change the repr of LB functions to display the expression; * teach the interpreter to compile late-bound defaults into one of these LB functions, including the source expression; * teach the interpreter that when retrieving default values from the function's `__defaults__`, if they are a LB function, it must call the function and use its return result as the actual default value; * update help() and other introspection tools to handle these LB functions; but if any tools don't get updated, you still get a
