date:20211030

On Sun, Oct 31, 2021 at 02:56:36PM +1100, Chris Angelico wrote:

> Current versions of the PEP do not use the term "default value" when
> referring to late binding (or at least, if I've made a mistake there,
> then please point it out so I can fix it). I'm using the term "default
> expression", or just "default" (to cover both values and expressions).

I was just thinking of suggesting that to you, so I'm glad to see you're 
much faster on the uptake than I am!

Of course all parameters are syntactically an expression, including now:

# the status quo
def func(arg=CONFIG.get('key', NULL)):

The default expression is evaluated at function definition time, and the 
result of that (an object, a.k.a. a value) is cached in the function 
object for later use. With late-binding:

def func(@arg=CONFIG.get('key', NULL)):

the expression is stashed away somewhere (implementation details), in 
some form (source code? byte-code? an AST?) rather than immediately 
evaluated. At function call time, the expression is evaluated, and the 
result (an object, a.k.a. a value) is bound to the parameter.

In neither case is it correct to say that the default value of arg is 
the *expression* `CONFIG.get('key', NULL)`, it is in both the early and 
late bound cases the *result* of *using* (evaluating) the expression to 
generate a value.

https://en.wikipedia.org/wiki/Use%E2%80%93mention_distinction

I'm fairly confident that everyone understands that:

"the default value is CONFIG.get('key', NULL)"

is shorthand for the tediously long and pedantic explanation that it's 
not the expression itself that is the default value, but the result of 
evaluating the expression. Just like we understand it here:

if arg is None:
arg = CONFIG.get('key', NULL)

The only difference is when the expression is evaluated.

If we can understand that arg gets set to the result of the expression 
in the second case (the None sentinel), we should be able to understand 
it in if the syntax changes to late-bound parameters.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NPTK2TZLBL6KZVKNYKLUXVK3A3NULDLN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 03:43:25PM +1100, Chris Angelico wrote:

> There is a downside: it is possible to flat-out lie to the
> interpreter, by mutating bisect_left.__defaults__, so that help() will
> give a completely false signature.

>>> def func(arg="finest green eggs and ham"):
... pass
... 
>>> inspect.signature(func)

>>>
>>> func.__defaults__ = ("yucky crap",)
>>> inspect.signature(func)


If help, or some other tool is caching the function signature, perhaps 
it shouldn't :-)


> But if you want to shoot yourself
> in the foot, there are already plenty of gorgeous guns available.

Indeed. Beyond avoiding segmentation faults, I don't think we need to 
care about people who mess about with the public attributes of 
functions. You can touch, you can even change them, but keeping the 
function working is no longer our responsibility at that point.

If you change the defaults, you shouldn't get a seg fault when you call 
the function, but you might get an exception.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T4YYAA3Q3TJRUNPAQ23TEM5XDZJ3CSML/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sat, Oct 30, 2021 at 06:54:51PM -0700, Brendan Barnwell wrote:

>   I mean, here's another way to come at this.  Suppose we have this 
>   under the proposal (using some random syntax picked at random):
> 
> def foo(a=1, b="two", c@=len(b), d@=a+c):
> 
>   You keep saying that c and d "are argument default" just like a and 
>   b. So can you tell me what the argument default for each of those 
>  arguments?

Sure.

If you fail to provide an argument for c, the value that is bound to c 
by default (i.e. the default argument) is len(b), whatever that happens 
to be. If you fail to provide a value for d, the value that is bound to 
d by default is a+c.

There is nothing in the concept of "default argument" that requires it 
to be known at function-definition time, or compile time, or when the 
function is typed into the editor.

Do you have a problem understanding me if I say that strftime defaults 
to the current time? Surely you don't imagine that I mean that it 
defaults to 15:34:03, which was the time a few seconds ago when I wrote 
the words "current time".

And you probably will understand me if I say that on POSIX systems such 
as Linux, the default permissions on newly created files is (indirectly) 
set by the umask.

>   Currently, every argument default is a first-class value.

And will remain so. This proposal does not add second-class values to 
the language.

By the time the body of the function is entered, the late-bound 
parameters will have had their defaults evaluated, and the result bound 
to the parameter.

Inside the function object itself, there will be some kind of opaque 
blob (possibly a function?) that holds the default's expression for 
later evaluation. That blob itself will be a first class object, like 
every other object in Python, even if its internal structure is 
partially or fully opaque.

> As I 
> understand it, your proposal breaks that assumption, and now argument 
> defaults can be some kind of "construct" that is not a first class 
> value, not any kind of object, just some sort of internal entity that no 
> one can see or manipulate in any way, it just gets automatically 
> evaluated later.

Kind of like functions themselves :-)

>>> (lambda x, y: 2*x + 3**y).__code__.co_code
b'd\x01|\x00\x14\x00d\x02|\x01\x13\x00\x17\x00S\x00'

The internal structure of that co_code object is a mystery, it is not 
part of the Python language, only of the implementation, but it remains 
a first-class value.

(My *guess* is that this may be the raw byte-code of the function body.)

One difference will be, regardless of how the expression for the 
late-bound default is stored, there will be at the very least an API to 
extract a human-readable string representing the expression.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ADAOKEN7CUQQUYPY32W5Q4IQMLVYSH3V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 3:31 PM Steven D'Aprano  wrote:
> >>> inspect.signature(bisect.bisect_left)
> 
>
> I challenge anyone to honestly say that if that signature read:
>
> 
>
> they would not be able to infer the meaning, or that Python would be a
> worse language if the interpreter managed the evaluation of that default
> so you didn't have to.

There is a downside: it is possible to flat-out lie to the
interpreter, by mutating bisect_left.__defaults__, so that help() will
give a completely false signature. But if you want to shoot yourself
in the foot, there are already plenty of gorgeous guns available.
Behold, the G3SG1 "High Seas" of footguns:

>>> def spam(x, y, z="foo", *, count=4): ...
...
>>> def ham(a, *, n): ...
...
>>> spam.__wrapped__ = ham
>>> inspect.signature(spam)


Ah whoops. We just managed to lie to ourselves. Good job, us.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/44AYKJUNNODIYB576TRIW4NF7IHQPSZA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 3:16 PM Steven D'Aprano  wrote:
>
> On Sun, Oct 31, 2021 at 12:29:18PM +1100, Chris Angelico wrote:
>
> > That's optional parameters. It's entirely possible to have optional
> > parameters with no concept of defaults; it means the function can be
> > called with or without those arguments, and the corresponding
> > parameters will either be set to some standard "undefined" value, or
> > in some other way marked as not given.
>
> That's certainly possible with languages like bash where there are
> typically (never?) no explicit parameters, the function is expected to
> pop arguments off the argument list and deal with them as it sees fit.
> Then a missing argument is literally missing from the argument list, and
> all the parameter binding logic that the Python interpreter handles for
> you has to be handled manually by the programmer. Bleh.
>
> We could emulate that in Python by having the interpreter flag "optional
> without a default" parameters in such a way that the parameter remains
> unbound when called without an argument, but why would we want such a
> thing? That's truly a YAGNI anti-feature.
>
> If you really want to emulate bash, we can just declare the function to
> take `*args` and manage it ourselves, like bash.
>
>
> > Python, so far, has completely conflated those two concepts: if a
> > parameter is optional, it always has a default argument value. (The
> > converse is implied - mandatory args can't have defaults.)
>
> If a mandatory parameter has a default argument, it would never be used,
> because the function is always called with an argument for that
> parameter. So we have a four-way table:
>
>
>MandatoryOptional
>Parameters   Parameters
> -  ---  
> No default:okay YAGNI [1]
> Default:   pointlessokay
> -  ---  
>
>
> [1] And if you do need it, it is easy to emulate with a sentinel:
>
> def func(arg=None):
> if arg is None:
> del arg
> process(arg)  # May raise UnboundLocalError
>
>
>
> None of this is relevant to the question of when the default values
> should be evaluated.
>

Agreed on all points, but I think the YAGNIness of it is less strong
than you imply. Rather than a sentinel object, this would be a
sentinel lack-of-object. The point of avoiding sentinels is, like
everywhere else, that the sentinel isn't really a value - it's just a
marker saying "this arg wasn't passed". So I agree with you that this
isn't a feature of huge value, and it's not one that I'm pushing for,
but it is at least internally consistent and well-defined.

Usually, what ends up happening is that there's code somewhere saying
"if arg is _sentinel:" (or "if arg is None:"), which would have
exactly the same meaning without the sentinel, if we had a way to say
"if arg isn't set".

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LZCIXL5KTK3LBQP2LHQHL5W3G76ZQNPV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sat, Oct 30, 2021 at 06:52:33PM -0700, Christopher Barker wrote:

> The point is: clearly specifying what's required, what's optional, and what
> the defaults are if optional, is really, really useful -- and this PEP will
> add another very handy feature to that.

+1

Earlier I said that better help() is "icing on the cake". I like icing 
on my cake, and I think that having late-bound defaults clearly readable 
without any extra effort is a good thing. The status quo is that if you 
introspect a function's parameters, you will see the sentinel, not the 
actual default value, or the expression that gives the default value, 
that the body of the function actually uses.

>>> inspect.signature(bisect.bisect_left)

I challenge anyone to honestly say that if that signature read:

they would not be able to infer the meaning, or that Python would be a 
worse language if the interpreter managed the evaluation of that default 
so you didn't have to.

And if you really want to manage the late evaluation of defaults 
yourself, you will still be able to.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZXM5PVQH3ZPPR6RQXGSULS5ODW5MWVOM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 3:08 PM Brendan Barnwell  wrote:
>
> On 2021-10-30 20:44, Chris Angelico wrote:
> >> The default of every argument should be a first-class value.  
> >> That's
> >> how things are now, and I think that's a very useful invariant to have.
> >>   If we want to break it we need a lot more justification than "I don't
> >> like typing if x is None".
> >
> > How important is this? Yes, it's the status quo, but if we evaluate
> > every proposal by how closely it retains the status quo, nothing would
> > ever be changed. Let's look at type checking for a moment, and compare
> > a few similar functions:
> 
>
> As I've said before, the problem is that the benefit of this feature 
> is
> too small to justify this large change to the status quo.
>
> > You use similarly mysterious "expressions" all the time. They can't be
> > treated as first-class values, but they can be used in the contexts
> > they are in. This is exactly the same. You've probably even used
> > expressions that are only evaluated conditionally:
> >
> > recip = 1 / x if x else 0
> >
> > Is the "1 / x" a first-class value here? No - and that's why if/else
> > is a compiler construct rather than a function.
>
> No, but that example illustrates why: the expression there is not a
> first-class value, but that's fine because it also has no independent
> status.  The expression as a whole is evaluated and the expression as a
> whole has a value and that is what you work with.  You don't get to
> somehow stash away just the 1/x part and use it later, but without
> evaluating it yet.  That is not even close to the situation envisioned
> in the proposal under discussion, because in the proposal this
> mysterious expression (the argument default), although not a first-class
> value, is going to be stored and used independently, yet without
> evaluating it and without "reifying" it into a function or other such object

It's actually the same. There's no "stashing away" of the expression
part; it's just part of the function, same as everything else is. (I
intend to have a string representation of it stashed away, but that IS
a first-class value - a str object, or a PyUnicode if you look in the
C API.) You can't use that later in any way other than by calling the
function while not passing it the corresponding argument. You cannot
use it independently, and you cannot retrieve an unevaluated version
of it.

> It's obvious that there are tons of things that aren't first-class
> values in Python (and in any language).  The plus sign isn't a first
> class value, the sequence of characters `a = "this".cou` in source code
> isn't a first class value.  But these usages aren't the same as what's
> in the proposal under discussion here.  I'm not sure if you actually
> disagree with this or if you're just being disingenuous with your examples.

Right. Some things are simply invalid, others are compiler constructs.
I'm pointing out that compiler constructs are very real things,
despite not being values.

I'm not sure about your ".cou" example; the only reason that isn't a
first-class value is that, when you try to look it up, you'll get an
error. Or are you saying that an assignment statement isn't a value?
If so, then it's the same thing again: a compiler construct, a
syntactic feature.

In general, syntactic features fall into three broad categories:

1) Literals, which have clear values
2) Expressions, which will yield values when evaluated
3) Everything else. Mostly statements. No value ever.

For instance, the notation >> 42j << is a complex literal (an
imaginary number), and >> a+b << is an expression representing a sum.
Thanks to constant folding, we can pretend that >> 3+4j << is a
literal too, although technically it's an expression.

Argument defaults are *always* expressions. They can be evaluated at
function definition time, or - if PEP 671 is accepted - at function
call time. The expression itself is nothing more than a compiler
construct, and isn't a first-class value. If it's early-bound, then
it'll be evaluated at def time to yield a value which then gets saved
on the function object and then gets assigned any time the parameter
has no value; if it's late-bound, then any time the parameter has no
value, it'll be evaluated at call time, to yield a value which then
gets assigned to the parameter. Either way, the same steps happen,
just in a different order.

> Anyway, I appreciate your engaging with me in this discussion,
> especially since I'm just some random guy whose opinion is not of major
> importance.  :-)  But I think we're both kind of just repeating
> ourselves at this point.  I acknowledge that your proposal is
> essentially clear and is aimed at serving a genuine use case, but I
> still see it as involving too much of a departure from existing
> conventions, too much hairiness in the details, and too little real
> benefit, to justify the change.

I think everyone's opinion is of major

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 12:29:18PM +1100, Chris Angelico wrote:

> That's optional parameters. It's entirely possible to have optional
> parameters with no concept of defaults; it means the function can be
> called with or without those arguments, and the corresponding
> parameters will either be set to some standard "undefined" value, or
> in some other way marked as not given.

That's certainly possible with languages like bash where there are 
typically (never?) no explicit parameters, the function is expected to 
pop arguments off the argument list and deal with them as it sees fit. 
Then a missing argument is literally missing from the argument list, and 
all the parameter binding logic that the Python interpreter handles for 
you has to be handled manually by the programmer. Bleh.

We could emulate that in Python by having the interpreter flag "optional 
without a default" parameters in such a way that the parameter remains 
unbound when called without an argument, but why would we want such a 
thing? That's truly a YAGNI anti-feature.

If you really want to emulate bash, we can just declare the function to 
take `*args` and manage it ourselves, like bash.

> Python, so far, has completely conflated those two concepts: if a
> parameter is optional, it always has a default argument value. (The
> converse is implied - mandatory args can't have defaults.)

If a mandatory parameter has a default argument, it would never be used, 
because the function is always called with an argument for that 
parameter. So we have a four-way table:

   MandatoryOptional
   Parameters   Parameters
-  ---    
No default:okay YAGNI [1]
Default:   pointlessokay
-  ---  

[1] And if you do need it, it is easy to emulate with a sentinel:

def func(arg=None):
if arg is None:
del arg
process(arg)  # May raise UnboundLocalError

None of this is relevant to the question of when the default values 
should be evaluated.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CSDVIYBORV4FVFSCIE7KY2KI5SRG6GXW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 2:52 PM David Mertz, Ph.D.
 wrote:
>
>
> On Sat, Oct 30, 2021, 11:03 PM Chris Angelico
>>
>> That's if you were to create a deferred *type*. An object which can be 
>> evaluated later. My proposal is NOT doing that, because there is no object 
>> that represents the unevaluated expression. You can't pull that expression 
>> out and evaluate it somewhere else. It wouldn't be meaningful.
>
>
> Put this way, I admit it is intelligible. Maybe it even reduces the order of 
> magnitude of my opposition to it.
>
> What you are trying to create remains a "unit of computation" even though it 
> is not itself a Python object. As Brendan points out, this is a fundamental 
> change to Python's object model.
>
> It amounts to saying "I want to perform some computation within the body of a 
> function, but I don't want to WRITE it in the body of a function." That feels 
> like an anti-goal to me.
>

Not quite; I want to perform it within the *context* of the function,
but not in the body. A function's context (its scope, available
variables, etc, etc, etc) is where most of its body is executed, but
for example, a list comprehension is part of the function's body while
not being in the same context (it's in a subcontext defined by a
nested function).

> I suppose I don't *hate* something like this, which can be done with no 
> syntax change:
>
> @rewrite
> def foo(a, size=Late("len(a)")):
> print("The list is length", size)
>
> In the functions I actually use and write, the problem all of this is trying 
> to solve is either trivial or irrelevant.
>

Thing is, that can't actually be done, because there's no way to make
that able to see the function's parameters without some VERY weird
shenanigans. You'd have to completely recreate the
argument-to-parameter allocation, apply all preceding defaults, and
then have a set of locals which can be passed to eval(); and then
having done all that, deconstruct your locals into *a,**kw to pass
back to the original function. Meanwhile, you have to capture any
nonlocals, in case len isn't actually a global.

In contrast, having this as a compiler construct is simple: In the
context of the function, if the parameter hasn't been given a value,
evaluate the expression and assign.

(Side point: The current reference implementation allows assignment
expressions inside default argument expressions, mainly because I
didn't go to the effort of blocking them. But if you ever ACTUALLY do
this, then. *wat*)

> In libraries like Pandas, for example, there are often tens of arguments with 
> defaults of None... But it is rarely a single default to resolve. Rather, 
> there is complex logic about the interaction of which arguments are provided 
> or absent, leading to numerous code paths to configure the actual behavior of 
> a call.
>
> For these, the logic of missing arguments must live in the body, where it 
> naturally belongs.
>

There will always be cases too complicated for simple default
expressions, just as there are already cases too complicated for
simple default values. They will continue to be handled by the body of
the function, as they currently are. This proposal isn't a replacement
for all argument processing.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4RRWHFOJRF26S54MDSXO3WH4I3OVYT7I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults


On 2021-10-30 20:44, Chris Angelico wrote:

The default of every argument should be a first-class value.  That's
how things are now, and I think that's a very useful invariant to have.
  If we want to break it we need a lot more justification than "I don't
like typing if x is None".


How important is this? Yes, it's the status quo, but if we evaluate
every proposal by how closely it retains the status quo, nothing would
ever be changed. Let's look at type checking for a moment, and compare
a few similar functions:



	As I've said before, the problem is that the benefit of this feature is 
too small to justify this large change to the status quo.



You use similarly mysterious "expressions" all the time. They can't be
treated as first-class values, but they can be used in the contexts
they are in. This is exactly the same. You've probably even used
expressions that are only evaluated conditionally:

recip = 1 / x if x else 0

Is the "1 / x" a first-class value here? No - and that's why if/else
is a compiler construct rather than a function.


	No, but that example illustrates why: the expression there is not a 
first-class value, but that's fine because it also has no independent 
status.  The expression as a whole is evaluated and the expression as a 
whole has a value and that is what you work with.  You don't get to 
somehow stash away just the 1/x part and use it later, but without 
evaluating it yet.  That is not even close to the situation envisioned 
in the proposal under discussion, because in the proposal this 
mysterious expression (the argument default), although not a first-class 
value, is going to be stored and used independently, yet without 
evaluating it and without "reifying" it into a function or other such object


	It's obvious that there are tons of things that aren't first-class 
values in Python (and in any language).  The plus sign isn't a first 
class value, the sequence of characters `a = "this".cou` in source code 
isn't a first class value.  But these usages aren't the same as what's 
in the proposal under discussion here.  I'm not sure if you actually 
disagree with this or if you're just being disingenuous with your examples.


	Anyway, I appreciate your engaging with me in this discussion, 
especially since I'm just some random guy whose opinion is not of major 
importance.  :-)  But I think we're both kind of just repeating 
ourselves at this point.  I acknowledge that your proposal is 
essentially clear and is aimed at serving a genuine use case, but I 
still see it as involving too much of a departure from existing 
conventions, too much hairiness in the details, and too little real 
benefit, to justify the change.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HTEYZGXHMPGAPVPAL6VOOF3EF32PIVCG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 2:43 PM <2qdxy4rzwzuui...@potatochowder.com> wrote:
>
> On 2021-10-30 at 18:54:51 -0700,
> Brendan Barnwell  wrote:
>
> > On 2021-10-30 18:29, Chris Angelico wrote:
>
> > > Right. That is a very real difference, which is why there is a very
> > > real difference between early-bound and late-bound defaults. But both
> > > are argument defaults.
> >
> >   I don't 100% agree with that.
>
> This seems to be the crux of this whole sub-discussion.  This whole
> thing scratches an itch I don't have, likely because of the way I
> learned to design interfaces on all levels.  A week or so ago, I was
> firmly in Brendan Barnwell's camp.  I really don't like how the phrase
> "default value" applies to PEP-671's late binding, and I'm sure that
> there will remain cases in which actual code inside the function will be
> required.  But I'm beginning to see the logic behind the arguments (pun
> intended) for the PEP.

Current versions of the PEP do not use the term "default value" when
referring to late binding (or at least, if I've made a mistake there,
then please point it out so I can fix it). I'm using the term "default
expression", or just "default" (to cover both values and expressions).

And yes; there will always be cases where you can't define the default
with a simple expression. For instance, a one-arg lookup might raise
an exception where a two-arg one could return a default value. That's
currently best written with a dedicated object:

_sentinel = object()
def fetch(thing, default=_sentinel):
... attempt to get stuff
if thing in stuff:
return stuff[thing]
if default is _sentinel:
raise ThingNotFoundError
return default

In theory, optional arguments without defaults could be written
something like this:

def fetch(thing, default=pass):
... as above
if not exists default:
raise ThingNotFoundError
return default

But otherwise, there has to be some sort of value for every parameter.

(I say this as a theory, but actually, the reference implementation of
PEP 671 has code very similar to this. There's a bytecode QUERY_FAST
which yields True if a local has a value, False if not. It's more
efficient than "try: default; True; except UnboundLocalError: False"
but will have the same effect.)

> The human description language/wording is different; , but what Python
> spells "default value," Common Lisp spells "initform."  Python is
> currently much less flexible about when and in what context default
> values are evaluated; PEP-671 attempts to close that gap, but is
> hampered by certain technical and emotional baggage.

Lisp's execution model is quite different from Python's, but I'd be
curious to hear more about this. Can you elaborate?

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/O2KQDNY2C3L6GW5AT2OJCOZZ3CMK7JG3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sat, Oct 30, 2021, 11:03 PM Chris Angelico

> That's if you were to create a deferred *type*. An object which can be
> evaluated later. My proposal is NOT doing that, because there is no object
> that represents the unevaluated expression. You can't pull that expression
> out and evaluate it somewhere else. It wouldn't be meaningful.
>

Put this way, I admit it is intelligible. Maybe it even reduces the order
of magnitude of my opposition to it.

What you are trying to create remains a "unit of computation" even though
it is not itself a Python object. As Brendan points out, this is a
fundamental change to Python's object model.

It amounts to saying "I want to perform some computation within the body of
a function, but I don't want to WRITE it in the body of a function." That
feels like an anti-goal to me.

But it's true that the code fragment 'if a is None: a = calculated()' is
also not itself a first class object. So a synonym for that I can
understand, albeit dislike.

I suppose I don't *hate* something like this, which can be done with no
syntax change:

@rewrite
def foo(a, size=Late("len(a)")):
print("The list is length", size)

In the functions I actually use and write, the problem all of this is
trying to solve is either trivial or irrelevant.

In libraries like Pandas, for example, there are often tens of arguments
with defaults of None... But it is rarely a single default to resolve.
Rather, there is complex logic about the interaction of which arguments are
provided or absent, leading to numerous code paths to configure the actual
behavior of a call.

For these, the logic of missing arguments must live in the body, where it
naturally belongs.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EWWXV4LONSWDHAGGCNLWDH2KPUOWERJI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 2:23 PM Brendan Barnwell  wrote:
>
> On 2021-10-30 19:11, Chris Angelico wrote:
> >> > I mean, here's another way to come at this.  Suppose we have 
> >> > this under
> >> >the proposal (using some random syntax picked at random):
> >> >
> >> >def foo(a=1, b="two", c@=len(b), d@=a+c):
> >> >
> >> > You keep saying that c and d "are argument default" just like a 
> >> > and b.
> >> >   So can you tell me what the argument default for each of those 
> >> > arguments?
> > The default for a is the integer 1. The default for b is the string
> > "two". The default for c is the length of b. The default for d is the
> > sum of a and c.
> >
> >> > The default for argument a is an integer.  The default for 
> >> > argument b
> >> >is a string.  Can you tell me, in comparable terms, what the defaults
> >> >for arguments c and d are?
> > You're assuming that every default is a*default value*. That is the
> > current situation, but it is by no means the only logical way a
> > default can be defined. See above: c's default is the length of b,
> > which is presumably an integer, and d's default is the sum of that and
> > a, which is probably also an integer (unless a is passed as a float or
> > something).
>
> Well, at least that clarifies matters.  :-)
>
> I was already -1 on this but this moves me to a firm -100.
>
> "The length of b" is a description in English, not any kind of
> programming construct.  "The length of b" has no meaning in Python.
> What we store for the default (even if we don't want to call it a value)
> has to be a Python construct, not a human-language description.  I could
> say "the default of b is the notion of human frailty poured into golden
> goblet", and that would be just as valid as "the length of b" as a
> description and just as meaningless in terms of
> Python's data model.

We have a very good construct to mean "the length of b". It looks like this:

len(b)

In CPython bytecode, it looks something like this:

LOAD_GLOBAL "len"
LOAD_FAST "b"
CALL_FUNCTION with 1 argument

(Very approximately, anyway.)

And that's exactly what PEP 671 uses: in the source code, it uses
len(b), and in the compiled executable, the corresponding bytecode
sequence. Since we don't have a way to define human frailty and golden
goblets, we don't have a good way to encode that in either source code
or bytecode.

> The default of every argument should be a first-class value.  That's
> how things are now, and I think that's a very useful invariant to have.
>   If we want to break it we need a lot more justification than "I don't
> like typing if x is None".

How important is this? Yes, it's the status quo, but if we evaluate
every proposal by how closely it retains the status quo, nothing would
ever be changed. Let's look at type checking for a moment, and compare
a few similar functions:

def spam(stuff: list, n: int):
...

Takes a list and an integer. Easy. The integer is required.

def spam(stuff: list, n: int = None):
if n is None: n = len(stuff)
...

Takes a list, and either an integer or None. What does None mean? Am I
allowed to pass None, or is it just a construct that makes the
parameter optional?

_sentinel = object()
def spam(stuff: list, n: int = _sentinel):
if n is _sentinel: n = len(stuff)
...

Takes a list, and either an integer or some random object. Can
anyone with type checking experience (eg MyPy etc) say how this would
best be annotated? Putting it like this doesn't work, nor does
Optional[int].

def spam(stuff: list, n: int => len(stuff)):
...

MyPy (obviously) doesn't yet support this syntax, so I can't test
this, but I would assume that it would recognize that len() returns an
integer, and accept this. (That's what it does if I use "= len(stuff)"
and a global named stuff.)

Currently, argument defaults have to be able to be evaluated at
function definition time. It's fine if they can't be precomputed at
compile time. Why do we have this exact restriction, neither more nor
less? Is that really as important an invariant as you say?

> Apart from all the other things about this proposal I don't support, I
> don't support the creation of a mysterious "expression" which is not a
> first class value and cannot be used or evaluated in any way except
> automatically in the context of calling a function.

You use similarly mysterious "expressions" all the time. They can't be
treated as first-class values, but they can be used in the contexts
they are in. This is exactly the same. You've probably even used
expressions that are only evaluated conditionally:

recip = 1 / x if x else 0

Is the "1 / x" a first-class value here? No - and that's why if/else
is a compiler construct rather than a function.

If you wouldn't use this feature, that's fine, but it shouldn't stand
or fall on something that the rest of the language doesn't follow.

ChrisA

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

2021-10-30 Thread 2QdxY4RzWzUUiLuE

On 2021-10-30 at 18:54:51 -0700,
Brendan Barnwell  wrote:

> On 2021-10-30 18:29, Chris Angelico wrote:

> > Right. That is a very real difference, which is why there is a very
> > real difference between early-bound and late-bound defaults. But both
> > are argument defaults.
> 
>   I don't 100% agree with that.

This seems to be the crux of this whole sub-discussion.  This whole
thing scratches an itch I don't have, likely because of the way I
learned to design interfaces on all levels.  A week or so ago, I was
firmly in Brendan Barnwell's camp.  I really don't like how the phrase
"default value" applies to PEP-671's late binding, and I'm sure that
there will remain cases in which actual code inside the function will be
required.  But I'm beginning to see the logic behind the arguments (pun
intended) for the PEP.

As prior art, consider Common Lisp's lambda expressions, which are
effectively anonymous functions (such expressions are often bound to
names, which is how Common Lisp creates named functions); see
https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node64.html for
reference.

The human description language/wording is different; , but what Python
spells "default value," Common Lisp spells "initform."  Python is
currently much less flexible about when and in what context default
values are evaluated; PEP-671 attempts to close that gap, but is
hampered by certain technical and emotional baggage.

(OTOH, Common Lisp's lambda expressions take one more step and include
so-called "aux variables," which aren't parameters at all, but variables
local to the function itself.  I don't have enough background or context
to know why these are included in a lambda expression.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MBGORGQFXHSZWJYERNYDMYJXIA4W4CPY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 2:20 PM Rob Cliffe via Python-ideas
 wrote:
> As for deferred evaluation objects:
> First, Python already has various ways of doing deferred evaluation:
> Lambdas
> Strings, which can be passed to eval / exec / compile.
> You can write decorator(s) for functions to make them defer their 
> evaluation.
> You can write a class of deferred-evaluation objects.
> None of these ways is perfect.  Each have their pros and cons.  The bottom 
> line, if I understand correctly (maybe I don't) is that there has to be a way 
> of specifying (implicitly or explicitly) when the (deferred) evaluation 
> occurs, and also what the evaluation context is (e.g. for eval, locals and 
> globals must be specified, either explicitly or implicitly).
>

That last part is the most important here: it has to be evaluated *in
the context of the function*. That's the only way for things like "def
f(a, n=len(a)):" to be possible. Every piece of code in Python is
executed, if it is ever executed, in the context that it was written
in. Before f-strings were implemented, this was debated in some
detail, and there is no easy way to transfer a context around
usefully.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VDPRQQLYEGVTDJPJAOJAQPGUNKKYBYZ7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults


On 2021-10-30 19:11, Chris Angelico wrote:

> I mean, here's another way to come at this.  Suppose we have this 
under
>the proposal (using some random syntax picked at random):
>
>def foo(a=1, b="two", c@=len(b), d@=a+c):
>
> You keep saying that c and d "are argument default" just like a and b.
>   So can you tell me what the argument default for each of those arguments?

The default for a is the integer 1. The default for b is the string
"two". The default for c is the length of b. The default for d is the
sum of a and c.


> The default for argument a is an integer.  The default for argument b
>is a string.  Can you tell me, in comparable terms, what the defaults
>for arguments c and d are?

You're assuming that every default is a*default value*. That is the
current situation, but it is by no means the only logical way a
default can be defined. See above: c's default is the length of b,
which is presumably an integer, and d's default is the sum of that and
a, which is probably also an integer (unless a is passed as a float or
something).


Well, at least that clarifies matters.  :-)

I was already -1 on this but this moves me to a firm -100.

	"The length of b" is a description in English, not any kind of 
programming construct.  "The length of b" has no meaning in Python. 
What we store for the default (even if we don't want to call it a value) 
has to be a Python construct, not a human-language description.  I could 
say "the default of b is the notion of human frailty poured into golden 
goblet", and that would be just as valid as "the length of b" as a 
description and just as meaningless in terms of

Python's data model.

	The default of every argument should be a first-class value.  That's 
how things are now, and I think that's a very useful invariant to have. 
 If we want to break it we need a lot more justification than "I don't 
like typing if x is None".


	Apart from all the other things about this proposal I don't support, I 
don't support the creation of a mysterious "expression" which is not a 
first class value and cannot be used or evaluated in any way except 
automatically in the context of calling a function.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HFRZ3Y7VCBRH7SIDFT563ZPVLMYEL5QU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

2021-10-30 Thread Rob Cliffe via Python-ideas


Warning: Bear of Very Little Brain talking.

Aren't we in danger of missing the wood for the (deferred-evaluation) 
trees here?


    Late-bound-arguments defaults are USEFUL.  Being able to specify 
that a default value is a new empty list or a new empty set is USEFUL.  
Plenty of people have asked on Stack Overflow why it doesn't "work" 
already.  Yes, lots of those people will still ask, but with late bound 
defaults they can get a completely satisfactory answer.
    Using None or other sentinel to mean "no value was passed" is 
basically a hack.  A kludge.  It is MUCH CLEANER to get a default value 
when no value is passed without resorting to such a hack. Which is fine 
if the default value is constant (please don't waste your time quibbling 
with me if I get the technicalities wrong) and can be evaluated early, 
but needs a late-bound-default if ... well, if it needs to be evaluated 
late.
    Furthermore, having a late-bound default which refers to preceding 
parameters such as

        ..., hi:=len(a) ...
is totally clear to anyone who understands function parameter default 
values (i.e. pretty much anybody who can read or write functions).  And 
again VERY convenient.

As Chris A has already said, the idiom
    if param = None:
        param = [] # or whatever
is VERY frequent.  And saving 2 lines of code in a particular scenario 
may be no great virtue, but when it happens so VERY frequently (with no 
loss of clarity) it certainly is.


PEP 671, if accepted, will undoubtedly be USEFUL to Python programmers.

As for deferred evaluation objects:
    First, Python already has various ways of doing deferred evaluation:
        Lambdas
        Strings, which can be passed to eval / exec / compile.
        You can write decorator(s) for functions to make them defer 
their evaluation.

        You can write a class of deferred-evaluation objects.
None of these ways is perfect.  Each have their pros and cons.  The 
bottom line, if I understand correctly (maybe I don't) is that there has 
to be a way of specifying (implicitly or explicitly) when the (deferred) 
evaluation occurs, and also what the evaluation context is (e.g. for 
eval, locals and globals must be specified, either explicitly or 
implicitly).
So maybe it would be nice if Python had its own "proper" 
deferred-evaluation model.

But it doesn't.
And as far as I know, there isn't a PEP, either completed, or at least 
well on the way, which proposes such a model.  (Perhaps it's a really 
difficult problem.  Perhaps there is not enough interest for someone to 
have already tried it.  Perhaps there are so many ways of doing it, as 
CHB says below, that it's hard to decide which.  I don't know.)  If 
there were, it would be perfectly reasonable to ask how it would 
interact with PEP 671.  But as it's just vapourware, it seems wrong to 
stall PEP 671 (how long? indefinitely?) because of the POSSIBILITY that 
it MIGHT turn out to more convenient for some future model (IF that ever 
happens) if PEP 671 had been implemented in a different way.
[I just saw a post from David Mertz giving an idea, using a "delay" 
keyword, before I finished composing this email.  But that is just a 
sketch, miles away from a fully-fledged proposal or PEP.  And probably 
just one of very many that have appeared over the years on envelope 
backs before vanishing into the sunset ...]


Best wishes
Rob Cliffe



On 31/10/2021 02:08, Christopher Barker wrote:
I'm a bit confused as to why folks are making pronouncements about 
their support for this PEP before it's even finished, but, oh well.


As for what seems like one major issue:

Yes, this is a kind of "deferred" evaluation, but it is not a general 
purpose one, and that, I think, is the strength of the proposal, it's 
small and specific, and, most importantly, the scope in which the 
expression will be evaluated is clear and simple. In contrast, a 
general deferred object would, to me, be really confusing about what 
scope it would get evaluated in -- I can't even imagine how I would do 
that -- how the heck am I supposed to know what names will be 
available in some function scope I pass this thing into??? Also, this 
would only allow a single expression, not an arbitrary amount of code 
-- if we're going to have some sort of "deferred object" -- folks will 
very soon want more than that, and want full deferred function 
evaluation. So that really is a whole other kettle of fish, and should 
be considered entirely separately.


As for inspect -- yes, it would be great for these late-evaluated 
defaults to have a good representation there, but I can only see that 
as opening the door to more featureful deferred object, certainly not 
closing it.


-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython

___
Python-ideas mailing list

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 1:52 PM Steven D'Aprano  wrote:
> I believe that the PEP should declare that how the unevaluated defaults
> are stored prior to evaluation is a private implementation detail. We
> need an API (in the inspect module? as a method on function objects?) to
> allow consumers to query those defaults for human-readable text, e.g.
> needed by help(). But beyond that, I think they should be an opaque
> blob.

I haven't re-posted, but while writing up the implementation, I did
add a section on implementation details to the PEP.

https://www.python.org/dev/peps/pep-0671/#implementation-details

(FWIW, it's very similar to what you were saying in the other thread,
only I chose to use Ellipsis, since it's commonly understood as a
placeholder, rather than NotImplemented, which is a special signal for
binary operators.)

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FQD7GBBF76UUJEPWRSJPGSZOFRQKGVR6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 1:28 PM David Mertz, Ph.D.
 wrote:
>
> On Sat, Oct 30, 2021, 9:40 PM Chris Angelico  wrote:
>>
>> > I'm not sure what I think of a general statement like:
>> >
>> > @do_later = fun1(data) + fun2(data)
>> >
>> > I.e. we expect to evaluate the first class object `do_later` in some other 
>> > context, but only if requested within a program branch where `data` is in 
>> > scope.
>>
>> If you want to create a "deferred" type, go ahead, but it won't
>> conflict with this. There wouldn't be much to gain by restricting it to 
>> function arguments.
>
>
> I agree there's no gain in restricting deferred computation to function 
> arguments, but that's EXACTLY what your proposal is.

That's if you were to create a deferred *type*. An object which can be
evaluated later. My proposal is NOT doing that, because there is no
object that represents the unevaluated expression. You can't pull that
expression out and evaluate it somewhere else. It wouldn't be
meaningful.

>> The way you've written it, it's bound to an assignment, which seems very 
>> odd. Are you creating an arbitrary object which can be evaluated in some 
>> other context? Wouldn't that be some sort of constructor call?
>
>
> It's true I don't particularly like the @ syntax. I was just speculating on 
> continuity with Steven's syntax.
>
> Here's my general proposal, which I actually want, but indeed don't have an 
> implementation for. I think a soft keyword is best, such as 'defer' or 
> 'later', but let's call it 'delay' for now to avoid the prior hang up on the 
> word.
>
> (A)
>
> def foo(a: list, size: int = delay len(a)) -> None:
> print("The list has length", size)
>
> Whenever a name is referenced in this future Python, the interpreter first 
> asks if it is a special delayed object. If not, do exactly what is done now. 
> However, if it *IS* that special kind of object, instead do something akin to 
> 'eval()'.
>

Okay. Picture this situation:

def ctx():
dflt, scratch = 1, 2
def set_default(x):
nonlocal dflt; dflt = x
def frob1(a=>dflt):
print(a)
def frob2(a = delay dflt):
print(a)
return set_default, frob

With frob1, the compiler knows exactly which names mean which
variables, just as in all current code. In this case, it knows that
dflt is a nonlocal, and CPython will use LOAD_DEREF to look it up.

With frob2, a is some opaque object that happens to be a deferred
expression. That expression could be evaluated in any context. The
compiler has to snapshot every variable in every containing scope (in
this case, 'scratch'), in case the evaluation of a might happen to
refer to them.

In fact, it's worse. *EVERY* closure has to retain *EVERY* containing
variable, just in case it's used in this way.

Either that, or these delayed expressions are just eval'd strings, and
need to be explicitly passed their globals and locals, which fails for
closures anyway.

In contrast, frob1 puts the code right there in the function, so
everything behaves correctly.

> No, the delayed object probably shouldn't just contain a string, but perhaps 
> a chunk of compiled bytecode.

I don't know about other Python implementations, but in CPython,
bytecode needs to know what kind of name a thing represents -
LOAD_FAST, LOAD_DEREF, LOAD_GLOBAL - and for closures (dereferences),
it needs to ensure that the surrounding context uses DEREF as well,
when it otherwise would use FAST. Plus, every name reference in
CPython bytecode is a lookup to a table of names (so it'll say, for
instance, LOAD_GLOBAL 3 and the third entry in the name list might be
"print", so it'll look up the global named print). CPython bytecode
isn't well suited to this. So it would have to be some other sort of
bytecode - something that retains very little context, and only has a
basic parse.

In fact, I think the AST is probably the closest (at least in CPython
- again, I don't know other Pythons) to what you want here. It's
basically the same thing as source code, but tokenized and turned into
a logical tree. Unfortunately, AST can't be executed as such, and
needs to be compiled into something ready to use.

> So what if we don't want to evaluate the delayed object? Either the same or a 
> different keyword can do that:
>
> def foo(a: list, size: int = delay len(a)) -> None:
> a.append(42)
> bar(a, delay size)
>
> def bar(a, the_length):
> print("The expanded list has length", the_length)

If it's the same keyword, you have a fundamental ambiguity: does that
mean "use the name size in the target context", or "use the deferred
object with its existing names"?

> What if we want to be more general with delaying (and potentially skipping) 
> actions?
>
> expensive1 = delay big_computation(data)
> expensive2 = delay slow_lookup(data)
>
> def get_answer(data):
> # use globals for the example, less so in practice
> if approximate_compute_cost(data) > 1_000_000:
> return expensive2
> else:
>

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sat, Oct 30, 2021 at 03:52:14PM -0700, Brendan Barnwell wrote:

>   The way you use the term "default value" doesn't quite work for me. 
> More and more I think that part of the issue here is that "hi=>len(a)" 
> we aren't providing a default value at all.  What we're providing is 
> default *code*.  To me a "value" is something that you can assign to a 
> variable, and current (aka "early bound") argument defaults are that.

I think you are twisting the ordinary meaning of "default value" to 
breaking point in order to argue against this proposal.

When we talk about "default values" for function parameters, we always 
mean something very simple: when you call the function, you can leave 
out the argument for some parameter from your call, and it will be 
automatically be assigned a default value by the time the code in the 
body of the function runs.

It says nothing about how that default value is computed, or where it 
comes from; it says nothing about whether it is evaluated at compile 
time, or function creation time, or when the function is called.

In this case, there are at least three models for providing that default 
value:

1. The value must be something which can be computed by the compiler, at 
compile time, without access to the runtime environment. Nothing that 
cannot be evaluated by static analysis can be used as the default.

(In practice, that may limit defaults to literals.)

2. The value must be something which can be computed by the interpreter 
at function definition time. (Early binding.) This is the status quo for 
Python, but not for other languages like Smalltalk.

3. The value can be computed at function call time. (Late binding.) 
That's what Lisp (Smalltalk? others?) does. They are still default 
values, and it is specious to call them "code". From the perspective of 
the programmer, the parameter is bound to a value at call time, just 
like early binding.

Recall that the interpreter still has to execute code to get the early 
bound value. It doesn't happen by magic: the interpreter needs to run 
code to fetch the precomputed value and bind it to the parameter. Late 
binding is *exactly* the same except we leave out the "pre". The thing 
you get bound to the parameter is still a value, not the code used to 
generate that value.

The Python status quo (early binding, #2 above) is that if you want to 
delay the computation until call time, you have to use a sentinel, then 
calculate the default value yourself inside the body of the function, 
then bind it to the parameter manually. But that's just a work-around 
for lack of interpreter support for late binding.

> But these new late-bound arguments aren't really default "values", 

Of course they are, in the only sense that matters: when the body of the 
function runs, the parameter is bound to an object. That object is a 
value. How the interpreter got that value from, and when it was 
evaluated, it neither here nor there. It is still a value one way or 
another.

> they're code that is run under certain circumstances.  If we want to 
> make them values, we need to define "evaluate len(a) later" as some kind 
> of first-class value.

No we don't. Evaluating the default value later from outside the 
function's local scope will usually fail, and even if it doesn't fail, 
there's no use-cases for it. (Yet. If a good one comes up, we can add an 
API for it later.)

And evaluating it from inside the function's local scope is unnecessary, 
as the interpreter has already done it for you.

I believe that the PEP should declare that how the unevaluated defaults 
are stored prior to evaluation is a private implementation detail. We 
need an API (in the inspect module? as a method on function objects?) to 
allow consumers to query those defaults for human-readable text, e.g. 
needed by help(). But beyond that, I think they should be an opaque 
blob.

Consider the way we implement comprehensions as functions. But we don't 
make that a language rule. It's just an implementation detail, and may 
change. Likewise unevaluated defaults may be functions, or ASTs, or 
blobs of compiled byte-code, or code objects, or even just stored as 
source code. Whatever the implementors choose.

>   Increasingly it seems to me as if you are placing inordinate weight 
>   on the idea that the benefit of default arguments is providing a "human 
> readable" description in the default help() and so on.  And, to be 
> frank, I just don't care about that. 

Better help() is just the icing on the cake. The bigger advantages 
include that we can reduce the need for special sentinels, and that 
default values no longer need to be evaluated by hand in the body of the 
function, as the interpreter handles them. And we write the default 
value in the function signature, where they belong, instead of in the 
body.

Don't discount value of having the interpreter take on the grunt-work of 
evaluating defaults. Whatever extra complexity goes into the

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sat, Oct 30, 2021, 9:40 PM Chris Angelico  wrote:

> > I'm not sure what I think of a general statement like:
> >
> > @do_later = fun1(data) + fun2(data)
> >
> > I.e. we expect to evaluate the first class object `do_later` in some
> other context, but only if requested within a program branch where `data`
> is in scope.
>
> If you want to create a "deferred" type, go ahead, but it won't
> conflict with this. There wouldn't be much to gain by restricting it to
> function arguments.
>

I agree there's no gain in restricting deferred computation to function
arguments, but that's EXACTLY what your proposal is.

The way you've written it, it's bound to an assignment, which seems very
> odd. Are you creating an arbitrary object which can be evaluated in some
> other context? Wouldn't that be some sort of constructor call?
>

It's true I don't particularly like the @ syntax. I was just speculating on
continuity with Steven's syntax.

Here's my general proposal, which I actually want, but indeed don't have an
implementation for. I think a soft keyword is best, such as 'defer' or
'later', but let's call it 'delay' for now to avoid the prior hang up on
the word.

(A)

def foo(a: list, size: int = delay len(a)) -> None:
print("The list has length", size)

Whenever a name is referenced in this future Python, the interpreter first
asks if it is a special delayed object. If not, do exactly what is done
now. However, if it *IS* that special kind of object, instead do something
akin to 'eval()'.

No, the delayed object probably shouldn't just contain a string, but
perhaps a chunk of compiled bytecode.

(B)

So what if we don't want to evaluate the delayed object? Either the same or
a different keyword can do that:

def foo(a: list, size: int = delay len(a)) -> None:
a.append(42)
bar(a, delay size)

def bar(a, the_length):
print("The expanded list has length", the_length)

(C)

What if we want to be more general with delaying (and potentially skipping)
actions?

expensive1 = delay big_computation(data)
expensive2 = delay slow_lookup(data)

def get_answer(data):
# use globals for the example, less so in practice
if approximate_compute_cost(data) > 1_000_000:
return expensive2
else:
return expensive1

That's it. It covers everything in your PEP, and great deal more that is
far more important, all using the same syntax.

A few special functions or operations should be able to look at the delayed
object without evaluating it. I'm not sure details, but e.g.

>>> print(delay expensive1)

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SU26ANO3ENROSLBLMETSP5GLKC2YE245/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

2021-10-30 Thread Christopher Barker

On Sat, Oct 30, 2021 at 7:03 PM Brendan Barnwell 
wrote:

> > Right. That is a very real difference, which is why there is a very
> > real difference between early-bound and late-bound defaults. But both
> > are argument defaults.
>
> I don't 100% agree with that.
>
> I mean, here's another way to come at this.  Suppose we have this
> under
> the proposal (using some random syntax picked at random):
>
> def foo(a=1, b="two", c@=len(b), d@=a+c):
>
> You keep saying that c and d "are argument default" just like a
> and b.
>   So can you tell me what the argument default for each of those arguments?
>

I'm trying to figure out if this is a (english language) semantics issue or
a real conceptual issue.

Yes, we don't know what the default "value" (by either the general english
definition or the python definition), but we do know that the default will
be set to the result of the depression when evaluated in the context of the
function, which is very clear to me, at least.

The default for argument a is an integer.  The default for argument
> b
> is a string.  Can you tell me, in comparable terms, what the defaults
> for arguments c and d are?
>

yes: the default for c is the result of evaluating `len(b)`, and default to
d is the result of evaluating `a+c`

in contrast, what are defaults in this case?

def foo(a=1, b="two", c=None, d=None):

obviously, they are None -- but how useful is that? How about:

def foo(a=1, b="two", c=None, d=None):
"""
...
if None, c is computed as the length of the value of b, and d is
computed as that length plus a
"""
if c is None:
c = len(b)
if d is None:
d = a + c

Is that really somehow more clear? And you'd better hope that the docstring
matches the code!

I'm having a really hard time seeing how this PEP would make anything less
clear or confusing.

-CHB







>
> Currently, every argument default is a first-class value.  As I
> understand it, your proposal breaks that assumption, and now argument
> defaults can be some kind of "construct" that is not a first class
> value, not any kind of object, just some sort of internal entity that no
> one can see or manipulate in any way, it just gets automatically
> evaluated later.
>
> I really don't like that.  One of the things I like about Python
> is the
> "everything is an object" approach under which most of the things that
> programmers work with, apart from a very few base syntactic constructs,
> are objects.  Many previous expansions to the language, like decorators,
> context managers, the iteration protocol, etc., worked by building on
> this object model.  But this proposal seems to diverge quite markedly
> from that.
>
> If the "late-bound default" is not an object of some kind just
> like the
> early-bound ones are, then I can't agree that both "are argument defaults".
>
> --
> Brendan Barnwell
> "Do not follow where the path may lead.  Go, instead, where there is no
> path, and leave a trail."
> --author unknown
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/Q7OGRKD7CRMNOB6NFF4KZTOC7ZJ3GNSO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KUHL5CP6NTJ3A6IE3Z3YCDLZHEJAGSUO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 1:03 PM Brendan Barnwell  wrote:
>
> On 2021-10-30 18:29, Chris Angelico wrote:
> >> > What I am saying is that there is a qualitative difference 
> >> > between "I
> >> >know now (at function definition time) what value to use for this
> >> >argument if it's missing" and "I know now (at function definition time)
> >> >*what I will do*  if this argument is missing".  Specifying "what you
> >> >will do" is naturally what you do inside the function.  It's a process
> >> >to be done later, it's logic, it's code.  It is not the same as
> >> >finalizing an actual VALUE at function definition time.  So yes, there
> >> >is a qualitative difference between:
> >> >
> >> ># this
> >> >if argument is undefined:
> >> > argument = some_constant_value
> >> >
> >> ># and this
> >> >if argument is undefined:
> >> > # arbitrary code here
> >> >
> >> > I mean, the difference is that in one case arbitrary code is 
> >> > allowed!
> >> >That's a big difference.
> > Right. That is a very real difference, which is why there is a very
> > real difference between early-bound and late-bound defaults. But both
> > are argument defaults.
>
> I don't 100% agree with that.
>
> I mean, here's another way to come at this.  Suppose we have this 
> under
> the proposal (using some random syntax picked at random):
>
> def foo(a=1, b="two", c@=len(b), d@=a+c):
>
> You keep saying that c and d "are argument default" just like a and b.
>   So can you tell me what the argument default for each of those arguments?

The default for a is the integer 1. The default for b is the string
"two". The default for c is the length of b. The default for d is the
sum of a and c.

> The default for argument a is an integer.  The default for argument b
> is a string.  Can you tell me, in comparable terms, what the defaults
> for arguments c and d are?

You're assuming that every default is a *default value*. That is the
current situation, but it is by no means the only logical way a
default can be defined. See above: c's default is the length of b,
which is presumably an integer, and d's default is the sum of that and
a, which is probably also an integer (unless a is passed as a float or
something).

> Currently, every argument default is a first-class value.  As I
> understand it, your proposal breaks that assumption, and now argument
> defaults can be some kind of "construct" that is not a first class
> value, not any kind of object, just some sort of internal entity that no
> one can see or manipulate in any way, it just gets automatically
> evaluated later.
>
> I really don't like that.  One of the things I like about Python is 
> the
> "everything is an object" approach under which most of the things that
> programmers work with, apart from a very few base syntactic constructs,
> are objects.  Many previous expansions to the language, like decorators,
> context managers, the iteration protocol, etc., worked by building on
> this object model.  But this proposal seems to diverge quite markedly
> from that.

What object is this?

a if random.randrange(2) else b

Is that an object? No; it's an expression. It's a rule. Not EVERYTHING
is an object. Every *value* is an object, and that isn't changing. Or
here, what value does a have?

def f():
if 0: a = 1
print(a)

Does it have a value? No. Is it an object? No. The lack of value is
not itself a value, and there is no object that represents it.

> If the "late-bound default" is not an object of some kind just like 
> the
> early-bound ones are, then I can't agree that both "are argument defaults".
>

Then we disagree. I see them both as perfectly valid defaults - one is
a default value, the other is a default expression.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YXU74N35DBDZE6O3TZYWGIUHJYT6NCZD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

2021-10-30 Thread Christopher Barker

I'm a bit confused as to why folks are making pronouncements about their
support for this PEP before it's even finished, but, oh well.

As for what seems like one major issue:

Yes, this is a kind of "deferred" evaluation, but it is not a general
purpose one, and that, I think, is the strength of the proposal, it's small
and specific, and, most importantly, the scope in which the expression will
be evaluated is clear and simple. In contrast, a general deferred object
would, to me, be really confusing about what scope it would get evaluated
in -- I can't even imagine how I would do that -- how the heck am I
supposed to know what names will be available in some function scope I pass
this thing into??? Also, this would only allow a single expression, not an
arbitrary amount of code -- if we're going to have some sort of "deferred
object" -- folks will very soon want more than that, and want full deferred
function evaluation. So that really is a whole other kettle of fish, and
should be considered entirely separately.

As for inspect -- yes, it would be great for these late-evaluated defaults
to have a good representation there, but I can only see that as opening the
door to more featureful deferred object, certainly not closing it.

-CHB

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DYH5LZ3Z6CPYDO7CR5OZRQXSSAE7VDD3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults


On 2021-10-30 18:29, Chris Angelico wrote:

> What I am saying is that there is a qualitative difference between "I
>know now (at function definition time) what value to use for this
>argument if it's missing" and "I know now (at function definition time)
>*what I will do*  if this argument is missing".  Specifying "what you
>will do" is naturally what you do inside the function.  It's a process
>to be done later, it's logic, it's code.  It is not the same as
>finalizing an actual VALUE at function definition time.  So yes, there
>is a qualitative difference between:
>
># this
>if argument is undefined:
> argument = some_constant_value
>
># and this
>if argument is undefined:
> # arbitrary code here
>
> I mean, the difference is that in one case arbitrary code is allowed!
>That's a big difference.

Right. That is a very real difference, which is why there is a very
real difference between early-bound and late-bound defaults. But both
are argument defaults.


I don't 100% agree with that.

	I mean, here's another way to come at this.  Suppose we have this under 
the proposal (using some random syntax picked at random):


def foo(a=1, b="two", c@=len(b), d@=a+c):

	You keep saying that c and d "are argument default" just like a and b. 
 So can you tell me what the argument default for each of those arguments?


	The default for argument a is an integer.  The default for argument b 
is a string.  Can you tell me, in comparable terms, what the defaults 
for arguments c and d are?


	Currently, every argument default is a first-class value.  As I 
understand it, your proposal breaks that assumption, and now argument 
defaults can be some kind of "construct" that is not a first class 
value, not any kind of object, just some sort of internal entity that no 
one can see or manipulate in any way, it just gets automatically 
evaluated later.


	I really don't like that.  One of the things I like about Python is the 
"everything is an object" approach under which most of the things that 
programmers work with, apart from a very few base syntactic constructs, 
are objects.  Many previous expansions to the language, like decorators, 
context managers, the iteration protocol, etc., worked by building on 
this object model.  But this proposal seems to diverge quite markedly 
from that.


	If the "late-bound default" is not an object of some kind just like the 
early-bound ones are, then I can't agree that both "are argument defaults".


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Q7OGRKD7CRMNOB6NFF4KZTOC7ZJ3GNSO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

2021-10-30 Thread Christopher Barker

On Sat, Oct 30, 2021 at 6:32 PM Chris Angelico  wrote:

> We could just write every function with *args, **kwargs, and then do
> all argument checking inside the function. We don't do this, because
> it's the job of the function header to manage this. It's not the
> function body's job to replace placeholders with actual values when
> arguments are omitted.
>

This is actually a great point, and I don't think it' sa straw man -- a
major project I'm soaring on is highly dynamic with a lot of subclassing
with complex __init__ signatures. And due to laziness or, to be generous,
attempts to be DRY, we have a LOT of mostly *args, **kwargs
parameterizations, and the result is that an, e.g. typo in  a parameter
name doesn't get caught till the very top of the class hierarchy, and it's
REALLY hard to tell where the issue is.

(and there is literally code manipulating kwargs, like: thing =
kwargs.get('something')

I'm pushing to refactor that code somewhat to have more clearly laid out
function definitions, even if that means some repetition -- after all, we
need to document it anyway, so why not have the interpreter do some of the
checking for us?

The point is: clearly specifying what's required, what's optional, and what
the defaults are if optional, is really, really useful -- and this PEP will
add another very handy feature to that.

-CHB

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2PBO43E6HQRCAXLWAYW4223LHLKBOTET/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 12:31 PM David Mertz, Ph.D.
 wrote:
>
> On Sat, Oct 30, 2021, 6:29 PM Chris Angelico  wrote:
>>
>> > At first I thought it might be harmless, but nothing I really care about. 
>> > After the discussion, I think the PEP would be actively harmful to future 
>> > Python features.
>
>
>> It's nothing more than an implementation detail. If you want to suggest - or 
>> better yet, write - an alternative implementation, I would welcome it. Can 
>> you explain how this is "actively harmful"?
>
>
> Both the choice of syntax and the discussion of proposed implementation (both 
> yours and Steven's) would make it more difficult later to advocate and 
> implement a more general "deferred" mechanism in the future.
>
> If you were proposing the form that MAL and I proposed (and a few others 
> basically agreed) of having a keyword like 'defer' that could, in concept, 
> only be initially available in function signatures but later be extended to 
> other contexts, I wouldn't see a harm. Maybe Steven's `@name=` could 
> accommodate that too.
>
> I'm not sure what I think of a general statement like:
>
> @do_later = fun1(data) + fun2(data)
>
> I.e. we expect to evaluate the first class object `do_later` in some other 
> context, but only if requested within a program branch where `data` is in 
> scope.

The problem here is that you're creating an object that can be
evaluated in someone else's scope. I'm not creating that. I'm creating
something that gets evaluated in its own scope - the function
currently being defined.

If you want to create a "deferred" type, go ahead, but it won't
conflict with this. There wouldn't be much to gain by restricting it
to function arguments. Go ahead and write up a competing proposal -
it's much more general than this.

> The similarity to decorators feels wrong, even though I think it's probably 
> not ambiguous syntactically.
>

The way you've written it, it's bound to an assignment, which seems
very odd. Are you creating an arbitrary object which can be evaluated
in some other context? Wouldn't that be some sort of constructor call?

> In a sense, the implementation doesn't matter as much if the syntax is 
> something that could be used more widely. Clearly, adding something to the 
> dunders of a function object isn't a general mechanism, but if behavior was 
> kept consistent, the underlying implementation could change in principle.
>
> Still, the check for a sentinel in the first few lines of a function body is 
> easy and fairly obvious, as well as long-standing. New syntax for a trivial 
> use is just clutter and cognitive burden for learners and users.
>

A trivial use? But a very common one. The more general case would be
of value, but would also be much more cognitive burden. But feel free
to write something up and see how that goes. Maybe it will make a
competing proposal to PEP 671; or maybe it'll end up being completely
independent.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6C667Y4SAAXINH2HGQZY42ISWKIUAZVN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sat, Oct 30, 2021, 6:29 PM Chris Angelico  wrote:

> > At first I thought it might be harmless, but nothing I really care
> about. After the discussion, I think the PEP would be actively harmful to
> future Python features.
>

It's nothing more than an implementation detail. If you want to suggest -
> or better yet, write - an alternative implementation, I would welcome it.
> Can you explain how this is "actively harmful"?
>

Both the choice of syntax and the discussion of proposed implementation
(both yours and Steven's) would make it more difficult later to advocate
and implement a more general "deferred" mechanism in the future.

If you were proposing the form that MAL and I proposed (and a few others
basically agreed) of having a keyword like 'defer' that could, in concept,
only be initially available in function signatures but later be extended to
other contexts, I wouldn't see a harm. Maybe Steven's `@name=` could
accommodate that too.

I'm not sure what I think of a general statement like:

@do_later = fun1(data) + fun2(data)

I.e. we expect to evaluate the first class object `do_later` in some other
context, but only if requested within a program branch where `data` is in
scope.

The similarity to decorators feels wrong, even though I think it's probably
not ambiguous syntactically.

In a sense, the implementation doesn't matter as much if the syntax is
something that could be used more widely. Clearly, adding something to the
dunders of a function object isn't a general mechanism, but if behavior was
kept consistent, the underlying implementation could change in principle.

Still, the check for a sentinel in the first few lines of a function body
is easy and fairly obvious, as well as long-standing. New syntax for a
trivial use is just clutter and cognitive burden for learners and users.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FC44QLOZKYOGAODUC3UIH5NHTF57VW3M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 12:17 PM Brendan Barnwell  wrote:
>
> On 2021-10-30 16:12, Chris Angelico wrote:
> >> > Increasingly it seems to me as if you are placing inordinate 
> >> > weight on
> >> >the idea that the benefit of default arguments is providing a "human
> >> >readable" description in the default help() and so on.  And, to be
> >> >frank, I just don't care about that.  We can already provide
> >> >human-readable descriptions in documentation and we should do that
> >> >instead of trying to create gimped human-readable descriptions that only
> >> >work in special cases.  Or, to put it even more bluntly, from my
> >> >perspective, having help() show something maybe sort of useful just in
> >> >the case where the person wrote very simple default-argument logic and
> >> >didn't take the time to write a real docstring is simply not a
> >> >worthwhile goal.
> > Interesting. Then why have default arguments at all? What's the point
> > of saying "if omitted, the default is zero" in a machine-readable way?
> > After all, you could just have it in the docstring. And there are
> > plenty of languages where that's the case.
>
> The point of default arguments is to allow users of the function to
> omit arguments at the call site.  It doesn't have anything to do with
> docstrings.

That's optional parameters. It's entirely possible to have optional
parameters with no concept of defaults; it means the function can be
called with or without those arguments, and the corresponding
parameters will either be set to some standard "undefined" value, or
in some other way marked as not given.

> Or do you mean why not just have all omitted arguments set to some 
> kind
> of "undefined" value and then check each one in the body of the function
> and replace it with a default if you want to?  Well, for one thing it
> makes for cleaner error handling, since Python can tell at the time of
> the call that a required argument wasn't supplied and raise that error
> right away.

That's the JavaScript way - every parameter is optional. But it's
entirely possible to have some mandatory and some optional, while
still not having a concept of defaults.

Python, so far, has completely conflated those two concepts: if a
parameter is optional, it always has a default argument value. (The
converse is implied - mandatory args can't have defaults.)

>  It's sort of like a halfway type check, where you're not
> actually checking that the correct types of arguments were passed, but
> at least you know that the arguments that need to be passed were passed
> and not left out entirely.

Given that Python doesn't generally have any sort of argument type
checking, that's exactly what we'll get by default.

> For another thing, it does mean that if you know the default at the
> time you're defining the function, you can specify it then.  What you
> can't do is specify the default if you don't know the default at
> function definition time, but only know "how you're going to decide what
> value to use" (which is a process, not a value).

Right. That's the current situation.

> > I'm of the opinion that having more information machine-readable is
> > always better. Are you saying that it isn't? Or alternatively, that
> > it's only useful when it fits in a strict subset of constant values
> > (values that don't depend on anything else, and can be calculated at
> > function definition time)?
>
> Now wait a minute, before you said the goal was for it to be human
> readable, but now you're saying it's about being machine readable!  :-)

Truly machine readable is the best: any tool can know exactly what
will happen. That is fundamentally not possible when the default value
is calculated. Mostly machine readable means that the machine can
figure out what the default is, even if it doesn't know what that
means. My proposal (not in the reference implementation as yet) is to
have late-bound defaults contain a marker saying "the default will be
len(a)", even though the "len(a)" part would be just a text string.

> What I am saying is that there is a qualitative difference between "I
> know now (at function definition time) what value to use for this
> argument if it's missing" and "I know now (at function definition time)
> *what I will do* if this argument is missing".  Specifying "what you
> will do" is naturally what you do inside the function.  It's a process
> to be done later, it's logic, it's code.  It is not the same as
> finalizing an actual VALUE at function definition time.  So yes, there
> is a qualitative difference between:
>
> # this
> if argument is undefined:
> argument = some_constant_value
>
> # and this
> if argument is undefined:
> # arbitrary code here
>
> I mean, the difference is that in one case arbitrary code is allowed!
> That's a big difference.

Right. That is a very real difference, which is why there is a very
real difference between early-bound and

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults


On 2021-10-30 16:12, Chris Angelico wrote:

> Increasingly it seems to me as if you are placing inordinate weight on
>the idea that the benefit of default arguments is providing a "human
>readable" description in the default help() and so on.  And, to be
>frank, I just don't care about that.  We can already provide
>human-readable descriptions in documentation and we should do that
>instead of trying to create gimped human-readable descriptions that only
>work in special cases.  Or, to put it even more bluntly, from my
>perspective, having help() show something maybe sort of useful just in
>the case where the person wrote very simple default-argument logic and
>didn't take the time to write a real docstring is simply not a
>worthwhile goal.

Interesting. Then why have default arguments at all? What's the point
of saying "if omitted, the default is zero" in a machine-readable way?
After all, you could just have it in the docstring. And there are
plenty of languages where that's the case.


	The point of default arguments is to allow users of the function to 
omit arguments at the call site.  It doesn't have anything to do with 
docstrings.


	Or do you mean why not just have all omitted arguments set to some kind 
of "undefined" value and then check each one in the body of the function 
and replace it with a default if you want to?  Well, for one thing it 
makes for cleaner error handling, since Python can tell at the time of 
the call that a required argument wasn't supplied and raise that error 
right away.  It's sort of like a halfway type check, where you're not 
actually checking that the correct types of arguments were passed, but 
at least you know that the arguments that need to be passed were passed 
and not left out entirely.


	For another thing, it does mean that if you know the default at the 
time you're defining the function, you can specify it then.  What you 
can't do is specify the default if you don't know the default at 
function definition time, but only know "how you're going to decide what 
value to use" (which is a process, not a value).



I'm of the opinion that having more information machine-readable is
always better. Are you saying that it isn't? Or alternatively, that
it's only useful when it fits in a strict subset of constant values
(values that don't depend on anything else, and can be calculated at
function definition time)?


	Now wait a minute, before you said the goal was for it to be human 
readable, but now you're saying it's about being machine readable!  :-)


	What I am saying is that there is a qualitative difference between "I 
know now (at function definition time) what value to use for this 
argument if it's missing" and "I know now (at function definition time) 
*what I will do* if this argument is missing".  Specifying "what you 
will do" is naturally what you do inside the function.  It's a process 
to be done later, it's logic, it's code.  It is not the same as 
finalizing an actual VALUE at function definition time.  So yes, there 
is a qualitative difference between:


# this
if argument is undefined:
argument = some_constant_value

# and this
if argument is undefined:
# arbitrary code here

	I mean, the difference is that in one case arbitrary code is allowed! 
That's a big difference.


	Based on some of your other posts, I'm guessing that what you mean 
about machine readability is that you appreciate certain kinds of 
labor-saving "self-documentation" techniques, whereby when we write the 
machine-readable code, the interpreter automatically derives some 
human-readable descriptions for stuff.  For instance when we write `def 
foo` we're just defining an arbitrary symbol to be used elsewhere in the 
code, but if we get an exception Python doesn't just tell us "exception 
in function number 1234" or the line number, but also tells us the 
function name.


	And yeah, I agree that can be useful.  And I agree that it would be 
"nice" if we could write "len(a)" without quotes as machine-readable 
code, and then have that stored as some human-readable thing that could 
be shown when appropriate.  But if that's nice, why is it only nice in 
function arguments?  Why is it only nice to be able to associate the 
code `len(a)` with the human-readable string "len(a)" just when that 
string happens to occur in a function signature?


	On top of that, even if I agree that that is useful, I see the benefit 
of that in this case (generating docstrings based on default arguments) 
as very marginal.  I think I agree with the spirit of what you mean 
"having more information machine-readable is always good", but of course 
I don't agree that that's literally true --- because you have to balance 
that good against other goods.  In this case, perhaps most notably, we 
have to balance it against the cognitive load of having two different 
ways to write arguments, which will have quite different semantics, 
which based on current proposals are going

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 9:54 AM Brendan Barnwell  wrote:
>
> On 2021-10-30 15:35, Chris Angelico wrote:
> > Bear in mind that the status quo is, quite honestly, a form of white
> > lie. In the example of bisect:
> >
> > def bisect(a, hi=None): ...
> > def bisect(a, hi=>len(a)): ...
> >
> > neither form of the signature will actually say that the default value
> > is the length of a. In fact, I have never said that the consumer
> > should eval it. There is fundamentally no way to determine the true
> > default value for hi without first figuring out what a is.
>
> The way you use the term "default value" doesn't quite work for me.
> More and more I think that part of the issue here is that "hi=>len(a)"
> we aren't providing a default value at all.  What we're providing is
> default *code*.  To me a "value" is something that you can assign to a
> variable, and current (aka "early bound") argument defaults are that.
> But these new late-bound arguments aren't really default "values",
> they're code that is run under certain circumstances.  If we want to
> make them values, we need to define "evaluate len(a) later" as some kind
> of first-class value.

This is correct. In fact, the way it is in the syntax, the second one
is a "default expression". But the Signature object can't actually
have the default expression, so it would have to have either a default
value, or some sort of marker.

Argument defaults, up to Python 3.11, are always default values. If
PEP 671 is accepted, it will make sense to have default values AND
default expressions.

> > So which is better: to have the value None, or to have a marker saying
> > "this will be calculated later, and here's a human-readable
> > description: len(a)"?
>
> Let me say that in a different way. . . :-)
>
> Which is better, to have a marker saying "this will be calculated
> later", or to have a marker saying "this will be calculated later, and
> here's a human-readable description"?
>
> My point is that None is already a marker.  It's true it's not a
> special-purpose marker meaning "this will be calculated later", but I
> think in practice it is a marker that says "be sure to read the
> documentation to understand what passing None will do here".

That's true. The trouble is that it isn't uniquely such a marker, and
in fact is very often the actual default value. When you call
dict.get(), the second argument has a default of None, and if you omit
it, you really truly do get None as a result.

Technically, for tools that look at func.__defaults__, I have Ellipsis
doing that kind of job. But (a) that's far less common than None, and
(b) there's a way to figure out whether it's a real default value or a
marker. Of course, there will still be functions that have
pseudo-defaults, so you can never be truly 100% sure, but at least you
get an indication for those functions that actually use default
expressions.

So what you have is a marker saying "this is either the value None, or
something that will be calculated later". Actually there are multiple
markers; None might mean that, but so might "", which is more likely to mean that it'll be
calculated later, but harder to recognize reliably. And in all cases,
it might not be a value that's calculated later, but it might be a
change in effect (maybe causing an exception to be raised rather than
a value being returned).

The markers currently are very ad-hoc and can't be depended on by
tools. There is fundamentally no way to do better than a marker, but
we can at least have more useful markers.

> Increasingly it seems to me as if you are placing inordinate weight on
> the idea that the benefit of default arguments is providing a "human
> readable" description in the default help() and so on.  And, to be
> frank, I just don't care about that.  We can already provide
> human-readable descriptions in documentation and we should do that
> instead of trying to create gimped human-readable descriptions that only
> work in special cases.  Or, to put it even more bluntly, from my
> perspective, having help() show something maybe sort of useful just in
> the case where the person wrote very simple default-argument logic and
> didn't take the time to write a real docstring is simply not a
> worthwhile goal.

Interesting. Then why have default arguments at all? What's the point
of saying "if omitted, the default is zero" in a machine-readable way?
After all, you could just have it in the docstring. And there are
plenty of languages where that's the case.

I'm of the opinion that having more information machine-readable is
always better. Are you saying that it isn't? Or alternatively, that
it's only useful when it fits in a strict subset of constant values
(values that don't depend on anything else, and can be calculated at
function definition time)?

> So really the status quo is "you can already have the human-readable
> description but you have to type it in the docstring yourself".

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On 2021-10-30 15:35, Chris Angelico wrote:

Bear in mind that the status quo is, quite honestly, a form of white
lie. In the example of bisect:

def bisect(a, hi=None): ...
def bisect(a, hi=>len(a)): ...

neither form of the signature will actually say that the default value
is the length of a. In fact, I have never said that the consumer
should eval it. There is fundamentally no way to determine the true
default value for hi without first figuring out what a is.

The way you use the term "default value" doesn't quite work for me.
More and more I think that part of the issue here is that "hi=>len(a)"
we aren't providing a default value at all. What we're providing is
default *code*. To me a "value" is something that you can assign to a
variable, and current (aka "early bound") argument defaults are that.
But these new late-bound arguments aren't really default "values",
they're code that is run under certain circumstances. If we want to
make them values, we need to define "evaluate len(a) later" as some kind
of first-class value.

So which is better: to have the value None, or to have a marker saying
"this will be calculated later, and here's a human-readable
description: len(a)"?

Let me say that in a different way. . . :-)

Which is better, to have a marker saying "this will be calculated
later", or to have a marker saying "this will be calculated later, and
here's a human-readable description"?

My point is that None is already a marker. It's true it's not a
special-purpose marker meaning "this will be calculated later", but I
think in practice it is a marker that says "be sure to read the
documentation to understand what passing None will do here".

Increasingly it seems to me as if you are placing inordinate weight on
the idea that the benefit of default arguments is providing a "human
readable" description in the default help() and so on. And, to be
frank, I just don't care about that. We can already provide
human-readable descriptions in documentation and we should do that
instead of trying to create gimped human-readable descriptions that only
work in special cases. Or, to put it even more bluntly, from my
perspective, having help() show something maybe sort of useful just in
the case where the person wrote very simple default-argument logic and
didn't take the time to write a real docstring is simply not a
worthwhile goal.

So really the status quo is "you can already have the human-readable
description but you have to type it in the docstring yourself". I don't
see that as a big deal. So yes, the status quo is better, because it is
not really any worse, and it avoids the complications that are arising
in this thread (i.e., what order are the arguments evaluated in, can
they reference each other, what symbol do we use, how do we implement it
without affecting existing introspection, etc.).

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."

--author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/CLKFLCAKSQZEY7FGFG5RIGIOL6OPWGHX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

2021-10-30 Thread Erik Demaine


On Sat, 30 Oct 2021, Brendan Barnwell wrote:

	I agree it seems totally absurd to add a type of deferred expression 
but restrict it to only work inside function definitions.


Functions are already a form of deferred evaluation.  PEP 671 is an 
embellishment to this mechanism for some of the code in the function signature 
to actually get executed within the body scope, *just like the body of the 
function*.  This doesn't seem weird to me.


If we have a way to create deferred expressions we should try to make them 
more generally usable.


Does anyone have a proposal for deferred expressions that could match the ease 
of use of PEP 671 in assigning a default argument of, say, `[]`?  The 
proposals I've seen so far in this thread involve checking `isdeferred` and 
then resolving that deferred.  This doesn't seem any easier than the existing 
sentinal approach for default arguments, whereas PEP 671 significantly 
simplifies this use-case.


I also don't see how a function could distinguish a deferred default argument 
and a deferred argument passed in from another function.  In my opinion, the 
latter would be really messy/dangerous to work with, because it could 
arbitrarily polute your scope.  Whereas late-bound default arguments make a 
lot of sense: they're written in the function itself (just in the signature 
instead of the body), so we can see by looking at the code what happens.


I've written code in dynamically scoped languages before.  I don't recall 
enjoying it.  But maybe I missed a proposal, or someone has an idea for how to 
fix these issues.


Erik
--
Erik Demaine  |  edema...@mit.edu  |  http://erikdemaine.org/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7ZJPAUJVUXJNI2SPAXK54CL3FGR22SCW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 9:20 AM Paul Moore  wrote:
>
> On Sat, 30 Oct 2021 at 23:13, Brendan Barnwell  wrote:
> >
> > On 2021-10-30 15:07, David Mertz, Ph.D. wrote:
> > > I'm -100 now on "deferred evaluation, but contorted to be useless
> > > outside of argument declarations."
> > >
> > > At first I thought it might be harmless, but nothing I really care
> > > about. After the discussion, I think the PEP would be actively harmful
> > > to future Python features.
> >
> > I'm not sure I'm -100, but still a hard -1, maybe -10.
> >
> > I agree it seems totally absurd to add a type of deferred expression
> > but restrict it to only work inside function definitions.  That doesn't
> > make any sense.  If we have a way to create deferred expressions we
> > should try to make them more generally usable.
>
> I was in favour of the idea, but having seen the implications I'm now
> -0.5, moving towards -1. I'm uncomfortable with *not* having a
> "proper" mechanism for building signature objects and other
> introspection (I don't consider having the expression as a string and
> requiring consumers to eval it, to be "proper"). And so, I think the
> implication is that this feature would need some sort of real deferred
> expression to work properly - and I'd rather deferred expressions were
> defined as a standalone mechanism, where the full range of use cases
> (including, but not limited to, late-bound defaults!) can be
> considered.
>

Bear in mind that the status quo is, quite honestly, a form of white
lie. In the example of bisect:

def bisect(a, hi=None): ...
def bisect(a, hi=>len(a)): ...

neither form of the signature will actually say that the default value
is the length of a. In fact, I have never said that the consumer
should eval it. There is fundamentally no way to determine the true
default value for hi without first figuring out what a is.

So which is better: to have the value None, or to have a marker saying
"this will be calculated later, and here's a human-readable
description: len(a)"?

I know that status quo wins a stalemate, but you're holding the new
feature to a FAR higher bar than current idioms.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5VREGOCSY7N6APQJUDYQ5QVOYKMS6R2R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 9:17 AM Brendan Barnwell  wrote:
>
> On 2021-10-26 20:15, Chris Angelico wrote:
> > One of my goals here is to ensure that the distinction between
> > early-bound and late-bound argument defaults is, again, orthogonal
> > with everything else. You should be able to change "x=None" to "x=>[]"
> > without having to wonder whether you're stopping yourself from adding
> > a type annotation in the future. This is why I'm strongly inclined to
> > syntaxes that adorn the equals sign, rather than those which put
> > tokens elsewhere (eg "@x=[]"), because it keeps the default part
> > self-contained.
>
> Another point that I forgot to mention when replying to this before:
>
> You are phrasing this in terms of orthogonality in argument-passing.
> But why think of it that way?  If we think of it in terms of expression
> evaluation, your proposal is quite non-orthogonal, because you're
> essentially creating a very limited form of deferred evaluation that
> works only in function arguments.  In a function argument, people will
> be able to do `x=>[]`, but they won't be able to do that anywhere else.
>   So you're creating a "mode" for deferred evaluation.
>
> This is why I don't get why you seem so resistant to the idea of a 
> more
> general deferred evaluation approach to this problem.  Generalizing
> deferred evaluation somehow would make the proposal MORE orthogonal to
> other features, because it would mean you could use a deferred
> expression as an argument in the same way you could use it in other places.
>

Please expand on this. How would you provide an expression that gets
evaluated in *someone else's context*? The way I've built it, the
expression is written and compiled in the context that it will run in.
The code for the default expression is part of the function that it
serves.

If I were to generalize this in any way, it would be to separate two
parts: "optional parameter, if omitted, leave unbound" and "if local
is unbound: do something". Not to "here's an expression, go evaluate
it later", which requires a lot more compiler help.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2V4LIN36AO2TBTSSI426JJDAY5K6QQBJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

On Sun, Oct 31, 2021 at 9:09 AM David Mertz, Ph.D.
 wrote:
>
> I'm -100 now on "deferred evaluation, but contorted to be useless outside of 
> argument declarations."
>
> At first I thought it might be harmless, but nothing I really care about. 
> After the discussion, I think the PEP would be actively harmful to future 
> Python features.
>

It's nothing more than an implementation detail. If you want to
suggest - or better yet, write - an alternative implementation, I
would welcome it.

Can you explain how this is "actively harmful"?

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/G3VL3BPCN4PUY2JC2WXZ7BRN7VARYO6Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

2021-10-30 Thread Paul Moore

On Sat, 30 Oct 2021 at 23:13, Brendan Barnwell  wrote:
>
> On 2021-10-30 15:07, David Mertz, Ph.D. wrote:
> > I'm -100 now on "deferred evaluation, but contorted to be useless
> > outside of argument declarations."
> >
> > At first I thought it might be harmless, but nothing I really care
> > about. After the discussion, I think the PEP would be actively harmful
> > to future Python features.
>
> I'm not sure I'm -100, but still a hard -1, maybe -10.
>
> I agree it seems totally absurd to add a type of deferred expression
> but restrict it to only work inside function definitions.  That doesn't
> make any sense.  If we have a way to create deferred expressions we
> should try to make them more generally usable.

I was in favour of the idea, but having seen the implications I'm now
-0.5, moving towards -1. I'm uncomfortable with *not* having a
"proper" mechanism for building signature objects and other
introspection (I don't consider having the expression as a string and
requiring consumers to eval it, to be "proper"). And so, I think the
implication is that this feature would need some sort of real deferred
expression to work properly - and I'd rather deferred expressions were
defined as a standalone mechanism, where the full range of use cases
(including, but not limited to, late-bound defaults!) can be
considered.

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3UNMX3BEFWA2HJMKQSPVGC574QCWP5UV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults


On 2021-10-26 20:15, Chris Angelico wrote:

One of my goals here is to ensure that the distinction between
early-bound and late-bound argument defaults is, again, orthogonal
with everything else. You should be able to change "x=None" to "x=>[]"
without having to wonder whether you're stopping yourself from adding
a type annotation in the future. This is why I'm strongly inclined to
syntaxes that adorn the equals sign, rather than those which put
tokens elsewhere (eg "@x=[]"), because it keeps the default part
self-contained.


Another point that I forgot to mention when replying to this before:

	You are phrasing this in terms of orthogonality in argument-passing. 
But why think of it that way?  If we think of it in terms of expression 
evaluation, your proposal is quite non-orthogonal, because you're 
essentially creating a very limited form of deferred evaluation that 
works only in function arguments.  In a function argument, people will 
be able to do `x=>[]`, but they won't be able to do that anywhere else. 
 So you're creating a "mode" for deferred evaluation.


	This is why I don't get why you seem so resistant to the idea of a more 
general deferred evaluation approach to this problem.  Generalizing 
deferred evaluation somehow would make the proposal MORE orthogonal to 
other features, because it would mean you could use a deferred 
expression as an argument in the same way you could use it in other places.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2DTNRAXSNDQL24LPWXK4QO443SQD7Y3K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults


On 2021-10-30 15:07, David Mertz, Ph.D. wrote:

I'm -100 now on "deferred evaluation, but contorted to be useless
outside of argument declarations."

At first I thought it might be harmless, but nothing I really care
about. After the discussion, I think the PEP would be actively harmful
to future Python features.


I'm not sure I'm -100, but still a hard -1, maybe -10.

	I agree it seems totally absurd to add a type of deferred expression 
but restrict it to only work inside function definitions.  That doesn't 
make any sense.  If we have a way to create deferred expressions we 
should try to make them more generally usable.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VYGHN757RIUFWJGSRLADCFYOTHBYRAXS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

I'm -100 now on "deferred evaluation, but contorted to be useless outside
of argument declarations."

At first I thought it might be harmless, but nothing I really care about.
After the discussion, I think the PEP would be actively harmful to future
Python features.

On Sat, Oct 30, 2021, 3:57 PM Eric V. Smith  wrote:

> On 10/26/2021 7:38 PM, Rob Cliffe via Python-ideas wrote:
> >
> > PS Can I echo Guido's plea that people don't derail this PEP by trying
> > to shoehorn deferred-evaluation-objects (or whatever you want to call
> > them) into it?  As Chris A says, that's a separate idea and should go
> > into a separate PEP.  If I need a screwdriver, I buy a screwdriver,
> > not an expensive Swiss Army knife.
> >
> As I've said before, I disagree with this. If you're going to introduce
> this feature, you need some way of building an inspect.Signature object
> that refers to the code to be executed. My concern is that if we add
> something that has deferred evaluation of code, but we don't think of
> how it might interact with other future uses of deferred evaluation, we
> might not be able to merge the two ideas.
>
> Maybe there's something that could be factored out of PEP 649 (Deferred
> Evaluation Of Annotations Using Descriptors) that could be used with PEP
> 671?
>
> That said, I'm still -1 on PEP 671.
>
> Eric
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/CCFPVNKCCPGTIXVB2IJ6LCAZPI7J5PCZ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BQXUCBMH4TD6NVNON7O4GBX4AOM5T3OZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

2021-10-30 Thread Eric V. Smith


On 10/26/2021 7:38 PM, Rob Cliffe via Python-ideas wrote:


PS Can I echo Guido's plea that people don't derail this PEP by trying 
to shoehorn deferred-evaluation-objects (or whatever you want to call 
them) into it?  As Chris A says, that's a separate idea and should go 
into a separate PEP.  If I need a screwdriver, I buy a screwdriver, 
not an expensive Swiss Army knife.


As I've said before, I disagree with this. If you're going to introduce 
this feature, you need some way of building an inspect.Signature object 
that refers to the code to be executed. My concern is that if we add 
something that has deferred evaluation of code, but we don't think of 
how it might interact with other future uses of deferred evaluation, we 
might not be able to merge the two ideas.


Maybe there's something that could be factored out of PEP 649 (Deferred 
Evaluation Of Annotations Using Descriptors) that could be used with PEP 
671?


That said, I'm still -1 on PEP 671.

Eric

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CCFPVNKCCPGTIXVB2IJ6LCAZPI7J5PCZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Context managers in expressions

2021-10-30 Thread Serhiy Storchaka

25.10.21 21:26, jcg.stu...@gmail.com пише:
> I've been thinking for a while that it would be good to be able to use 
> context managers in an expression context, and I haven't been able to come up 
> with any arguments against it (although it may not be to everyone's taste) 
> and I can't see any sign that it's come up before (although the words of it 
> don't make very specific search terms).
> My suggestion is that a "with" clause could be written after an expression 
> (consistent with the conditional operator being written with "if" after the 
> first expression).
> 
> In general, this would be:
>   a(b) with c(d) as b
> or it could allow multiple context managers:
>   a(b, c) with d(e) as b, f(g) as c
> 
> My original motivating example was this:
> 
> if name.startswith("https://;):
> data = requests.get(name).json()
> else:
> with open(name) as stream:
> data = json.load(stream)
> 
> which I'd much rather be able to write with a single expression:
> 
> data = (requests.get(name).json()
> if name.startswith("https://;)
> else (json.load(stream)
>   with open(name) as stream))
> 
> It would behave just like a "with" statement, but pass a value / values back.

I was going to propose this idea for discussion but did not have enough
time to formulate it.

It is not just a "what if we combine two random features idea", it would
cover most cases of recurrent requests for adding functions which merge
open() and json.load(), etc or adding new very specialized methods to
the Path object. Currently you can write

with open(filename, 'rb') as f:
data = json.load(f)

The only minor drawback of this idiom is that it is not expression (but
you always can write a new function). If you could write

data = json.load(f) with open(filename, 'rb') as f

this counter-argument will disappear.

Now, Mark Gordon asked a good question -- what to do if the context
manager suppresses an exception. We do not have value and we do not have
exception. The only answer I have -- raise a RuntimeError (or other
specialized exception). It should be an error to use context managers
which suppress exceptions in the "with" expression. Maybe there are
better ideas.

Before adding the "with" expressions we should consider other feature:
the "with" clause in comprehensions.

Currently comprehensions can contain "for" and "if" clauses. There
should be at least one "for" clause and it should be the first clause. I
propose to add the with clause. It will be interpreted the same way as
"if" and nested "for" clauses. [expr for x in iterable if cond() with
cm() as y] should be equivalent to:

result = []
for x in iterable:
if cond():
with cm() as y:
result.append(expr)

It should be discussed because there is a conflict between its syntax
and the "with" expression. The "with" clause should have priority over
the "with" expression (as the "if" clause has priority over the "if"
expression), but it is a breaking change if the latter will be
implemented first.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5QGMJWY7UGLEIKCXMIHMIM7ZLVCLJ646/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Context managers in expressions

2021-10-30 Thread John Sturdy

Thanks, Mark... I had missed that aspect entirely.

I think there might be a way round it... which actually was something I had
been thinking of including in the same suggestion but thought was perhaps
too obscure, but now I've realized it would integrate well with it.  That
is to have a value-returning form of try... except...
But perhaps the two of them together are too much of a change.

John

On Mon, Oct 25, 2021 at 10:03 PM Mark Gordon  wrote:

> What should happen if the context manager attempts to suppress a raised
> exception? In cases where you applied the context manager to an entire
> line, e.g.
>
> data = fail() with contextlib.suppress(Exception)
>
> Then it would make sense to treat it like
>
> with contextlib.suppress(Exception):
> data = fail()
>
> Where `data` remains unassigned after the block executes assuming `fail`
> raises an exception. However, with the initial proposal you run into
> trouble when you apply this to sub-expressions that are expected to
> themselves have a value. For example, what should happen here?
>
> more_work(fail() with contextlib.suppress(Exception))
>
> We have no value to pass as an argument to `more_work` so there's no way
> we can call it. Yet it would be odd to not call it if there's no exception
> being raised since it exists outside of any context manager itself.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/Y7WZDD2AFGUX3ND2OX3EUN2VUK27O4E5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DAY57GUMKNHWKJYAAQWFIDVVD4AU6TDL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 proof-of-concept: A less invasive implementation

2021-10-30 Thread Eric V. Smith


On 10/30/2021 10:40 AM, Chris Angelico wrote:

And, seeing something in help(fn) largely necessitates that the source
code be retained. I don't know of any other way to do it. If you say
that the default argument is "len(a)", then that's what help() should
say.


"from __future__ import annotations" works by decompiling the AST into a 
string. From PEP 563: "The string form is obtained from the AST during 
the compilation step, which means that the string form might not 
preserve the exact formatting of the source."


I'm not saying this is a better way to do it, but it is another way. And 
it might save some memory. I don't think the slight differences 
mentioned at the end of that sentence would make a difference in practice.


Eric

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3YLTC4EB3DJKANQMZCDD5HFTVICYXFZM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 proof-of-concept: A less invasive implementation

2021-10-30 Thread Jonathan Fine

Hi Chris

I like your questions. You ask: How would fn.__wibble__ be different from
checks at the top of fn.__code__?

They'd be in two different code blocks. I see a function call going as
follows.
1. Process the supplied arguments in the usual way.
2. Create a new frame object and place it on the stack.
3. In that new frame execute fn.__wibble.
4. When fn.__wibble__ is done, execute fn.__code__ IN THE SAME FRAME.

I think step 4 is a tail call, as in https://en.wikipedia.org/wiki/Tail_call,
which includes the concept of tail recursion.

Your other question is: And, seeing something in help(fn) largely
necessitates that the source code be retained.

Yes, this is true whatever syntax is used. In help(fn) inspect.signature
repr() is used to produce help text. There's no extra storage overhead for
that. Both your implementation and mine will require source text to be
stored (unless the module is compiled as optimised).

Oh, but I've made a mistake. If the module is compiled non-optimised then
the compile code contains points to the source file. These are used in
traceback when an exception occurs.

I'm not to say at this point which approach is best for the person who
reads help(fn), except the lawyer's answer "it just depends". At this point
my focus is on designing a less invasive implementation.

Your good questions have led me to rethink. The tail call in my proposed
implementation can be removed and then fn.__wibble__ would not be needed.
It would be the same as checks at the top of fn.__code__.

But instead of fn.__wibble__ we have a pointer (as in fn.__code__) to some
location in the body of fn. (Or  as fn.__code__ is already well equipped
with pointers, we equip fn with a pointer to one of these pointers.)

So all that's required now is
1. A syntax in source files that allows the author of fn to specify the end
of the 'preamble extra help' in the body of fn.
2. An addition to help(fn) that provides the 'preamble' of fn as an extra
help message.

with kind regards

Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TKV7LIEXBICK3SHMLENYY5YX3IAMVODG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 proof-of-concept: A less invasive implementation

On Sat, Oct 30, 2021 at 10:32 PM Jonathan Fine  wrote:
> I suggest that an implementation which provides additional flexibility in the 
> manner in which the code frame is initialised would be less invasive. 
> Necessarily, PEP 671 allows programmer supplied code to be used in the 
> 'initialisation phase'. The previous attempts place that code in fn.__code__.
>
> I suggest that the implementation adds a new attribute fn.__wibble__ to fn, 
> which can either be None or a code object. And if fn.__wibble__ is not None, 
> then it is used to initialise the code frame in which fn.__code__ executes. 
> It would as before take as input the arguments actually supplied by the user.
>
> I stop here, saying nothing for now about two important questions. First, 
> what is the programmer syntax for creating such a 'two-part' function fn. 
> Second, what does the user see as a result of help(fn). Or in other words, 
> how to extend the semantics of inspect.signature.
>

The code has to be executed in the context of the called function. How
would fn.__wibble__ be different from checks at the top of
fn.__code__?

And, seeing something in help(fn) largely necessitates that the source
code be retained. I don't know of any other way to do it. If you say
that the default argument is "len(a)", then that's what help() should
say.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/X42RZ57LC44QV6VAFWTX6DEIVXCY6XR3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Take two -- PEP 671 proof-of-concept implementation

2021-10-30 Thread Jeremiah Vivian

Side note: I've added operators and even a whole new builtin object for fun.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PMRNEIXR3S6E6XITIVCTNPMTD72RIICW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Take two -- PEP 671 proof-of-concept implementation

2021-10-30 Thread Jeremiah Vivian

> Still interested in coauthors who know CPython internals. That hasn't changed.
I know some part of CPython internals, but I'm not quite sure how would a 
coauthor be recruited. I'm studying your implementation to understand more 
about how parsing to executing bytecode works in Python.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JBOQ67MECUT3FPNKAUGCQCVN55PWWYYF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] PEP 671 proof-of-concept: A less invasive implementation

2021-10-30 Thread Jonathan Fine

Hi

One of the motives for PEP 671 is that the signature of a function fn, and
hence the associated help(fn) is sometimes opaque regarding default values.
I won't repeat the excellent examples already given.

In the current implementation default values are handled outside the
compiled code of the function, which is available at fn.__code__. Instead
they are stored in 'metadata' associated with the function.

Here's one way to see this.

from inspect import signature as sig
def fn(a, b=1, c=2): return a, b, c
sig(fn) # Gives 
fn.__defaults__ = ('hi', 'there')
sig(fn) # Gives 

We can also change the __code__ object, but care is needed here.

def gn(): return (1, 2, 3)
fn.__code__ = gn.__code__
fn() # Gives (1, 2, 3).
sig(fn) # Gives 
fn.__defaults__  # Gives ('hi', 'there')

The signature of fn, together with the arguments actually supplied, is used
to initialise the code frame which is put on the top of the stack, and in
which fn.__code__ executes.

I suggest that an implementation which provides additional flexibility in
the manner in which the code frame is initialised would be less invasive.
Necessarily, PEP 671 allows programmer supplied code to be used in the
'initialisation phase'. The previous attempts place that code in
fn.__code__.

I suggest that the implementation adds a new attribute fn.__wibble__ to fn,
which can either be None or a code object. And if fn.__wibble__ is not
None, then it is used to initialise the code frame in which fn.__code__
executes. It would as before take as input the arguments actually supplied
by the user.

I stop here, saying nothing for now about two important questions. First,
what is the programmer syntax for creating such a 'two-part' function fn.
Second, what does the user see as a result of help(fn). Or in other words,
how to extend the semantics of inspect.signature.

with kind regards

Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/H6NXL3WWHNMD3I32YPYZC32IHXEQWHTT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Take two -- PEP 671 proof-of-concept implementation

2021-10-30 Thread Abdur-Rahmaan Janhangeer

> Still interested in coauthors who know CPython internals. That hasn't
changed.

Should have been included since first mail. But this peaked my interest to
dive into
the C source

Kind Regards,

Abdur-Rahmaan Janhangeer
about  | blog

github 
Mauritius
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CY5OWXHOPNSBTEGLN4YSJIAIO2LRQQ26/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Take two -- PEP 671 proof-of-concept implementation