[Python-ideas] PEP 468

[guido]

Gesendet von Mail für Windows 10

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Coming up with an alternative to PEP 505's None-aware operators

[Nick Coghlan]

On 17 February 2018 at 02:31, Ethan Furman  wrote:
> On 02/15/2018 11:55 PM, Nick Coghlan wrote:
>> However, while I think that looks nicer in general, we'd still have to
>> choose between two surprising behaviours:
>>
>> * implicitly delete the statement locals after the statement where
>> they're set (which still overwrites any values previously bound to
>> those names, similar to what happens with exception clauses)
>
>
> If we're overwriting locals anyway, don't delete it.  The good reason for
> unsetting an exception variable doesn't apply here.
>
>> * skip deleting, which means references to subexpressions may last
>> longer than expected (and we'd have the problem where embedded
>> assignments could overwrite existing local variables)
>
> Odds are good that we'll want/need that assignment even after the immediate
> expression it's used in.  Let it stick around.

If we want to use a subexpression in multiple statements, then regular
assignment statements already work fine - breaking out a separate
variable assignment only feels like an inconvenience when we use a
subexpression twice in a single statement, and then don't need it any
further.

By contrast, if we have an implicit del immediately after the
statement for any statement local variables, then naming a
subexpression only extends its life to the end of the statement, not
to the end of the current function, and it's semantically explicit
that you *can't* use statement locals to name subexpressions that
*aren't* statement local.

The other concern I have with any form of statement local variables
that can overwrite regular locals is that we'd be reintroducing the
problem that comprehensions have in Python 2.x: unexpectedly rebinding
things in non-obvious ways. At least with an implicit "del" the error
would be more readily apparent, and if we disallow closing over
statement local variables (which would be reasonable, since closures
aren't statement local either), then we can avoid interfering with
regular locals without needing to introduce a new execution scope.

So let's consider a spec for statement local variable semantics that
looks like this:

1. Statement local variables do *not* appear in locals()
2. Statement local variables are *not* visible in nested scopes (of any kind)
3. Statement local variables in compound statement headers are visible
in the body of that compound statement
4. Due to 3, statement local variable references will be syntactically
distinct from regular local variable references
5. Nested uses of the same statement local variable name will shadow
outer uses, rather than overwriting them

The most discreet syntactic marker we have available is a single
leading dot, which would allow the following (note that in the simple
assignment cases, breaking out a preceding assignment would be easy,
but the perk of the statement local spelling is that it works in *any*
expression context):

value = .s.strip()[4:].upper() if (var1 as .s) is not None else None

value = .s[4:].upper() if (var1 as .s) is not None else None

value = .v if (var1 as .v) is not None else .v if (var2 as .v)
is not None else var3

value = .v if not math.isnan((var1 as .v)) else tmp if not
math.isnan((var2 as .v)) else var3

value = .f() if (calculate as .f) is not None else default

filtered_values = [.v for x in keys if (get_value(x) as .v) is not None]

range((calculate_start() as .start), .start+10)
data[(calculate_start() as .start):.start+10]

value if (lower_bound() as .min_val) <= value < .min_val+tolerance else 0

print(f"{(get_value() as .v)!r} is printed in pure ASCII as
{.v!a} and in Unicode as {.v}")

if (pattern.search(data) as .m) is not None:
# .m is available here as the match result
else:
# .m is also available here (but will always be None given the
condition)
# .m is no longer available here

Except clauses would be updated to allow the "except ExceptionType as
.exc" spelling, which would give full statement local semantics (i.e.
disallow closing over the variable, hide it from locals), rather than
just deleting it at the end of the clause execution.

Similarly, with statements would allow "with cm as .enter_result" to
request statement local semantics for the enter result. (One potential
concern here would be the not-immediately-obvious semantic difference
between "with (cm as .the_cm):" and "with cm as .enter_result:").

To make that work at an implementation level we'd then need to track
the following in the compiler:

* the current nested statement level in the current compilation (so we
can generate distinct names at each level)
* a per-statement set of local variable names (so we can clear them at
the end of the statement)
* the largest number of concurrently set statement local variables (so
we can allocate space for them in the frame)
* the storage offset to use for each statement local variable

and then frames would need an additional 

Re: [Python-ideas] Temporary variables in comprehensions

[Steven D'Aprano]

On Sun, Feb 18, 2018 at 12:23:28AM +0800, fhsxfhsx wrote:
> Thank you Paul, what you said is enlightening and I agree on most part of it.
> 
> I'll propose two candidate syntaxs.
> 1. `with ... as ...`
> This syntax is more paralles as there would be `for` and `with` 
> clause as well as `for` and `with` statement. However, the 
> existing `with` statement is semantically different from this one, 
> although similar.

I don't think they are even a little bit similar. The existing `with` 
statement is for controlling cleanup code (using a context manager). 
This proposal doesn't have anything to do with context managers or 
cleanup code. It's just a different way to spell "name = value" inside 
comprehensions.


> 2. `for ... is ...`
> This syntax is more uniform as the existing `for` clause make an 
> iterator which is a special kind of variable. However, I'm afraid 
> this syntax might be confused with `for ... in ...` as they differ 
> only on one letter.

Indeed.

And frankly, treated as English grammar, "for value is name" doesn't 
make sense and is horribly ugly to my eyes:

result = [x for value in sequence for value+1 is x]


> And here is an example which appears quite often in my code where I 
> think a new syntax can help a lot: Suppose I have an list of goods 
> showing by their ids in database, and I need to transform the ids into 
> json including information from two tables, `Goods` and 
> `GoodsCategory`, where the first table recording `id`, `name` and 
> `category_id` indicating which category the goods belongs to, the 
> second table recording `id`, `name` and `type` to the categories.
>
> With the new syntax, I can write
> [
>   {
> 'id': goods.id,
> 'name': goods.name,
> 'category': gc.name,
> 'category_type': gc.type,
>   }
>   for goods_id in goods_id_list
>   for goods is Goods.get_by_id(goods_id)
>   for gc is GoodsCategory.get_by_id(goods.category_id)
> ]
> 
> And I cannot think of any good solutions as this one without it.

I can of a few, starting with the most simple: write a helper function. 
Not every problem needs to be solved with new syntax.

def dict_from_id(goods_id):
goods = Goods.get_by_id(goods_id)
gc = GoodsCategory.get_by_id(goods.category_id)
return {'id': goods.id,
'name': goods.name,
'category': gc.name,
'category_type': gc.type
}

result = [dict_from_id(goods_id) for goods_id in goods_id_list]

That's much nicer to read, you can document and test the dict_from_id() 
function, no new systax is required, it is easy to refactor, and I very 
much doubt that adding one extra function call is going to be a 
significant slowdown compared to the cost of two calls to get_by_id() 
methods and constructing a dict.

(And if as you add more fields to the dict, the overhead of the function 
call becomes an even smaller proportion.)

Or you can make this a method of the goods object, which is arguably a 
better OO design. Let the goods object be responsible for creating the 
dict.

result = [Goods.get_by_id(goods_id).make_dict() for goods_id in goods_id_list]
# or if you prefer
result = [goods.make_dict() for goods in map(Goods.get_by_id, goods_id_list)]

Here is a third solution: use a for-loop iterating over a single-item 
tuple to get the effect of a local assignment to a temporary variable:

result = [{ 
   # dict display truncated for brevity...
   } 
   for goods_id in goods_id_list
   for goods in (Goods.get_by_id(goods_id),)
   for gc in (GoodsCategory.get_by_id(goods.category_id),)
   ]

If you don't like the look of single-item tuples (foo,) you can use 
single-item lists instead [x] but they are a tiny bit slower to create.

Serhiy has suggested that the interpreter can optimize the single-item 
loop to make it as fast as a bare assignment:

https://bugs.python.org/issue32856

I think this is a neat trick, although Yuri thinks it is an ugly hack 
and doesn't want to encourage it. Neat or ugly, I think it is better 
than "for value is name".


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[fhsxfhsx]

Thanks so much for the comments and the collect on this syntax!

Comments on *previous talk*
The list also mentioned some other previous proposals, so I myself search it 
and find that there're even more in the mailing list since 2008.
https://mail.python.org/pipermail/python-ideas/2008-August/001842.html

Comments on *previous talk: PEP*
The PEP seems to be about an explicit temporary namespace where any objects 
(including functions, classes, etc.) could be held.
However, I find that this syntax may not work for temporary variables in 
comprehensions.

We might expect this syntax work like
[?.y+2 for x in range(5) given: y = x+1]
But notice that the proposed `given` statement here appeared in comprehensions, 
where syntax changes in comprehensions are needed, and nothing about this is 
mentioned in the PEP.
What's more, the proposed syntax is a statement like `for` statement, `if` 
statement, rather than a for-clause or if-clause in comprehensions. In my 
opinion, it's not a good idea to have a statement in comprehensions. Instead, 
another given-clause should be added if one wants to write code like above, 
doing assignments in comprehensions, which can look like:
[?.y+2 for x in range(5) given y = x+1]
So the ? symbol seems useless here, so maybe
[y+2 for x in range(5) given y = x+1]
make it quite similar to the `where` syntax you proposed.

So, I agree with you that it is a good idea to have a new PEP, though for a 
different reason.
 
Comments on *Choice of syntax*
I gave two candidate syntaxs in 
https://mail.python.org/pipermail/python-ideas/2008-August/001842.html, one 
said `for ... in ...`, another said `with ... as ...`.
The first has the same problem as `for ... = ...` you proposed. And the biggest 
problem I think the second will face is the difference in semantic between the 
`with` statement and it.
When it comes to `where ... = ...`, there are one possible problem I can think 
of.
`where` is not now a keyword in Python. There are WHERE clauses in SQL, so in 
many modules including peewee and SQLAlchemy, `where` is an important method. 
The new syntax would cause quite incompatibilities.

Personally I agreed with you that postfix notation would have advantage over 
prefix notation. Other than `where`, `with` is quite readable in my opinion. So 
maybe `with ... = ...` can be another candidate?

Plus the `given ... = ...` I mentioned above, there are several more candidates 
now. Personally I perfer `with ... = ...`, because `with` is a python keyword 
so it would be good for backward compatibility.

*About comprehensions and expressions*
You gave `print(y+2 where y = x+1)` as an example, I think it should be 
clarified that it's not, or at least, does not look like a comprehension, but 
an expression. It should give an object `y+2` rather than a list or a 
generator. (otherwise what list can it give?)
There are for-clause and if-clause for comprehensions, and if-clause (aka 
ternary operators) for expressions. So, In my opinion, there should be 
additional discuss and examples to show that it's helpful to have such syntax.

For the where-clause in expressions, I think we could refer to how python 
handles if-clause.
The following setences are legal:
[x if x else 0 for x in mylist if x > 10]
The following illegal:
[x if x for x in mylist if x > 10]
[x if x else 0 for x in mylist if x > 10 else 10]
That's to say, the two kinds of if-clause, one is only used in expressions (aka 
`ternary operator`), the other is only used in comprehensions. They slightly 
differ in syntax.

The where-clause might work in similar ways. Then
[z+2 where z = y+1 for x in mylist where y = x+1]
means
[(z+2 where z=y+1) for x in mylist where y = x+1]
where the parenthesis (also the part before the first `for`) is a expression, 
the rest is comprehension clauses.
To be more accurate, the new syntax would be:

test: where_test ['if' where_test 'else' test] | lambdef
where_test: or_test | ( '(' or_test 'where' NAME '=' testlist_star_expr ')' )

Mandatory parenthesis in where_test is to resolve the ambiguity in code like
print(y+2 where y=x+1 if x>0 else x-1 if x>1 else 0)
It could be analysed like
print((y+2 where y=x+1 if x>0 else x-1) if x>1 else 0)
or
print(y+2 where y=(x+1 if x>0 else x-1 if x>1 else 0)).
I guess thektulu may have mandatory parenthesis for the same reason.

I haven't check the new syntax very carefully so there might be other 
ambiguities.

Another example is
print(y+2 if x>0 else y-2 where y=x+1)
with mandatory parenthesis, one must write
print((y+2 if x>0 else y-2 where y=x+1))
or
print(y+2 if x>0 else (y-2 where y=x+1))

However, it might still confuse many people. I wonder whether it's a good idea 
to have such syntax.

It would be much easier to add assignments in comprehensions.
comp_iter: comp_for | comp_if | comp_where
comp_where: 'where' NAME '=' testlist_star_expr [comp_iter]

Comments on *Goals of the new syntax*
I have a real-world example in