[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[miss-islington]

miss-islington  added the comment:


New changeset 9ecbe3321f7bb3726017a053e583ca507d4453fc by Miss Islington (bot) 
in branch '3.7':
Revert "closes bpo-27494: Fix 2to3 handling of trailing comma after a generator 
expression (GH-3771)" (GH-8241)
https://github.com/python/cpython/commit/9ecbe3321f7bb3726017a053e583ca507d4453fc


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[miss-islington]

Change by miss-islington :


--
pull_requests: +8089

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 4b8a7f51da224d1a0ad8159935f78ba4e6e16037 by Serhiy Storchaka in 
branch 'master':
 Revert "closes bpo-27494: Fix 2to3 handling of trailing comma after a 
generator expression (GH-3771)" (#8241)
https://github.com/python/cpython/commit/4b8a7f51da224d1a0ad8159935f78ba4e6e16037


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Nick Coghlan]

Nick Coghlan  added the comment:

With issue 32023 (base class lists) and 32034 (fixing the documentation for 
decorator factory function calls) covering the other refinements, this 
particular issue is done now.

The most recent PR is the one for issue 32023.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4348

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 9165f77d5f93a2c12aa0e90853e3ae7212800d3c by Serhiy Storchaka in 
branch 'master':
bpo-32012: Disallow trailing comma after genexpr without parenthesis. (#4382)
https://github.com/python/cpython/commit/9165f77d5f93a2c12aa0e90853e3ae7212800d3c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Guido van Rossum]

Guido van Rossum  added the comment:

It's a (small) mistake that we didn't make the syntax for argument lists in 
decorators the same as argument lists everywhere else, and that should be fixed 
to allow exactly what's allowed in regular calls. (That syntax is weird because 
we don't want e.g. `@foo().bar` but we do want e.g. `@foo.bar()`.)

I am honestly not sure that we should change anything here, since the meaning 
is not actually ambiguous: the syntax for generator expressions doesn't allow 
e.g. `x for x in 1, 2, 3` -- you have to write `x for x in (1, 2, 3)`. (A 
regular for-loop *does* allow this, but there the context makes it unambiguous 
-- that's why genexprs are different.)

But I'm fine with changing it, as long as we do it consistently.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Yury Selivanov]

Change by Yury Selivanov :


--
nosy:  -yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Stefan Krah]

Change by Stefan Krah :


--
nosy:  -skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Stefan Krah]

Stefan Krah  added the comment:

Yes Sir!

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Stefan, `[1,2,3]` is an expression, but `x for x in [1,2,3]` is not. If you 
want to change the Python language specification, please open a topic on 
Python-Dev and provide the rationale.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Stefan Krah]

Change by Stefan Krah :


--
nosy:  -skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I would prefer to fix all related cases in one issue, for having all examples 
in one place and having only one reference. All this cases are caused by the 
limitation of the parser used in CPython, and using different grammar rules. 
This If you want to change the language specification for decorator expression 
and class definition, it should be discussed before merging PR 4382, and I 
would make corresponding changes in it. In any case it is harder to fix 
issue32023 without fixing the original issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Stefan Krah]

Stefan Krah  added the comment:

On Tue, Nov 14, 2017 at 01:31:52PM +, Nick Coghlan wrote:
> If limited to the original scope, this isn't a new special case, it's fixing 
> a bug in the implementation of the existing special case (where it's ignoring 
> the trailing comma when it shouldn't be).

This ignores the trailing comma:

f([1,2,3],)

And this:

f(x for x in [1,2,3],)

Seems logical to me.

Do you want to allow the 1,2 to be read as a tuple?

   f(x for x in 1,2)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Nick Coghlan]

Nick Coghlan  added the comment:

If limited to the original scope, this isn't a new special case, it's fixing a 
bug in the implementation of the existing special case (where it's ignoring the 
trailing comma when it shouldn't be).

If it hadn't been for the scope creep to include a couple of other cases where 
the implementation is arguably more permissive than the language spec says it 
should, it would have already been merged.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Stefan Krah]

Stefan Krah  added the comment:

I would prefer to do nothing about the subject of this issue. I still
don't see any ambiguity, except in a very broad colloquial sense.

Why introduce another special case?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Nick Coghlan]

Nick Coghlan  added the comment:

OK, I've filed https://bugs.python.org/issue32024 to cover the decorator syntax 
discrepancy.

So I'd still prefer to restrict the patch for *this* issue to just the 
genuinely ambiguous case, and leave the 
unambiguous-but-inconsistent-with-the-language-spec cases to their respective 
issues.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Stefan Krah]

Stefan Krah  added the comment:

I think "ambiguous" is not the right word. If a single argument
can be a non-parenthesized generator and all arguments can be
followed by a trailing comma, it's clear.

The language spec is often behind in my experience.

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Henk-Jaap Wagenaar]

Henk-Jaap Wagenaar  added the comment:

I think this showcases how difficult it is to get this right, requires 
carefully reading the EBNF language spec, not just the text, and the behaviour 
is unexpected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

No, it doesn't match the "@dotted_name(arg_list)" pattern.

   decorator: "@" `dotted_name` ["(" [`argument_list` [","]] ")"] NEWLINE
   call: `primary` "(" [`argument_list` [","] | `comprehension`] ")"
   argument_list: `positional_arguments` ["," `starred_and_keywords`]
:   ["," `keywords_arguments`]
: | `starred_and_keywords` ["," `keywords_arguments`]
: | `keywords_arguments`

The call syntax contains a special case for generator expression. The decorator 
expression syntax dosn't contain it. You should change the grammar rule to

   decorator: "@" `dotted_name` ["(" [`argument_list` [","] | `comprehension`] 
")"] NEWLINE

for supporting this syntax. Please open a separate topic on Python-Dev for 
discussing this language change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Nick Coghlan]

Nick Coghlan  added the comment:

I created https://bugs.python.org/issue32023 to explicitly cover the base class 
list case, and after checking the language spec, I agree that case should be a 
syntax error.

However, `@deco(x for x in [])` should *not* be a syntax error, as:

* it's a call with one argument, so the genexp parentheses can be omitted as 
described in 
https://docs.python.org/3/reference/expressions.html#generator-expressions
* it matches the "@dotted_name(arg_list)" pattern permitted by 
https://docs.python.org/3/reference/compound_stmts.html#function-definitions

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The problem with these constructions is that they are not allowed by the Python 
language specification. It should be explicitly changed for allowing them. And 
this change should be accepted by Guido.

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Nick Coghlan]

Nick Coghlan  added the comment:

In a function call, `f(x for x in iterable)` is roughly equivalent to 
`f(iter(iterable))`, not `f(*iterable)` (the genexp based equivalent of the 
latter would be ``f(*(x for x in iterable))`).

Thus the base class list is no different from any other argument list in this 
case - it's just that generator objects aren't valid base classes.

Getting back on topic for this particular bug fix though: as noted in my last 
PR review, I think the latest version goes too far by disallowing `@deco(x for 
x in iterable)` and `class C(x for x in iterable):`. While semantically 
questionable, there's nothing *syntactically* invalid about those - they pass a 
single generator expression, and that generator expression is correctly 
surrounded by parentheses. There's no more reason to prohibit a genexp in 
either of those situations at compile time than there is to prohibit a list 
comprehension.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I think this issue is not the best way for answering your question, but I will 
make a try.

The fact that "class C(x for x in [object]): ..." does not cause a syntax error 
is a bug. This issue fixes it. The fact that corrected "class C((x for x in 
[object])): ..." doesn't work is expected, because a generator instance is not 
a class.

The equivalence between a decorator expression and explicit calling a decorator 
function is true only in one direction and only for valid Python syntax. Saying 
about equivalence of syntactically incorrect Python code doesn't make sense.

Yes, an inheritance list can contain keyword arguments. They are passed to a 
metaclass constructor as well as positional arguments.

The syntaxes of the for statement and comprehensions are different.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Henk-Jaap Wagenaar]

Henk-Jaap Wagenaar  added the comment:

[As a follow-on, should I open a new issue/discuss on python-dev? Willing to 
help out with a solution on way or another! I know 
https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence, "In my head" <> 
"Should be the case" etc. very much applies.]

In my head

@...
def foo(): pass

is equivalent to

def _foo(): pass
foo = ...()
del _foo

However the following shows this is not the case:

@0
def foo(): pass

throws a syntax error, whereas

def _foo(): pass
foo = 0(_foo)

throws a type error. This might seem silly, but it is still unexpected.

https://docs.python.org/3/reference/compound_stmts.html#grammar-token-decorator 
has

decorator ::= "@" dotted_name ["(" [argument_list [","]] ")"] NEWLINE

which in my head is

decorator ::= "@" atom NEWLINE

Similarly for classes: 
https://docs.python.org/3/reference/compound_stmts.html#class-definitions

inheritance ::= "(" [argument_list] ")"

which allows for keyword arguments (does that make any sense!?). In my head it 
is (compare with call: 
https://docs.python.org/3/reference/expressions.html#calls)

inheritance ::= "(" [positional_arguments [","] | comprehension] ")"

[Tangentially related, this is how I originally got onto the mailing lists, my 
unhappiness with the definition of the for statement 
(https://docs.python.org/3/reference/compound_stmts.html#the-for-statement):

for_stmt ::= "for" target_list "in" expression_list ":" suite ["else" ":" 
suite]

Which I would expect to be:

for_stmt ::= comp_for ":" suite ["else" ":" suite]

so you could e.g. have if statements.
]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Henk-Jaap Wagenaar]

Henk-Jaap Wagenaar  added the comment:

Currently,

Class C(*some_classes): ...

works 'as expected' and is within grammar and language specification whereas

Class C(x for x in [object]): ...

does not work but does not cause a syntax error. I can see a use case for both 
in dynamic class factories. I was going to do this, but was thwarted by another 
issue (__doc__ cannot be assigned after creation, nor can it be defined as 
anything but a pure string, any work around or reason that is the case? Not 
true for e.g. functions).

I think having one of these within the language specification and not the other 
is odd.

--
nosy: +Henk-Jaap Wagenaar

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It is easy to forbid the above cases, but I don't know what error message is 
appropriate. General "invalid syntax"?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 4382 doesn't change the grammar, it changes only checks in the CST to AST 
transformer. Maybe it would be better to change the grammar. Currently it 
doesn't match the language specification and allows the following constructions:

@deco(x for x in [1])
def f(): ...

class C(x for x in [1]): ...

And I think the part of issue27494 should be reverted.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +4330
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32012] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

The syntax f(x for x in [1],) is ambiguous and reasons that allow it (omitting 
parenthesis in generator expression and using trailing comma in call 
expression) are not applicable in this case. Rationales see on Python-Dev: 
https://mail.python.org/pipermail/python-dev/2017-November/150481.html.

--
components: Interpreter Core
messages: 306132
nosy: benjamin.peterson, brett.cannon, ncoghlan, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Disallow ambiguous syntax f(x for x in [1],)
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com