Re: [Python-Dev] problem with genexp
On 10/10/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > There's a problem with genexp's that I think really needs to get > fixed. See http://python.org/sf/1167751 the details are below. This > code: > > >>> foo(a = i for i in range(10)) > > I agree with the bug report that the code should either raise a > SyntaxError or do the right thing. The change to Grammar/Grammar below seems to fix the problem and all the tests pass. Can anyone comment on whether this fix is correct/appropriate? Is there a better way to fix the problem? -argument: [test '='] test [gen_for] # Really [keyword '='] test +argument: test [gen_for] | test '=' test ['(' gen_for ')'] # Really [keyword '='] test n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
On 10/16/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > On 10/10/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > > There's a problem with genexp's that I think really needs to get > > fixed. See http://python.org/sf/1167751 the details are below. This > > code: > > > > >>> foo(a = i for i in range(10)) > > > > I agree with the bug report that the code should either raise a > > SyntaxError or do the right thing. > > The change to Grammar/Grammar below seems to fix the problem and all > the tests pass. Can anyone comment on whether this fix is > correct/appropriate? Is there a better way to fix the problem? > > -argument: [test '='] test [gen_for] # Really [keyword '='] test > +argument: test [gen_for] | test '=' test ['(' gen_for ')'] # Really > [keyword '='] test The other option would be changes in the Python/compile.c (somewhat) like following diff -r2.352 compile.c 6356c6356,6362 - if (TYPE(n) == argument && NCH(n) == 3) { --- + if (TYPE(n) == argument && NCH(n) == 4) { + PyErr_SetString(PyExc_SyntaxError, + "invalid syntax"); + symtable_error(st, n->n_lineno); + return; + } + else if (TYPE(n) == argument && NCH(n) == 3) { but IMO, changing the Grammar looks more obvious. > > n > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/seojiwon%40gmail.com > > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
On 10/16/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > On 10/10/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > > There's a problem with genexp's that I think really needs to get > > fixed. See http://python.org/sf/1167751 the details are below. This > > code: > > > > >>> foo(a = i for i in range(10)) > > > > I agree with the bug report that the code should either raise a > > SyntaxError or do the right thing. > > The change to Grammar/Grammar below seems to fix the problem and all > the tests pass. Can anyone comment on whether this fix is > correct/appropriate? Is there a better way to fix the problem? Since no one responded other than Jiwon, I checked in this change. I did *not* backport it since what was syntactically correct in 2.4.2 would raise an error in 2.4.3. I'm not sure which is worse. I'll leave it up to Anthony whether this should be backported. BTW, the change was the same regardless of old code vs. new AST code. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
Regarding this Grammar change; (last October) from argument: [test '=' ] test [gen_for] to argument: test [gen_for] | test '=' test ['(' gen_for ')'] - to raise error for "bar(a = i for i in range(10)) )" I think we should change it to argument: test [gen_for] | test '=' test instead of argument: test [gen_for] | test '=' test ['(' gen_for ')'] that is, without ['(' gen_for ')'] . We don't need that extra term, because "test" itself includes generator expressions - with all those parensises. Actually with that extra ['(' gen_for ')'] , foo(a= 10 (for y in 'a')) is grammartically correct ; although that error seems to be checked elsewhere. I tested without ['(' gen_for ')'] , and worked fine passing Lib/test/test_genexps.py -Jiwon On 10/20/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > On 10/16/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > > On 10/10/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > > > There's a problem with genexp's that I think really needs to get > > > fixed. See http://python.org/sf/1167751 the details are below. This > > > code: > > > > > > >>> foo(a = i for i in range(10)) > > > > > > I agree with the bug report that the code should either raise a > > > SyntaxError or do the right thing. > > > > The change to Grammar/Grammar below seems to fix the problem and all > > the tests pass. Can anyone comment on whether this fix is > > correct/appropriate? Is there a better way to fix the problem? > > Since no one responded other than Jiwon, I checked in this change. I > did *not* backport it since what was syntactically correct in 2.4.2 > would raise an error in 2.4.3. I'm not sure which is worse. I'll > leave it up to Anthony whether this should be backported. > > BTW, the change was the same regardless of old code vs. new AST code. > > n > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/seojiwon%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
On 2/20/06, Jiwon Seo <[EMAIL PROTECTED]> wrote: > Regarding this Grammar change; (last October) > from argument: [test '=' ] test [gen_for] > to argument: test [gen_for] | test '=' test ['(' gen_for ')'] > > - to raise error for "bar(a = i for i in range(10)) )" > > I think we should change it to > argument: test [gen_for] | test '=' test > > instead of > argument: test [gen_for] | test '=' test ['(' gen_for ')'] > > that is, without ['(' gen_for ')'] . We don't need that extra term, > because "test" itself includes generator expressions - with all those > parensises. Works for me, committed. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
On 10/10/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > There's a problem with genexp's that I think really needs to get > fixed. See http://python.org/sf/1167751 the details are below. This > code: > > >>> foo(a = i for i in range(10)) > > generates "NameError: name 'i' is not defined" when run because: [SNIP] > If you add parens around the code: foo(a = i for i in range(10)) > You get something quite different: Do you mean having ``(foo(a = i for i in range(10))``? Otherwise I see no difference when compared to the first value. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
On 10/10/05, Brett Cannon <[EMAIL PROTECTED]> wrote: > On 10/10/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > > There's a problem with genexp's that I think really needs to get > > fixed. See http://python.org/sf/1167751 the details are below. This > > code: > > > > >>> foo(a = i for i in range(10)) > > > > generates "NameError: name 'i' is not defined" when run because: > [SNIP] > > If you add parens around the code: foo(a = i for i in range(10)) > > You get something quite different: > > Do you mean having ``(foo(a = i for i in range(10))``? Otherwise I > see no difference when compared to the first value. Sorry, I think I put it in the bug report, but forgot to add it here: >>> foo(a = (i for i in range(10))) n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
Neal Norwitz wrote: > There's a problem with genexp's that I think really needs to get > fixed. See http://python.org/sf/1167751 the details are below. This > code: > I agree with the bug report that the code should either raise a > SyntaxError or do the right thing. I agree it should be a SyntaxError - I believe the AST compiler actually raises one in this situation. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
On 10/11/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Neal Norwitz wrote: > > There's a problem with genexp's that I think really needs to get > > fixed. See http://python.org/sf/1167751 the details are below. This > > code: > > I agree with the bug report that the code should either raise a > > SyntaxError or do the right thing. > > I agree it should be a SyntaxError - I believe the AST compiler actually > raises one in this situation. Could someone add a test for this on the AST branch? BTW, it looks like doctest is the way to go for SyntaxError tests. There are older tests, like test_scope.py, that use separate files with bad syntax (and lots of extra kludges in the infrastructure to ignore the fact that those .py files can't be compiled). Jeremy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
Nick Coghlan wrote: > Neal Norwitz wrote: > >>There's a problem with genexp's that I think really needs to get >>fixed. See http://python.org/sf/1167751 the details are below. This >>code: >>I agree with the bug report that the code should either raise a >>SyntaxError or do the right thing. > > > I agree it should be a SyntaxError - I believe the AST compiler actually > raises one in this situation. I was half right. Both the normal compiler and the AST compiler give a SyntaxError if you write: foo((a=i for i in range(10))) The problem is definitely on the parser end though: Py> compiler.parse("foo(x=i for i in range(10))") Module(None, Stmt([Discard(CallFunc(Name('foo'), [Keyword('x', Name('i'))], None, None))])) It's getting to what looks like a valid keyword argument in "x=i" and throwing the rest of it away, when it should be flagging a syntax error (the parser's limited lookahead should still be enough to spot the erroneous 'for' keyword and bail out). The error will be even more obscure if there is an "i" visible from the location of the function call. Whereas when it's parenthesised correctly, the parse tree looks more like this: Py> compiler.parse("foo(x=(i for i in range(10)))") Module(None, Stmt([Discard(CallFunc(Name('foo'), [Keyword('x', GenExpr(GenExprInner(Name('i'), [GenExprFor(AssName('i', 'OP_ASSIGN'), CallFunc(Name('range'), [Const(10)], None, None), [])])))], None, None))])) Cheers, Nick. P.S. I added a comment showing the parser output to the SF bug report. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com