[issue19335] codeop misclassifies incomplete code with 'nonlocal'
Lysandros Nikolaou added the comment: That's right, but the SyntaxError message is somewhat confusing. At least the fact that the SyntaxError message is some for both of def a(): def b(): nonlocal x and def a(): nonlocal x seems a bit misleading. No? -- ___ Python tracker <https://bugs.python.org/issue19335> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19335] codeop misclassifies incomplete code with 'nonlocal'
Lysandros Nikolaou added the comment: I feel that I wasn't clear at all in my previous responses, so let me try to have another go at explaining how I see this: We have two distinct problems, that are totally unrelated to one another: 1) nonlocal should raise when it's not in a nested function (question for the more experienced here: is this the *only* place where nonlocal is allowed?) 2) comc should return None, signifying an incomplete input, when presented with the bloack: def a(): def b(): nonlocal c because c, the variable, could be declared in a, the function, some time after the definition of b, the function, is done. The way I see this, like I have expressed in my previous comments, is that whatever solution we come up with should not involve the parser, because it should not care about statement semantics. I'm not really sure what the best solutions are, but I'd propose the following for the problems I listed above: 1) We should alter code in symtable.c to check whether the namespace in which the nonlocal statement appears in is a function block and whether it is nested or not. 2) A check for the SyntaxError message to see if it comes from a nonlocal statement surely sounds like a hack, but I don't think that's a dealbreaker. I'd probably go with this rather than undergoing the effort of exposing the indentation stack in Python and then examining it in codeop. Thoughts? -- ___ Python tracker <https://bugs.python.org/issue19335> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19335] codeop misclassifies incomplete code with 'nonlocal'
Lysandros Nikolaou added the comment: > From what I understand, "checking for two or more hanging INDENTS" and, > "hardcoding a check for nonlocal SyntaxErrors in codeop._maybe_compile" are > two different solutions, right? If yes, do we have an answer to which one of > them is more cleaner, and henceforth, the preferable solution? Note that these are two solution that take very different approaches. What Nick is suggesting with "checking for two or more hanging INDENTS" would drastically change how codeop._maybe_compile does its thing, while his other proposed solution, ""hardcoding a check for nonlocal SyntaxErrors in codeop._maybe_compile", would just fix this issue. > I, personally, like the idea of checking INDENTS primarily because of it's > reduced specificity, but I am in no position to comment on this (I already > kinda did ':D), and you folks know better! For all we know, we should be > optimizing for specificity. You're right that this idea is more general. It would require significantly more effort from whoever tackles this though. > Also, reading Nick's comments and the comc's code, gives me the feeling that > a fix for this wouldn't require drastic changes. That really depends on what solution is chosen out of the two. > I'm slowly starting my journey with CPython, and I'd like to contribute a > patch if that is the case. Thanks! If there's consensus around one proposed approach, you could certainly take this up. I'd be glad to help out with workflow stuff or provide a first review. Let me know if you've got more questions. -- keywords: +3.3regression ___ Python tracker <https://bugs.python.org/issue19335> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41084] Signify that a SyntaxError comes from an fstring in the error message
Lysandros Nikolaou added the comment: Thanks, Victor! The fix is merged, so I'm closing this again. -- ___ Python tracker <https://bugs.python.org/issue41084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41084] Signify that a SyntaxError comes from an fstring in the error message
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue41084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19335] codeop misclassifies incomplete code with 'nonlocal'
Lysandros Nikolaou added the comment: > If I understand correctly, Nick is talking about modifying the different > iterations for different errors in codeop no? I was talking about msg200936, where Nick proposed to just hardcode a check for nonlocal SyntaxErrors in codeop._maybe_compile, which should return None (which means incomplete) when the SyntaxError comes as a result of a nonlocal binding not existing. -- ___ Python tracker <https://bugs.python.org/issue19335> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19335] codeop misclassifies incomplete code with 'nonlocal'
Lysandros Nikolaou added the comment: > What do either of you think? Can the new parser handle it better? Pablo, correct me if I'm wrong, but I think that the parser has nothing to do with this. It's not the parser that produces this SyntaxError, it's some later compilation phase (I guess symtable creation phase?), which I know very very little about, so I can't really offer an opinion on what can be done. In any case, this should actually indicate that this is not a parser issue: (venv) ➜ cpython git:(master) ./python.exe Python 3.10.0a0 (heads/master:7f569c9bc0, Jun 29 2020, 14:48:39) [Clang 11.0.3 (clang-1103.0.32.62)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> a = '''\ ... def a(): ... def b(): ... nonlocal c ... ''' >>> a 'def a():\ndef b():\nnonlocal c\n' >>> compile(a, '?', 'single') Traceback (most recent call last): File "", line 1, in File "?", line 3 SyntaxError: no binding for nonlocal 'c' found >>> import ast >>> ast.dump(ast.parse(a, mode='single')) "Interactive(body=[FunctionDef(name='a', args=arguments(posonlyargs=[], args=[], kwonlyargs=[], kw_defaults=[], defaults=[]), body=[FunctionDef(name='b', args=arguments(posonlyargs=[], args=[], kwonlyargs=[], kw_defaults=[], defaults=[]), body=[Nonlocal(names=['c'])], decorator_list=[])], decorator_list=[])])" All in all, I think that Nick's hack is the way to go here, since it's comc's responsibility to correctly check if a statement is incomplete, right? > When the new parser compiles a function, does it know or could it know > whether it is nested? It currently doesn't, but the tokenizer holds a stack of indent tokens, which can be accessed by the parser. This way we could indeed check if there are two or more pending indent tokens, before generating an AST for nonlocal. However, I don't think that this is generally a good idea. Parsing should not care about statement semantics, in that a nonlocal statement should be like any other statement in the parser's point of view. -- ___ Python tracker <https://bugs.python.org/issue19335> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41084] Signify that a SyntaxError comes from an fstring in the error message
Change by Lysandros Nikolaou : -- pull_requests: +20366 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/21212 ___ Python tracker <https://bugs.python.org/issue41084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41076] Pre-feed the parser with the f-string expression location
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue41076> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41132] F-String parser uses raw allocator
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue41132> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41132] F-String parser uses raw allocator
Change by Lysandros Nikolaou : -- pull_requests: +20340 pull_request: https://github.com/python/cpython/pull/21184 ___ Python tracker <https://bugs.python.org/issue41132> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41132] F-String parser uses raw allocator
Change by Lysandros Nikolaou : -- pull_requests: +20339 pull_request: https://github.com/python/cpython/pull/21183 ___ Python tracker <https://bugs.python.org/issue41132> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41084] Signify that a SyntaxError comes from an fstring in the error message
Lysandros Nikolaou added the comment: Should we maybe backport this to 3.9 as well? -- ___ Python tracker <https://bugs.python.org/issue41084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41132] F-String parser uses raw allocator
Lysandros Nikolaou added the comment: Ok, I've opened a PR. Should be backport this to previous versions as well? -- ___ Python tracker <https://bugs.python.org/issue41132> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41132] F-String parser uses raw allocator
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +20331 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21173 ___ Python tracker <https://bugs.python.org/issue41132> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41132] F-String parser uses raw allocator
New submission from Lysandros Nikolaou : The f-string parser uses the raw allocator in various places. We had a very very brief discussion about this with Pablo and he suggested to change the new parser and the hand-written f-string parser to use the pymalloc allocator. Eric, is there a specific reason the raw allocator was used? -- components: Interpreter Core messages: 372442 nosy: eric.smith, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: F-String parser uses raw allocator versions: Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue41132> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41119] Wrong error message for list/tuple followed by a colon
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue41119> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41084] Signify that a SyntaxError comes from an fstring in the error message
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue41084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41119] Wrong error message for list/tuple followed by a colon
Change by Lysandros Nikolaou : -- pull_requests: +20330 pull_request: https://github.com/python/cpython/pull/21172 ___ Python tracker <https://bugs.python.org/issue41119> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41119] Wrong error message for list/tuple followed by a colon
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +20320 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21160 ___ Python tracker <https://bugs.python.org/issue41119> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41119] Wrong error message for list/tuple followed by a colon
New submission from Lysandros Nikolaou : Brandt found this out while testing his implementation of the `match` statement. When a list or tuple are followed by a colon without an annotation, the old parser used to say "invalid syntax", while the new parser considers this an annotation and outputs something along the lines of "only single target (not tuple) can be annotated". For example: ➜ cpython git:(master) ./python.exe Python 3.10.0a0 (heads/master:06a40d7359, Jun 26 2020, 01:33:34) [Clang 11.0.3 (clang-1103.0.32.62)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> (a, b): File "", line 1 (a, b): ^ SyntaxError: only single target (not tuple) can be annotated >>> [a, b]: File "", line 1 [a, b]: ^ SyntaxError: only single target (not list) can be annotated >>> a,: File "", line 1 a,: ^ SyntaxError: only single target (not tuple) can be annotated The behavior of the old parser seems more logical. -- assignee: lys.nikolaou messages: 372390 nosy: brandtbucher, gvanrossum, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: Wrong error message for list/tuple followed by a colon type: behavior versions: Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue41119> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41084] Signify that a SyntaxError comes from an fstring in the error message
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +20250 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21084 ___ Python tracker <https://bugs.python.org/issue41084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41084] Signify that a SyntaxError comes from an fstring in the error message
Lysandros Nikolaou added the comment: > My only reservation would be: if the parsing of f-strings is moved into the > parser, would it be possible to maintain the error new messages? Probably without a single change. -- ___ Python tracker <https://bugs.python.org/issue41084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41084] Signify that a SyntaxError comes from an fstring in the error message
New submission from Lysandros Nikolaou :
It's relatively easy to identify if a SyntaxError occurs when parsing an
fstring expression or not. The idea is to slightly change the error message to
start with "f-string: " when it does (in the same way in which SyntaxError
messages are produced by the hand-written fstring parser). I have a working PR
that produces the following:
$ cat a.py
f'{a $ b}'
$ ./python.exe a.py
File "/Users/lysnikolaou/Repositories/cpython/a.py", line 1
(a $ b)
^
SyntaxError: f-string: invalid syntax
Thoughts?
--
components: Interpreter Core
messages: 372134
nosy: eric.smith, gvanrossum, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Signify that a SyntaxError comes from an fstring in the error message
type: enhancement
versions: Python 3.10, Python 3.9
___
Python tracker
<https://bugs.python.org/issue41084>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41076] Pre-feed the parser with the f-string expression location
Change by Lysandros Nikolaou : -- title: Pre-feed the parser with the f-string location -> Pre-feed the parser with the f-string expression location ___ Python tracker <https://bugs.python.org/issue41076> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41076] Pre-feed the parser with the f-string location
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +20225 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21054 ___ Python tracker <https://bugs.python.org/issue41076> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41076] Pre-feed the parser with the f-string location
New submission from Lysandros Nikolaou :
Inspired by bpo-41064, I sat down to try and find problems with f-string
locations in the new parser. I was able to come up with a way to compute the
locations of the f-string expressions that I think is more consistent and
allows us to delete all the code that was fixing the expression locations after
the actual parsing, which accounted for about 1/6 of string_parser.c.
A high-level explanation of the change:
Before this change we were pre-feeding the parser with the location of the
f-string itself. The parser was then parsing the expression and was computing
the locations of all the nodes based on the offset of the f-string. After the
parsing was done, we were identifying the offset and the lineno of the
expression *within* the fstring and were fixing the node locations accordingly.
For example, for an f-string like `a = 0; f'irrelevant {a}'` we were doing the
following:
- Pre-feed the parser with lineno=0 and col_offset=7 (the offset of the
f-string itself in the current line).
- Parse the expression (adding 7 to the col_offset of each parsed node, lineno
remains the same since it's 0).
- Fix the node locations by shifting the Name node by 14, which is the number
of characters in the f-string (counting the `f` and the opening quote) before
the start of the expression.
With this change we now pre-feed the parser with the exact lineno and offset of
the expression itself, not the f-string. This allows us to completely skip the
third step of shifting the node locations.
--
assignee: lys.nikolaou
components: Interpreter Core
messages: 372086
nosy: eric.smith, gvanrossum, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Pre-feed the parser with the f-string location
type: behavior
versions: Python 3.10, Python 3.9
___
Python tracker
<https://bugs.python.org/issue41076>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Lysandros Nikolaou added the comment: Thanks Victor for the catch! I've opened a PR that deprecates PyNode_Compile in 3.9. -- ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Change by Lysandros Nikolaou : -- pull_requests: +20207 pull_request: https://github.com/python/cpython/pull/21036 ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41060] `with a as b` segfault in new peg parser
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue41060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41060] `with a as b` segfault in new peg parser
Change by Lysandros Nikolaou : -- pull_requests: +20196 pull_request: https://github.com/python/cpython/pull/21024 ___ Python tracker <https://bugs.python.org/issue41060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41060] `with a as b` segfault in new peg parser
Lysandros Nikolaou added the comment: Thanks for the report, Steve. I've now opened a PR that should fix this. -- ___ Python tracker <https://bugs.python.org/issue41060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41060] `with a as b` segfault in new peg parser
Change by Lysandros Nikolaou : -- assignee: -> lys.nikolaou ___ Python tracker <https://bugs.python.org/issue41060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41060] `with a as b` segfault in new peg parser
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +20192 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21020 ___ Python tracker <https://bugs.python.org/issue41060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41060] `with a as b` segfault in new peg parser
Lysandros Nikolaou added the comment: On it. -- ___ Python tracker <https://bugs.python.org/issue41060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Change by Lysandros Nikolaou : -- pull_requests: +20190 pull_request: https://github.com/python/cpython/pull/21016 ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Change by Lysandros Nikolaou : -- pull_requests: +20186 pull_request: https://github.com/python/cpython/pull/21012 ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Change by Lysandros Nikolaou : -- pull_requests: +20179 pull_request: https://github.com/python/cpython/pull/21005 ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Lysandros Nikolaou added the comment: I'm currently testing a commit that removes all these files on my fork, before I push it upstream. A question that I'm not 100% sure about is if we can already remove the symbol module. I guess it's okay since it got deprecated in 3.9 (bpo-40759) and the old parser is also out, but just to make sure. -- ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Lysandros Nikolaou added the comment: All parts of the implementation are now in. New issues should be opened for any potential bugs. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +20150 pull_request: https://github.com/python/cpython/pull/20973 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Change by Lysandros Nikolaou : -- pull_requests: +20148 pull_request: https://github.com/python/cpython/pull/20970 ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Change by Lysandros Nikolaou : -- pull_requests: +20146 pull_request: https://github.com/python/cpython/pull/20968 ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: > WIll make a PR soon No need. Already got something ready. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40985] PEG Parser: SyntaxError text empty when last line has a LINECONT
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40985> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40985] PEG Parser: SyntaxError text empty when last line has a LINECONT
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +20071 stage: -> patch review pull_request: https://github.com/python/cpython/pull/20888 ___ Python tracker <https://bugs.python.org/issue40985> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40985] PEG Parser: SyntaxError text empty when last line has a LINECONT
Lysandros Nikolaou added the comment: I keep doing something wrong. I hope it is clean by now what the difference is. -- ___ Python tracker <https://bugs.python.org/issue40985> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40985] PEG Parser: SyntaxError text empty when last line has a LINECONT
Change by Lysandros Nikolaou : -- nosy: +gvanrossum, pablogsal ___ Python tracker <https://bugs.python.org/issue40985> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40985] PEG Parser: SyntaxError text empty when last line has a LINECONT
Lysandros Nikolaou added the comment: Copying and pasting the example messed the formatting up. Here it is again. [16:49:16] lysnikolaou:cpython git:(master*) $ cat t.py x = 6\% [16:49:23] lysnikolaou:cpython git:(master*) $ ./python t.py File "/home/lysnikolaou/repos/cpython/t.py", line 2 ^ SyntaxError: unexpected EOF while parsing [16:49:56] lysnikolaou:cpython git:(master*) $ python3.9 -X oldparser t.py File "/home/lysnikolaou/repos/cpython/t.py", line 2 x = 6\ ^ SyntaxError: unexpected EOF while parsing -- ___ Python tracker <https://bugs.python.org/issue40985> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40985] PEG Parser: SyntaxError text empty when last line has a LINECONT
New submission from Lysandros Nikolaou : While investigating bpo-40958, the following came up: When a file ends with a line that contains a line continuation character the text of the emitted SyntaxError is empty, contrary to the old parser, where the error text contained the text of the last line. Here is an example: cpython git:(master)$ cat t.py x = 6\% cpython git:(master)$ ./python t.py File "/home/lysnikolaou/repos/cpython/t.py", line 2 ^ SyntaxError: unexpected EOF while parsing cpython git:(master)$ python3.9 -X oldparser t.py File "/home/lysnikolaou/repos/cpython/t.py", line 2 x = 6\ ^ SyntaxError: unexpected EOF while parsing -- components: Interpreter Core messages: 371544 nosy: lys.nikolaou priority: normal severity: normal status: open title: PEG Parser: SyntaxError text empty when last line has a LINECONT type: behavior versions: Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue40985> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: > +Py_ssize_t linesize = PyUnicode_GET_LENGTH(line); This line is wrong though, since PyUnicode_GET_LENGTH returns the length in code points and PyUnicode_DecodeUTF8 expects the number of bytes. For non-ascii input this would push the caret further to the left. For example: $ ./python.exe Python 3.10.0a0 (heads/master-dirty:e2fb8a2c42, Jun 12 2020, 20:22:52) [Clang 11.0.0 (clang-1100.0.33.8)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> "Ṕýţĥòñ" + File "", line 1 "Ṕýţĥòñ" + ^ SyntaxError: invalid syntax -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: > Can you run the test suite before/after of pyflakes to make sure we don't > introduce any regression to double check? If everything looks fine, open a PR > :) The exact same errors before and after! I'll wait on the PR though till someone has checked if this indeed solves the heap-buffer-overflow. I'm not at all confident that is does and I can't test. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment:
I'm guessing that some parts of the conversion code were only there to
circumvent issues in displaying the error messages that weren't pegen's fault.
These were fixed by Guido in GH-20072, so I think we can delete some of them.
For example, this patch works just fine for me.
diff --git a/Parser/pegen.c b/Parser/pegen.c
index e29910bf86..2c348178fb 100644
--- a/Parser/pegen.c
+++ b/Parser/pegen.c
@@ -150,10 +150,6 @@ byte_offset_to_character_offset(PyObject *line, int
col_offset)
return 0;
}
Py_ssize_t size = PyUnicode_GET_LENGTH(text);
-str = PyUnicode_AsUTF8(text);
-if (str != NULL && (int)strlen(str) == col_offset) {
-size = strlen(str);
-}
Py_DECREF(text);
return size;
}
@@ -400,9 +396,6 @@ _PyPegen_raise_error_known_location(Parser *p, PyObject
*errtype,
if (!error_line) {
Py_ssize_t size = p->tok->inp - p->tok->buf;
-if (size && p->tok->buf[size-1] == '\n') {
-size--;
-}
error_line = PyUnicode_DecodeUTF8(p->tok->buf, size, "replace");
if (!error_line) {
goto error;
--
___
Python tracker
<https://bugs.python.org/issue40958>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: > What happens with: >>> "Ṕýţĥòñ" + Exact same thing. The offset is 16 at the start and gets decreased to 15 in the line I linked to in my previous post. And then col_offset gets converted to col_number which is 9, which seems correct. Although it is correct, it feels correct by accident, since the if-branch on https://github.com/python/cpython/blob/e2fb8a2c42ee60c72a40d93da69e9efc4e359023/Parser/pegen.c#L154 only evaluates to false due to the decrease of size and I don't know if that was intended. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: > Note that although we could just exit if the length of the line is smaller > than the column offset before calling > https://github.com/python/cpython/blob/master/Parser/pegen.c#L148 (I assume > that is the problem) is more important to understand how are we reaching that > situation. That's because of https://github.com/python/cpython/blob/e2fb8a2c42ee60c72a40d93da69e9efc4e359023/Parser/pegen.c#L404, which decreases the size of the string to be decoded by 1 if the last character is a newline (or, equivalently, if the error offset points past the end of the line). Thus, PyUnicode_DecodeUTF8 returns an object that is one character shorter than the col_offset and that's how we get to the situation you mentioned. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: Is the only way to get gcc-10.1 to build from source? Because currently my internet connection is too slow to get all of gcc downloaded and `apt install gcc-10` installs 10.0.1, which I'm getting totally unrelated erros with. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: > Lysandros, are you using a debug build? The assert() won't trigger if you > don't have run configure with --with-pydebug. BTW I'm not talking about the assert not triggering. I'm only saying that ASAN/UBSAN do not report an error when running `yield from` in the REPL. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: I now configured Python with ./configure --with-pydebug --with-address-sanitizer --with-undefined-behavior-sanitizer. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: Let me try that. I'm currently using gcc 9.3. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: Running Pablo's examples in the REPL does not reproduce the errors either: ➜ cpython git:(master) ✗ LSAN_OPTIONS="suppressions=asan-suppression.txt,print_suppressions=0" ./python Modules/posixmodule.c:14682:9: runtime error: left shift of 34 by 26 places cannot be represented in type 'int' Python 3.10.0a0 (heads/master:e2fb8a2c42, Jun 12 2020, 16:59:27) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> "Ṕýţĥòñ" + File "", line 1 "Ṕýţĥòñ" + ^ SyntaxError: invalid syntax >>> yield from File "", line 1 yield from ^ SyntaxError: invalid syntax >>> ^D ➜ cpython git:(master) ✗ -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: I cannot reproduce this on my Ubuntu 20.04: ➜ cpython git:(master) ✗ LSAN_OPTIONS="suppressions=asan-suppression.txt,print_suppressions=0" ./python -m test -v test_eof Modules/posixmodule.c:14682:9: runtime error: left shift of 34 by 26 places cannot be represented in type 'int' == CPython 3.10.0a0 (heads/master:e2fb8a2c42, Jun 12 2020, 16:59:27) [GCC 9.3.0] == Linux-5.4.0-33-generic-x86_64-with-glibc2.31 little-endian == cwd: /home/lysnikolaou/repos/cpython/build/test_python_71258 == CPU count: 4 == encodings: locale=UTF-8, FS=utf-8 0:00:00 load avg: 2.11 Run tests sequentially 0:00:00 load avg: 2.11 [1/1] test_eof test_EOFC (test.test_eof.EOFTestCase) ... ok test_EOFS (test.test_eof.EOFTestCase) ... ok test_eof_with_line_continuation (test.test_eof.EOFTestCase) ... ok test_line_continuation_EOF (test.test_eof.EOFTestCase) A continuation at the end of input must be an error; bpo2180. ... ok test_line_continuation_EOF_from_file_bpo2180 (test.test_eof.EOFTestCase) Ensure tok_nextc() does not add too many ending newlines. ... Modules/posixmodule.c:14682:9: runtime error: left shift of 34 by 26 places cannot be represented in type 'int' ok -- Ran 5 tests in 0.413s OK == Tests result: SUCCESS == 1 test OK. Total duration: 654 ms Tests result: SUCCESS -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: Agreed. For that I'd probably need the input Christian is getting the error with. I'm currently trying things out, but haven't gotten the error yet. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40958] ASAN/UBSAN: heap-buffer-overflow in pegen.c
Lysandros Nikolaou added the comment: > Lysandros, could you take a look? Yup, I'm on it. -- ___ Python tracker <https://bugs.python.org/issue40958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Lysandros Nikolaou added the comment: > This change broke this buildbot: GH-20802 fixes this breakage. -- ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Lysandros Nikolaou added the comment: > Lysandros can you modify PR 20802 to remove it? Yup! -- ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Change by Lysandros Nikolaou : -- pull_requests: +19998 pull_request: https://github.com/python/cpython/pull/20802 ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Lysandros Nikolaou added the comment: > I would personally would like to keep the easter egg, but I assume is better > to rename it to "__peg_parser__". Ok then! On it. -- ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Lysandros Nikolaou added the comment: > Can we rename the keyword to "__peg_parser__"? I guess we could just remove this, as soon as the old parser is out. We were only using this to differentiate between the two parsers, when we were testing enabling/disabling the old one. I could get a PR ready to be merged after GH-20768 is there. -- ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40939] Remove the old parser
Change by Lysandros Nikolaou : -- pull_requests: +19996 pull_request: https://github.com/python/cpython/pull/20800 ___ Python tracker <https://bugs.python.org/issue40939> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40847] New parser considers empty line following a backslash to be a syntax error, old parser didn't
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +19967 stage: -> patch review pull_request: https://github.com/python/cpython/pull/20769 ___ Python tracker <https://bugs.python.org/issue40847> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30533] missing feature in inspect module: getmembers_static
Change by Lysandros Nikolaou : -- nosy: -lys.nikolaou ___ Python tracker <https://bugs.python.org/issue30533> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22021] shutil.make_archive() root_dir do not work
Lysandros Nikolaou added the comment: Since GH-10367 is now merged, should we close this? -- ___ Python tracker <https://bugs.python.org/issue22021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40614] ast.parse doesn't respect feature_version for debug f-strings
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40614> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40847] New parser considers empty line following a backslash to be a syntax error, old parser didn't
Lysandros Nikolaou added the comment:
This is limited to cases where the line continuation character is on an
otherwise empty line. For example this works correctly:
$ cat t.py
print("hello world")
print("hello world 2") \
print("hello world 3")
$ ./python.exe t.py
hello world
hello world 2
hello world 3
The actual problem is at the tokenizer level, where a line with only a
continuation character is not considered an empty line and thus two NEWLINE
tokens get emitted, one after the other. The old parser was somehow working
around this, probably by having this in the grammar:
file_input: (NEWLINE | stmt)* ENDMARKER
The PEG parser OTOH does not allow this.
The question now is, is it reasonable to change the tokenizer to consider a
lone backslash an empty line? Do you also consider this a bug? Or should we
change the new parser?
--
___
Python tracker
<https://bugs.python.org/issue40847>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40880] Invalid read in pegen.c
Change by Lysandros Nikolaou : -- nosy: +gvanrossum, pablogsal ___ Python tracker <https://bugs.python.org/issue40880> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38964] Output of syntax error in f-string contains wrong filename
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue38964> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38964] Output of syntax error in f-string contains wrong filename
Change by Lysandros Nikolaou : -- pull_requests: +19666 pull_request: https://github.com/python/cpython/pull/20404 ___ Python tracker <https://bugs.python.org/issue38964> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Change by Lysandros Nikolaou : -- resolution: -> wont fix stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40246] Different error messages for same error - invalid string prefixes
Change by Lysandros Nikolaou : -- pull_requests: +19664 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/20402 ___ Python tracker <https://bugs.python.org/issue40246> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38964] Output of syntax error in f-string contains wrong filename
Change by Lysandros Nikolaou : -- versions: +Python 3.10 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue38964> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38964] Output of syntax error in f-string contains wrong filename
Change by Lysandros Nikolaou : -- keywords: +patch nosy: +lys.nikolaou nosy_count: 4.0 -> 5.0 pull_requests: +19662 stage: -> patch review pull_request: https://github.com/python/cpython/pull/20399 ___ Python tracker <https://bugs.python.org/issue38964> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40688] PEG Generator: Fix scripts to always use the correct parser
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40716] Pegen: improve error messages for unparenthesized from imports with trailing comma
Change by Lysandros Nikolaou : -- nosy: +gvanrossum, lys.nikolaou, pablogsal ___ Python tracker <https://bugs.python.org/issue40716> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40688] PEG Generator: Fix scripts to always use the correct parser
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +19522 stage: -> patch review pull_request: https://github.com/python/cpython/pull/20235 ___ Python tracker <https://bugs.python.org/issue40688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40688] PEG Generator: Fix scripts to always use the correct parser
New submission from Lysandros Nikolaou : All the scripts in Tools/peg_generator/scripts need to be updated, since they mostly assume that ast.parse and compile use the old parser. We can use the _peg_parser extension module instead, but it well need some enhancements so that it also supports compilation to bytecode. -- assignee: lys.nikolaou components: Demos and Tools messages: 369381 nosy: gvanrossum, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: PEG Generator: Fix scripts to always use the correct parser type: behavior versions: Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue40688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40614] ast.parse doesn't respect feature_version for debug f-strings
Change by Lysandros Nikolaou : -- nosy: +lys.nikolaou ___ Python tracker <https://bugs.python.org/issue40614> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40546] Inconsistencies between PEG parser and traceback SyntaxErrors
Lysandros Nikolaou added the comment: Fixed in GH-20072 and bpo-40612. -- resolution: -> fixed stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40546> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40669] PEG Parser benchmarks fail if memory_profiler is not installed
Change by Lysandros Nikolaou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40669> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40669] PEG Parser benchmarks fail if memory_profiler is not installed
Change by Lysandros Nikolaou : -- pull_requests: +19484 pull_request: https://github.com/python/cpython/pull/20183 ___ Python tracker <https://bugs.python.org/issue40669> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40669] PEG Parser benchmarks fail if memory_profiler is not installed
Lysandros Nikolaou added the comment: > Doc/Makefile uses a virtual environment to install Sphinx and Sphinx > extensions. I suggest to do something similar. I like this approach. Let me try it. -- ___ Python tracker <https://bugs.python.org/issue40669> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40669] PEG Parser benchmarks fail if memory_profiler is not installed
Lysandros Nikolaou added the comment: Because `make time` and all the related Makefile targets use the local built Python in the cpython parent directory. In order to install it, one would have to go through various steps, like installing openssl, configuring python with ssl support, building and installing python, creating a venv etc. Since it's more time-consuming that just running `make time` and since it's only needed for something that in most cases is not the most important thing, I think we can just go ahead and make it optional. Is there a downside to this change I'm missing? -- ___ Python tracker <https://bugs.python.org/issue40669> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40669] PEG Parser benchmarks fail if memory_profiler is not installed
Change by Lysandros Nikolaou : -- keywords: +patch pull_requests: +19471 stage: -> patch review pull_request: https://github.com/python/cpython/pull/20172 ___ Python tracker <https://bugs.python.org/issue40669> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40669] PEG Parser benchmarks fail if memory_profiler is not installed
New submission from Lysandros Nikolaou : Running the PEG Parser benchmarks requires having memory_profiler installed. In Tools/peg_generator: ➜ peg_generator git:(master) make time_stdlib ../../python -m zipfile -e data/xxl.zip data ../../python scripts/benchmark.py --parser=cpython --target=xxl compile Traceback (most recent call last): File "/home/lysnikolaou/repos/cpython/Tools/peg_generator/scripts/benchmark.py", line 9, in import memory_profiler ModuleNotFoundError: No module named 'memory_profiler' I propose to make that optional and only compute the timing benchmarks, in case memory_profiler is not available, since it's these benchmarks that are most important and most frequently run. -- assignee: lys.nikolaou messages: 369204 nosy: gvanrossum, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: PEG Parser benchmarks fail if memory_profiler is not installed ___ Python tracker <https://bugs.python.org/issue40669> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Lysandros Nikolaou added the comment: I'm not getting any compiler warnings on macOS (clang) and on Ubuntu (gcc) and I can't find any related warnings on the Windows Github Actions logs either. Could you maybe post a verbose log of test_peg_generator, Raymond? -- ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40661] The new parser segfaults when parsing invalid input
Change by Lysandros Nikolaou : -- pull_requests: +19464 pull_request: https://github.com/python/cpython/pull/20165 ___ Python tracker <https://bugs.python.org/issue40661> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19456 pull_request: https://github.com/python/cpython/pull/20153 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19454 pull_request: https://github.com/python/cpython/pull/20151 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40334] PEP 617: new PEG-based parser
Change by Lysandros Nikolaou : -- pull_requests: +19413 pull_request: https://github.com/python/cpython/pull/20106 ___ Python tracker <https://bugs.python.org/issue40334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40631] PEG Parser: Cannot used starred expression in parenthesised expr
Change by Lysandros Nikolaou : -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40631> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40631] PEG Parser: Cannot used starred expression in parenthesised expr
Lysandros Nikolaou added the comment: Whoops, false alarm. It's just that we moved the check for invalid starred expressions to the parser, while it previously was in the compiler. ╰─ ./python.exe -X oldparser Python 3.9.0a6+ (heads/master-dirty:003708bcf8, May 15 2020, 15:08:21) [Clang 11.0.0 (clang-1100.0.33.8)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> (*a) File "", line 1 SyntaxError: can't use starred expression here Sorry for the noise! -- ___ Python tracker <https://bugs.python.org/issue40631> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
