New submission from Serhiy Storchaka <storchaka+cpyt...@gmail.com>:
SyntaxError can be raised at different stages of compiling. In some places the
source text is not available and should be read from the file using the file
name. Which does not work in case of compiling a string or reading from stdin.
>>> 0z
File "<stdin>", line 1
0z
^
SyntaxError: invalid syntax
>>> 0xz
File "<stdin>", line 1
SyntaxError: invalid hexadecimal literal
In the second example above the source line and the caret are absent in REPL.
The proposed PR fixes two errors in raising a SyntaxError in the tokenizer.
1. The text of the source line was not set if an exception was raised in the
tokenizer. Since most of these exceptions (with more detailed description) were
added in 3.8 I consider this a regressions.
2. The offset attribute was an offset in bytes. Now it is an offset in
characters.
It only fixes errors in the tokenizer. There are similar bugs in other parts of
the compiler.
This issue is based on the article
https://aroberge.blogspot.com/2019/12/a-tiny-python-exception-oddity.html .
----------
components: Interpreter Core
messages: 359331
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Fix attributes of syntax errors raized in the tokenizer
type: behavior
versions: Python 3.8, Python 3.9
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue39219>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
