Mark Shannon added the comment:
This problem is the parsing of f-strings.
The expressions in an f-string are not "eval"ed in the sense of the eval()
function. They are evaluated exactly the same as any other Python expression.
However the parsing of f-strings does not provide correct line numbers.
This problem also manifests itself in the ast and tokenize modules.
>>> m = ast.parse("""f'''
... {
... FOO
... }
... '''
... """)
>>> m.body[0].value.values[1].value.id
'FOO'
>>> m.body[0].value.values[1].value.lineno
2
That 2 should be a 3, and yet
eval(compile(m, "test2", "exec"))
File "<stdin>", line 1, in <module>
File "test2", line 5, in <module>
NameError: name 'FOO' is not defined
gives line 5 for the error, so not only are the line numbers wrong they are
inconsistent.
The problem is that the internals of the f-string are not tokenized and parsed
using the normal mechanism, but in an ad-hoc fashion in Python-ast.c as
demonstrated when tokenizing the source
$ python3.6 -m tokenize test2
0,0-0,0: ENCODING 'utf-8'
1,0-5,3: STRING "f'''\n{\nFOO\n}\n'''"
5,3-5,4: NEWLINE '\n'
6,0-6,0: ENDMARKER ''
The f-string could should be tokenized as something like:
FSTRING_START f'''
STRING_PART \n
LEFT_BRACE {
NEWLINE
IDENTIFIER FOO
NEWLINE
RIGHT_BRACE }
STRING_PART \n
FSTRING_END '''
Although this would complicate the tokenizer, it would mean that the internals
of f-strings could be made explicit in the grammar, and that the compiler could
generate correct offsets.
----------
nosy: +Mark.Shannon
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue29051>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
