New submission from Lysandros Nikolaou <[email protected]>:
Inspired by bpo-41064, I sat down to try and find problems with f-string
locations in the new parser. I was able to come up with a way to compute the
locations of the f-string expressions that I think is more consistent and
allows us to delete all the code that was fixing the expression locations after
the actual parsing, which accounted for about 1/6 of string_parser.c.
A high-level explanation of the change:
Before this change we were pre-feeding the parser with the location of the
f-string itself. The parser was then parsing the expression and was computing
the locations of all the nodes based on the offset of the f-string. After the
parsing was done, we were identifying the offset and the lineno of the
expression *within* the fstring and were fixing the node locations accordingly.
For example, for an f-string like `a = 0; f'irrelevant {a}'` we were doing the
following:
- Pre-feed the parser with lineno=0 and col_offset=7 (the offset of the
f-string itself in the current line).
- Parse the expression (adding 7 to the col_offset of each parsed node, lineno
remains the same since it's 0).
- Fix the node locations by shifting the Name node by 14, which is the number
of characters in the f-string (counting the `f` and the opening quote) before
the start of the expression.
With this change we now pre-feed the parser with the exact lineno and offset of
the expression itself, not the f-string. This allows us to completely skip the
third step of shifting the node locations.
----------
assignee: lys.nikolaou
components: Interpreter Core
messages: 372086
nosy: eric.smith, gvanrossum, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Pre-feed the parser with the f-string location
type: behavior
versions: Python 3.10, Python 3.9
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue41076>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
