[issue16806] col_offset is -1 for multiline string expressions resembling docstrings
Carsten Klein added the comment: Please see the attached patch that will resolve the issue. It also includes a test case in test_ast.py. What the patch does is as follows: - tok_state is extended by two fields, namely first_lineno and multi_line_start - first_lineno will be set by tok_get as soon as the beginning of a STRING is detected and it will be set to the current line tok-lineno. - multi_line_start is the beginning of the first line of a string - in parsetok we now distinguish between STRING nodes and other nodes. in case of STRING nodes, we will use the values of the above fields for determining the actual lineno and the col_offset, otherwise tok-col_offset and tok-lineno will be used when creating the token. The included test case ensures that the col_offset and lineno of multi line strings is calculated correctly. -- keywords: +patch Added file: http://bugs.python.org/file28477/issue1680.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16806 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16806] col_offset is -1 for multiline string expressions resembling docstrings
New submission from Carsten Klein:
Given an input module such as
class klass(object):
multi line comment
continued on this line
single line comment
Another multi
line
comment
and implementing a custom ast.NodeVisitor such as
import as
class CustomVisitor(ast.NodeVisitor):
def visit_ClassDef(self, node):
for childNode in node.body:
self.visit(childNode)
def visit_Expr(self, node):
print(node.col_offset)
print(node.value.col_offset)
and feeding it the compiled ast from the module above
f = open('./module.py')
source = f.read()
node = ast.parse(source, mode = 'exec')
visitor = CustomVisitor()
visitor.visit(node)
should yield -1/-1 for the docstring that is the first
child node expression of the classdef body.
it will, however, yield the correct col_offset of 4/4 for
the single line docstring following the first one.
the multi line docstring following that will again
yield a -1/-1 col_offset.
It believe that this behaviour is not correct and instead
the col_offset should be 4 for both the expression node
and its str value.
--
components: Interpreter Core
messages: 178444
nosy: carsten.kl...@axn-software.de
priority: normal
severity: normal
status: open
title: col_offset is -1 for multiline string expressions resembling docstrings
type: enhancement
versions: Python 3.4
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16806
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16806] col_offset is -1 for multiline string expressions resembling docstrings
Carsten Klein added the comment: Please note that, regardless of the indent level, the col_offset for multi line str expressions will always be -1. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16806 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16806] col_offset is -1 for multiline string expressions resembling docstrings
Carsten Klein added the comment: In addition, the reported lineno will be set to the last line of the multi line string instead of the first line where parsing the parse began parsing the string. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16806 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
