New submission from STINNER Victor: I wrote an AST optimizer which build AST from source code using ast.parse(), modify AST in place, and then compile the AST to bytecode.
The problem is that the lineno and col_offset attributes of the "arg" AST node are uninitialized when AST is rebuild from Python objects: compile(ast_tree, ...). As a consequence, the compilation may fail because lineno or col_offset values are invalid. Attached patch enhances Parser/asdl_c.py to take "arg" attributes in account. It also updates the generated files Include/Python-ast.h and Python/Python-ast.c, and fix usage of the arg() function in Python/ast.c. It looks like only the "arg" AST node had attributes which were initialized (other AST nodes are handled with a different code path which already filled attributes). See Parser/Python.asdl for the definition of the "arg" AST node. Note: Python 2.7 is not affected. In Python 2, function arguments are simple expressions, they don't have a dedicated type with lineno and col_offset attributes. "arg" was introduced in Python 3 with the PEP 3107 (function annotations). ---------- files: ast.patch keywords: patch messages: 254093 nosy: benjamin.peterson, haypo, matrixise, yselivanov priority: normal severity: normal status: open title: Fix parser and AST: fill lineno and col_offset when compiling AST from Python objects versions: Python 3.5, Python 3.6 Added file: http://bugs.python.org/file40949/ast.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25555> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
