[issue24119] Carry comments with the AST

New submission from Brett Cannon: One thing about https://www.python.org/dev/peps/pep-0484/ is that it makes comments potentially semantically meaningful. Unfortunately the AST doesn't carry comments with it in any way, making it difficult to build a tool to implement a linter for PEP 484 usin

[issue24119] Carry comments with the AST

Mark Shannon added the comment: Comments don't belong on the AST. Where would you attach them? The tokenizer module provides all information about comments. Tools can get the information quite easily if they need it. -- nosy: +Mark.Shannon ___ Pytho

[issue24119] Carry comments with the AST

Brett Cannon added the comment: Normally I would agree comments don't belong there, but if we are going to start giving them semantic meaning then I don't think it's not so clear to me anymore. As to where to attach, simple place is off of the Module node. Another is to have it be fundamental

[issue24119] Carry comments with the AST

Changes by Christian Heimes : -- nosy: +christian.heimes ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https:

[issue24119] Carry comments with the AST

Brett Cannon added the comment: Another option is to provide a tool in 'tokenize' or 'ast' which will take the source and some comment regex and then attach the found comment metadata to the AST. -- ___ Python tracker

[issue24119] Carry comments with the AST

Antoine Pitrou added the comment: Or a separate AST node -> comment mapping. -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-l

[issue24119] Carry comments with the AST

Mateusz Bysiek added the comment: For some time now, there's an alternate ast implementation https://github.com/python/typed_ast that carries PEP 484 type comments with the AST as attributes of certain nodes. Their approach is described here: https://github.com/python/typed_ast/blob/master/ty

[issue24119] Carry comments with the AST

Brett Cannon added the comment: The type annotation is already in the AST so there's nothing to carry over from typed_ast (we only care about the latest Python version while typed_ast tries to be version-agnostic). -- ___ Python tracker

[issue24119] Carry comments with the AST

Louie Lu added the comment: Brett, which implement method will you prefer? If we want to carry comment at builtin_compile_impl, it will need to change the grammar since tokenize just drop the comment when dealing with source code. But if just using regex, will it be more easy with just combine

[issue24119] Carry comments with the AST

Raymond Hettinger added the comment: After PEP 526, the need for this proposal may have evaporated. -- nosy: +rhettinger ___ Python tracker ___ __

[issue24119] Carry comments with the AST

Brett Cannon added the comment: There's potentially some usefulness from other tools, but Raymond is right that the main motivation is definitely gone long-term. Dropping this down to "low" priority simply because others have asked for this kind of support before. -- priority: normal -

[issue24119] Carry comments with the AST

Serhiy Storchaka added the comment: Where would comment be attached in the following case? a = ('start' # comment 1 'continuation') # comment 2 -- nosy: +serhiy.storchaka ___ Python tracker __

[issue24119] Carry comments with the AST

Change by kernc : -- nosy: +kernc ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mai

[issue24119] Carry comments with the AST

Raymond Hettinger added the comment: Since the AST carries the module/lineno attributes isn't there already a way trace back into the token stream to recover comments? -- ___ Python tracker

[issue24119] Carry comments with the AST

Raymond Hettinger added the comment: Possible superceder: https://bugs.python.org/issue7 -- ___ Python tracker ___ ___ Python-

[issue24119] Carry comments with the AST

Guido van Rossum added the comment: See also issue35766. -- nosy: +gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing l

[issue24119] Carry comments with the AST

Change by Batuhan Taskaya : -- nosy: +BTaskaya ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue24119] Carry comments with the AST

Guido van Rossum added the comment: I propose to close this issue, since (as of Python 3.8) we now have ast.parse(source, type_comments=True). -- ___ Python tracker ___ _