>> But I also think this means we definitely have to get a parser module
What is in this context a "parse" module? Because that will massively change depending who you ask. We already expose APIs that return AST objects that can be used for all sort of things and a tokenizer module that exposes some form of lexing that is relatively close to the one that CPython uses internally. The only missing piece would be something that returns a CST with enough information to reconstruct the source but at this point that is absolutely arbitrary because nothing in CPython would use that tree. Not only that, but the requirements from such CST will change quite a lot depending on who you ask and that impacts a lot the APIs that we would need to offer. Offering a parse module here can involve quite a high maintainance cost without the certainly that will be useful to all set of users. That also without considering that many tools parsing Python code are not written on Python and will not be able to leverage it. On Mon, 20 Sep 2021, 22:39 Brett Cannon, <br...@python.org> wrote: > > > On Mon, Sep 20, 2021 at 8:58 AM Thomas Grainger <tagr...@gmail.com> wrote: > >> I don't think the python syntax should be beholden to syntax highlighting >> tools, eventually some syntax feature that PEG enables will require every >> parser or highlighter to switch to a similar or more powerful parse tool >> > > But that's not how syntax highlighting works in editors. You typically > don't get to choose the parsing tool used for syntax highlighting, you just > define the grammar using whatever is provided by the editor (which has > always been regexes based on my experience). So there's no way to "require" > every editor out there to switch to a PEG parser or equivalent to support > Python's grammar because that's asking every editor to change how syntax > highlighting is implemented at a lower level. > > Having said all that, I think as long as we understand that this is a > side-effect then it's fine; syntax highlighting is usually not tied to > semantics in an editor so it shouldn't be a blocker on this. If people care > they simply won't use the same type of quotes in their code (which I bet is > what most people will do unless Black says otherwise 😉). > > But I also think this means we definitely have to get a parser module for > tools together as this is way more potential breakage than just parentheses > for `with` statements and I don't know if formatting tools can just move to > the AST module at that point. 😅 > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/I4POAK22LZW4RNFGFFKQ6BILRLCSQO2I/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JCGJIU6ZHH4LLSL666EP2GBLMEJDVIOL/ Code of Conduct: http://python.org/psf/codeofconduct/
