[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Gregory P. Smith]

Change by Gregory P. Smith :


--
resolution: out of date -> fixed
stage: patch review -> commit review
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[miss-islington]

miss-islington  added the comment:


New changeset 20bc40ef44b820733848d5838e803b5fe4350b93 by Miss Islington (bot) 
in branch '3.9':
bpo-36541: Add lib2to3 grammar PEP-570 pos-only arg parsing (GH-23759)
https://github.com/python/cpython/commit/20bc40ef44b820733848d5838e803b5fe4350b93


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[miss-islington]

miss-islington  added the comment:


New changeset 06bfd033e847bedb6e123d131dcf46393a4555df by Miss Islington (bot) 
in branch '3.8':
bpo-36541: Add lib2to3 grammar PEP-570 pos-only arg parsing (GH-23759)
https://github.com/python/cpython/commit/06bfd033e847bedb6e123d131dcf46393a4555df


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[miss-islington]

Change by miss-islington :


--
pull_requests: +22625
pull_request: https://github.com/python/cpython/pull/23769

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[miss-islington]

Change by miss-islington :


--
pull_requests: +22624
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/23768

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset 42c9f0fd0a5e67d4ae0022bfd7370cb9725a5b01 by Gregory P. Smith in 
branch 'master':
bpo-36541: Add lib2to3 grammar PEP-570 pos-only arg parsing (GH-23759)
https://github.com/python/cpython/commit/42c9f0fd0a5e67d4ae0022bfd7370cb9725a5b01


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Gregory P. Smith]

Gregory P. Smith  added the comment:

While I said i didn't care... and don't really want to... I found a reason to 
at least not omit pep-570 positional only arg parsing support give things like 
yapf still use it rather than forking their own copy.  PR testing.

--
assignee:  -> gregory.p.smith
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Gregory P. Smith]

Change by Gregory P. Smith :


--
pull_requests: +22615
pull_request: https://github.com/python/cpython/pull/23759

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

> Parsing support for `f(**mapping)` support is indeed still missing.
>
> as lib2to3 is pending deprecation at this point, i'm not going to work on 
> this.  anyone is welcome to pick it up.  modifying the lib2to3 grammar, and 
> any related code, and adding a test is what's needed to parse that syntax.

I'd also agree, and not supporting to add features from now on. If someone 
really needs this to be added [and backported], please re-open the issue.

--
resolution:  -> out of date
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

> Every piece of code that uses either lib2to3 or a parser derived from it 
> (including parso and LibCST) will eventually not be able to upgrade the 
> parser because PEG can handle grammars that LL(k) can't.

Since these projects are external, depending on the functionality they are 
free-to-roll their own parser implementations or make hacks to pass away 
things. Or fork the Grammar/python.gram to preserve all tokens and generate a 
Python parser from it.


> If people are interested in using ast.parse extensions as a replacement for 
> lib2to3, I suggest discussing at

I don't quite get what you are proposing here, 

>I propose implementing an optional pass over the parse tree that records 
>lib2to3's "prefix" with each leaf node. The interface would be something like:

How would you do that? By augmenting the AST with the information retrieved 
from tokens? If so, check this out; 
https://github.com/leo-editor/leo-editor/blob/master/leo/core/leoAst.py and 
asttokens.

Also, please move the discussion to somewhere else (like discuss.python.org 
etc.) since this is not the ideal place to discuss and people might be 
distracted! (feel free to cc me where you move the discussion)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Peter Ludemann]

Peter Ludemann  added the comment:

Every piece of code that uses either lib2to3 or a parser derived from it 
(including parso and LibCST) will eventually not be able to upgrade the parser 
because PEG can handle grammars that LL(k) can't. That's why I proposed adding 
some functionality to ast.parse, to make the whitespace and token information 
easily available - this seems to be what @BTaskaya says is "easy" (maybe they 
mean it's easy using LibCST? It seems to be fiddly using ast.parse). The 
alternative is that all these projects (black, LibCST, yapf, etc.) will have to 
roll their own solutions, which doesn't seem a very productive use of people's 
time and makes version upgrades slow.

If people are interested in using ast.parse extensions as a replacement for 
lib2to3, I suggest discussing at 
https://mail.python.org/archives/list/python-id...@python.org/thread/X2HJ6I6XLIGRZDB27HRHIVQC3RXNZAY4/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

I don't see the point of augmenting the ast.parse, since we already have 
variants of proper CST implementations outside the core python. Such as 
github.com/davidhalter/parso/ or LibCST. 

Also for basic refactorings, it is so easy to use tokens for the refactoring 
and AST for the analysis! Even the ast.unparse() can be partially used (like 
first finding the related segment of the code through AST analysis, building 
the corresponding variant, unparsing it, finding the region of related tokens 
in the source code and replacing them). There are also quite a few libraries 
for using tokenize in different purposes (or wrappers) such as 
https://github.com/asottile/tokenize-rt or github.com/isidentical/brm.

--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Peter Ludemann]

Peter Ludemann  added the comment:

I made a suggestion for augmenting ast.parse with some of lib2to3's features; 
but nobody seemed interested. 

RIP lib2to3. Like many pieces of software, it was used for far more than for 
what it was originally intended.

https://mail.python.org/archives/list/python-id...@python.org/thread/X2HJ6I6XLIGRZDB27HRHIVQC3RXNZAY4/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Parsing support for `f(**mapping)` support is indeed still missing.

as lib2to3 is pending deprecation at this point, i'm not going to work on this. 
 anyone is welcome to pick it up.  modifying the lib2to3 grammar, and any 
related code, and adding a test is what's needed to parse that syntax.

--
assignee: gregory.p.smith -> 
stage: patch review -> needs patch
versions: +Python 3.10 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Support for `:=` is in, are we still lacking `f(**not x)` support?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Gregory P. Smith]

Gregory P. Smith  added the comment:

master/3.9 changeset:
https://github.com/python/cpython/commit/3c3aa4516c70753de06bb142b6793d01330fcf0f

3.8 changeset: 
https://github.com/python/cpython/commit/1098671e4e5ec1513247f05598158eaa3428c5be

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset 96c5f5a3a3fabf43e8114d0dbc30bed409da1ba6 by Tim Hatch in branch 
'3.7':
[3.7] bpo-36541: lib2to3: Support named assignment expressions (GH-12702) 
(GH-19317)
https://github.com/python/cpython/commit/96c5f5a3a3fabf43e8114d0dbc30bed409da1ba6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Tim Hatch]

Change by Tim Hatch :


--
pull_requests: +18682
pull_request: https://github.com/python/cpython/pull/19317

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 9.0 -> 10.0
pull_requests: +18680
pull_request: https://github.com/python/cpython/pull/19315

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[fireattack]

Change by fireattack :


--
nosy: +fireattack

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus

[Gregory P. Smith]

Change by Gregory P. Smith :


--
title: Make lib2to3 grammar more closely match Python -> Make lib2to3 grammar 
better match Python, support the := walrus

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com