Re: [Python-Dev] Put token information in one place

2017-05-31 Thread Guido van Rossum
On Wed, May 31, 2017 at 4:00 AM, Serhiy Storchaka wrote: > [...] Can pgen be implemented in Python? > Yes, an implementation already exists in the stdlib under lib2to3/pgen2/... -- --Guido van Rossum (python.org/~guido) ___

Re: [Python-Dev] Put token information in one place

2017-05-31 Thread Serhiy Storchaka
31.05.17 20:58, Brett Cannon пише: I assume there's a build rule for Python/graminit.c and porting pgen to Python would require Python be installed to do a build from scratch. Have we made that a requirement yet? If so then rewriting pgen in Python would make it easier to maintain (although

Re: [Python-Dev] Put token information in one place

2017-05-31 Thread Brett Cannon
On Wed, 31 May 2017 at 04:01 Serhiy Storchaka wrote: > Currently when you add a new token you need to change a couple of files: > > * Include/token.h > * _PyParser_TokenNames in Parser/tokenizer.c > * PyToken_OneChar(), PyToken_TwoChars() or PyToken_ThreeChars() in >

[Python-Dev] Put token information in one place

2017-05-31 Thread Serhiy Storchaka
Currently when you add a new token you need to change a couple of files: * Include/token.h * _PyParser_TokenNames in Parser/tokenizer.c * PyToken_OneChar(), PyToken_TwoChars() or PyToken_ThreeChars() in Parser/tokenizer.c * Lib/token.py (generated from Include/token.h) * EXACT_TOKEN_TYPES in