[Python-Dev] Lexical analysis and NEWLINE tokens

2005-10-06 Thread Matthew F. Barnes
I posted this question to python-help, but I think I have a better chance of getting the answer here. I'm looking for clarification on when NEWLINE tokens are generated during lexical analysis of Python source code. In particular, I'm confused about some of the top-level components in Python's gr

Re: [Python-Dev] Lexical analysis and NEWLINE tokens

2005-10-06 Thread Michael Hudson
"Matthew F. Barnes" <[EMAIL PROTECTED]> writes: > I posted this question to python-help, but I think I have a better chance > of getting the answer here. > > I'm looking for clarification on when NEWLINE tokens are generated during > lexical analysis of Python source code. In particular, I'm conf

Re: [Python-Dev] Lexical analysis and NEWLINE tokens

2005-10-06 Thread Guido van Rossum
I think it is a relic from the distant past, when the lexer did generate NEWLINE for every blank line. I think the only case where you can still get a NEWLINE by itself is in interactive mode. This code is extremely convoluted and may be buggy in end cases; this could explain why you get a continua

Re: [Python-Dev] Lexical analysis and NEWLINE tokens

2005-10-06 Thread Phillip J. Eby
At 07:36 AM 10/6/2005 -0500, Matthew F. Barnes wrote: >I posted this question to python-help, but I think I have a better chance >of getting the answer here. > >I'm looking for clarification on when NEWLINE tokens are generated during >lexical analysis of Python source code. If you're talking abou