New submission from Joshua Landau: By the docs,
Except at the beginning of a logical line or in string literals, the whitespace characters space, tab and formfeed can be used interchangeably to separate tokens. Whitespace is needed between two tokens only if their concatenation could otherwise be interpreted as a different token (e.g., ab is one token, but a b is two tokens). "_ if 1else _" should compile equivalently to "_ if 1 else _". The tokenize module does this correctly: import io import tokenize def print_tokens(string): tokens = tokenize.tokenize(io.BytesIO(string.encode("utf8")).readline) for token in tokens: print("{:12}{}".format(tokenize.tok_name[token.type], token.string)) print_tokens("_ if 1else _") #>>> ENCODING utf-8 #>>> NAME _ #>>> NAME if #>>> NUMBER 1 #>>> NAME else #>>> NAME _ #>>> ENDMARKER but it fails when compiled with, say, "compile", "eval" or "ast.parse". import ast compile("_ if 1else _", "", "eval") #>>> Traceback (most recent call last): #>>> File "", line 32, in <module> #>>> File "<string>", line 1 #>>> _ if 1else _ #>>> ^ #>>> SyntaxError: invalid token eval("_ if 1else _") #>>> Traceback (most recent call last): #>>> File "", line 40, in <module> #>>> File "<string>", line 1 #>>> _ if 1else _ #>>> ^ #>>> SyntaxError: invalid token ast.parse("_ if 1else _") #>>> Traceback (most recent call last): #>>> File "", line 48, in <module> #>>> File "/usr/lib/python3.4/ast.py", line 35, in parse #>>> return compile(source, filename, mode, PyCF_ONLY_AST) #>>> File "<unknown>", line 1 #>>> _ if 1else _ #>>> ^ #>>> SyntaxError: invalid token Further, some other forms work: 1 if 0b1else 0 #>>> 1 1 if 1jelse 0 #>>> 1 See http://stackoverflow.com/questions/23998026/why-isnt-this-a-syntax-error-in-python particularly, http://stackoverflow.com/a/23998128/1763356 for details. ---------- messages: 219614 nosy: Joshua.Landau priority: normal severity: normal status: open title: "_ if 1else _" does not compile type: compile error versions: Python 3.4 _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue21642> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com