Ammar Askar <am...@ammaraskar.com> added the comment:

Actually, echo implicitly puts a newline at the end. If you run with echo -n, 
this is the output:

$ echo -n 'print("a");print("b")' | python3 -m tokenize
1,0-1,5:            NAME           'print'
1,5-1,6:            OP             '('
1,6-1,9:            STRING         '"a"'
1,9-1,10:           OP             ')'
1,10-1,11:          OP             ';'
1,11-1,16:          NAME           'print'
1,16-1,17:          OP             '('
1,17-1,20:          STRING         '"b"'
1,20-1,21:          OP             ')'
2,0-2,0:            ENDMARKER      ''

No newline token present.

----------

_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue33766>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to