Petri Lehtinen <pe...@digip.org> added the comment: tokenize processes a line at a time, and noticing that an ending triple quote is missing would mean reading the whole file in the worst case. As tokenize seems to work in a generator-like fashion, it's probably not desired to cache all the input to be able to restart from some previous line.
So, I'd go with documenting the behavior. ---------- nosy: +petri.lehtinen _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue12063> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
