Am I misunderstanding a tokenize parse rule or is this an error:
###
def tok(s):
import tokenize
from StringIO import StringIO
t = StringIO(s).readline
for ti in tokenize.generate_tokens(t):
print ti
tok("'''quote: \''''")
###
produces
(3, "'''quote: '''", (1, 0), (1, 13), "'''quote: ''''")
(52, "'", (1, 13), (1, 14), "'''quote: ''''")
(0, '', (2, 0), (2, 0), '')
It's taking the escaped quote to be one of the closing triple quotes and then
tokenizing off the last quote. I expected it to give me a single quote only and
then the end-of-record:
quote: \'
/Chris
_______________________________________________
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
