On Friday, 3 August 2012 at 14:49:55 UTC, [email protected]

If I may add, there are several possitilities here:
 1- a real slice of the input range
2- a slice of the input range created with .save and takeExactly
 3- a slice allocated in GC memory by the lexer
4- a slice of memory owned by the lexer, which is reused for the next
token (thus, the next call to popFront invalidates the token).
 5- a slice of memory from a lookup table.

#6 None. Just provide information of token type and location (i.e. for syntax highlighting).

Reply via email to