On Friday, 3 August 2012 at 14:49:55 UTC,
[email protected]
If I may add, there are several possitilities here:
1- a real slice of the input range
2- a slice of the input range created with .save and
takeExactly
3- a slice allocated in GC memory by the lexer
4- a slice of memory owned by the lexer, which is reused for
the next
token (thus, the next call to popFront invalidates the token).
5- a slice of memory from a lookup table.
#6 None. Just provide information of token type and location
(i.e. for syntax highlighting).