Am 05.10.2013 02:24, schrieb Andrei Alexandrescu:
Instead of associating token types with small integers, we associate them with string addresses. (For efficiency we may use pointers to zero-terminated strings, but I don't think that's necessary).
would it be also more efficent to generate a big string out of the token list containing all tokes concatenated and use a generated string-slice for the associated string accesses?
imutable string generated_flat_token_stream = "...publicprivateclass..." "public" = generated_flat_token_stream[3..9] or would that kill caching on todays machines?
