On 10/06/2013 10:18 AM, Sönke Ludwig wrote:
I also see no fundamental reason why the API forbids extension for shared sting tables or table-less lexing.
The current API requires to copy slices of the const(ubyte)[] input to string values in every token. This can't be done efficiently without a
string table. But a string table is unnecessary for many use-cases, so the API has a built-in performance/memory issue.