On 10/08/2013 05:05 PM, Andrei Alexandrescu wrote:
But no matter. My most significant bit is, we need a trie lexer generator ONLY from the token strings, no TK_XXX user-provided symbols necessary. If all we need is one language (D) this is a non-issue because the library writer provides the token definitions. If we need to support user-provided languages, having the library manage the string -> small integer mapping becomes essential.
It's good to get rid of the symbol names. You should try to map the strings onto an enum so that final switch works. final switch (t.type_) { case t!"<<": break; // ... }