Am 12.09.2013 10:15, schrieb Brian Schott:
I got some time to work on the lexer this evening. Changeset
here:
https://github.com/Hackerpilot/phobos/commit/9bdb7f97bb8021f3b0d0291896b8fe21a6fead23#std/d/lexer.d
The DDoc page has moved here:
http://hackerpilot.github.io/experimental/std_lexer/phobos/std_d_lexer.html
* There are a few more unit tests now
* bitAnd renamed to amp
* slice rename to dotdot
* Much more cross-referencing in the doc comments
* Start line and column can be specified in the lexer config
problem: many occurences of the same string
you should use constants for the tokens (and others)
string asm_token = "asm";
...
immutable(string[TokenType.max + 1]) tokenValues = [
...
asm_token
...
]
and reuse these constants in your "optimization"
maybe you can replace these lines with something getting feed with
asm_token and give the same result but without 'a' and "sm" as splitted
and different magic values - maybe a nice template or subrange...
case 'a': if (input[1..$].equal("sm")) return TokenType.asm_; else
...
break;
and in your unit tests
for example
on auto expected =
