12-Sep-2013 12:05, Jacob Carlborg пишет:
On 2013-09-11 17:01, Dicebot wrote:
std.d.lexer is standard module for lexing D code, written by Brian Schott
Finally :)
* How does it handler errors, just returns TokenType.invalid?
* Personally I think the module is too big. I would go with:
- std.d.lexer.token
- std.d.lexer.tokentype
These could be one module. There is really no meaningful way to use
token type separately from token.
- std.d.lexer.lexer - contains the rest
- std.d.lexer.config - IterationStyle, TokenStyle, LexerConfig
Contrary I see this break down pointless - do you really want to use
config without the lexer?
- CircularRange, StringCache, possibly put somewhere else. I assume this
can be used for other things than lexing?
- Trie related code, same as above
No good public interface defined is the reason. Basically the same as
with Trie in the new std.uni module - needs its own review.
* I see that errorMessage throws an exception. Do we really want that? I
would except it just returns an invalid token.
If we do decide it should throw, it should absolutely _not_ throw a
plain Exception. Create a new type, LexException or similar. I hate when
code throws plain Exceptions, it makes it useless to catch them.
I would also expect this LexException to contain a Token. It shouldn't
be needed to parse the exception message to get line and column
information.
Better yet to have a std exception hierarchy... so that all parsing
modules can be tied to ParseException. So this needs to be resolved in a
forward-compatible way.
--
Dmitry Olshansky
