Am 22.08.2014 18:13, schrieb Sönke Ludwig:
Am 22.08.2014 17:47, schrieb Jacob Carlborg:
* Opening braces should be put on their own line to follow Phobos style
guides
Will do.
* I'm wondering about the assert in lexer.d, line 160. What happens if
two invalid tokens after each other occur?
There are actually no invalid tokens at all, the "invalid" enum value is
only used to denote that no token is currently stored in _front. If
readToken() doesn't throw, there will always be a valid token.
Renamed from "invalid" to "none" now to avoid confusion ->
* I think we have talked about this before, when reviewing D lexers. I'm
thinking of how to handle invalid data. Is it the best solution to throw
an exception? Would it be possible to return an error token and have the
client decide what to do about? Shouldn't it be possible to build a JSON
validator on this?
That would indeed be a possibility, it's how I used to handle it in my
private version of std.lexer, too. It could also be made a compile time
option.
and an additional "error" kind has been added, which implements the
above. Enabled using LexOptions.noThrow.
* The lexer seems to always convert JSON types to their native D types,
is that wise to do? That's unnecessary if you're implementing syntax
highlighting
It's basically the same trade-off as for unescaping string literals. For
"string" inputs, it would be more efficient to just store a slice, but
for generic input ranges it avoids the otherwise needed allocation. The
proposed flag could make an improvement here, too.
