On Saturday, 7 July 2012 at 20:04:21 UTC, Andrei Alexandrescu
wrote:
Doesn't ANTLR use full-fledged character-level LL(*) parsing
even in the tokenizer?
Since I didn't understand your question I assume that my
statement was somehow incorrect (likely because I made some wrong
assumptions about ANTLR). I didn't know about its existence until
today and still don't understand it completely. What I think I
understood is that it uses DFA for deciding which grammar rule to
apply instead of doing backtracking. I also think that it uses
DFA for low-level scanning (I'm not sure).
The idea to introduce DFA for both determining which rule to
apply and lexing of terminal symbols appeared to me much earlier,
and the suggestion to introduce them into Pegged is one of
options which I think could extremely improve performance.
