On Saturday, 7 July 2012 at 20:27:14 UTC, Dmitry Olshansky wrote:
So my view on the matter: whether or not lexer is integrated is
two-fold question: notation vs implementation.
E.g. it may make regular lexer-like things a part of notation
thus having rules like:
Identifier -> Alpha (Alpha|Number|_)*
Yet it may or may use the same engine for _all_ _rules_, in
fact if implementation optimize things by ripping off regular
parts of grammar to small DFA engines it would make some kind
of lexer behind the scenes!
This is the implementation I have in mind as the only sane way to
parse PEG efficiently. Plus specializing DFA to only check for
those terminals which are allowed according to available
structural information at the moment of their invocation.
Anyway highly hybrid parsers are all the rage now, and reasons
are obvious - they runs fast and can be bend by some magic to
quite convoluted grammars of modern languages ;)