I've been adding grammars to
https://mingodad.github.io/parsertl-playground/playground/ an "Yacc/Lex"
compatible wasm/online editor/tester and for grammars with big number of
soft keywords like PostgreSQL, Vitess, ... the resources usage in terms
of memory/time/size for the parser grows exponentially proportional to
the number of such tokens.
The Lemon parser generator has an option "%fallback" (see
https://sqlite.org/src/doc/trunk/doc/lemon.html#pfallback ) that reduce
the resources usage and also has less probabilities of generating conflicts.
I've implemented it on
https://mingodad.github.io/parsertl-playground/playground/ and I've got
good performance gains:
Statistics of Vitess SQL parser (be patient)(partially working):
====
build master parser -> Time taken : 87ms
parse user grammar -> Time taken : 5ms
start build user grammar -> Time taken : 2ms
build user parser -> Time taken : 1481ms
dump input parser tree -> Time taken : 10ms
====
Statistics of Vitess SQL parser with fallback (partially working):
====
build master parser -> Time taken : 34ms
parse user grammar -> Time taken : 3ms
start build user grammar -> Time taken : 0ms
build user parser -> Time taken : 446ms
dump input parser tree -> Time taken : 5ms
====
Would be nice to have the same implemented in bison.
What is the opinion of people on this list ?
Cheers !
