Tomek Sowiński wrote:
Interesting idea. Here's another: D will soon need bindings for CORBA, Thrift, etc, so lexers will have to be written all over to grok interface files. Perhaps a generic tokenizer which can be parametrized with a lexical grammar would bring more ROI, I got a hunch D's templates are strong enough to pull this off without any source code generation ala JavaCC. The books I read on compilers say tokenization is a solved problem, so the theory part on what a good abstraction should be is done. What you think?
Lexers are so simple, it is less work to just build them by hand than use lexer generator tools.
