On 10/22/10 14:17 CDT, Walter Bright wrote:
Tomek Sowiński wrote:
Interesting idea. Here's another: D will soon need bindings for CORBA,
Thrift, etc, so lexers will have to be written all over to grok
interface files. Perhaps a generic tokenizer which can be parametrized
with a lexical grammar would bring more ROI, I got a hunch D's
templates are strong enough to pull this off without any source code
generation ala JavaCC. The books I read on compilers say tokenization
is a solved problem, so the theory part on what a good abstraction
should be is done. What you think?
Lexers are so simple, it is less work to just build them by hand than
use lexer generator tools.
I wrote a C++ lexer. It wasn't at all easy except if I compared it
against the work necessary to build a full compiler.
Andrei
