Re: DMD's lexer available on code.dlang.org

[Vadim Lopatin via Digitalmars-d-announce]

On Sunday, 4 January 2015 at 13:09:42 UTC, Rikki Cattermole wrote:
On 5/01/2015 2:07 a.m., Daniel Murphy wrote:
I've created a dub package for the D version of DMD's lexer, 
generated
automatically from the C++ source.
github: https://github.com/yebblies/ddmd
dub: http://code.dlang.org/packages/ddmd

There are a few annoying limitations, such that it uses dmd's 
error
printing and allocation functions, and requires configuration 
through
'global'.

Here is an example program that uses the lexer:

======================

import std.stdio;
import std.file;

import ddmd.tokens;
import ddmd.lexer;

/////////////////////////

void main()
{
   Lexer.initLexer();

   string data = "void blah() {} // stuff";
   auto l = new Lexer("myfile", data.ptr, 0, data.length, 0, 
0);
   l.nextToken();
   do
   {
       printf("token: %s\n", l.token.toChars());
   }
   while (l.nextToken() != TOKeof);
}

======================

Prints:

token: void
token: blah
token: (
token: )
token: {
token: }

I saw that. I'm really looking forward to getting my teeth into 
it and doing some good old refactoring. Although that will be a 
while because of the auto generated thing.

I have a bit similar project - lexer for D written in D.
Written just based on "Lexical" documentation page.

https://github.com/buggins/ddc

Trying make it fast and to do as few memory allocations as 
possible.