DMD's lexer available on code.dlang.org
I've created a dub package for the D version of DMD's lexer, generated
automatically from the C++ source.
github: https://github.com/yebblies/ddmd
dub: http://code.dlang.org/packages/ddmd
There are a few annoying limitations, such that it uses dmd's error printing
and allocation functions, and requires configuration through 'global'.
Here is an example program that uses the lexer:
==
import std.stdio;
import std.file;
import ddmd.tokens;
import ddmd.lexer;
/
void main()
{
Lexer.initLexer();
string data = "void blah() {} // stuff";
auto l = new Lexer("myfile", data.ptr, 0, data.length, 0, 0);
l.nextToken();
do
{
printf("token: %s\n", l.token.toChars());
}
while (l.nextToken() != TOKeof);
}
==
Prints:
token: void
token: blah
token: (
token: )
token: {
token: }
Re: DMD's lexer available on code.dlang.org
On 5/01/2015 2:07 a.m., Daniel Murphy wrote:
I've created a dub package for the D version of DMD's lexer, generated
automatically from the C++ source.
github: https://github.com/yebblies/ddmd
dub: http://code.dlang.org/packages/ddmd
There are a few annoying limitations, such that it uses dmd's error
printing and allocation functions, and requires configuration through
'global'.
Here is an example program that uses the lexer:
==
import std.stdio;
import std.file;
import ddmd.tokens;
import ddmd.lexer;
/
void main()
{
Lexer.initLexer();
string data = "void blah() {} // stuff";
auto l = new Lexer("myfile", data.ptr, 0, data.length, 0, 0);
l.nextToken();
do
{
printf("token: %s\n", l.token.toChars());
}
while (l.nextToken() != TOKeof);
}
==
Prints:
token: void
token: blah
token: (
token: )
token: {
token: }
I saw that. I'm really looking forward to getting my teeth into it and
doing some good old refactoring. Although that will be a while because
of the auto generated thing.
Re: DMD's lexer available on code.dlang.org
"Rikki Cattermole" wrote in message news:m8be2m$1dlp$1...@digitalmars.com... I saw that. I'm really looking forward to getting my teeth into it and doing some good old refactoring. Although that will be a while because of the auto generated thing. There's plenty of refactoring to be done on the C++ side too.
Re: DMD's lexer available on code.dlang.org
On Sunday, 4 January 2015 at 13:07:34 UTC, Daniel Murphy wrote:
I've created a dub package for the D version of DMD's lexer,
generated automatically from the C++ source.
github: https://github.com/yebblies/ddmd
dub: http://code.dlang.org/packages/ddmd
There are a few annoying limitations, such that it uses dmd's
error printing and allocation functions, and requires
configuration through 'global'.
Here is an example program that uses the lexer:
==
import std.stdio;
import std.file;
import ddmd.tokens;
import ddmd.lexer;
/
void main()
{
Lexer.initLexer();
string data = "void blah() {} // stuff";
auto l = new Lexer("myfile", data.ptr, 0, data.length, 0, 0);
l.nextToken();
do
{
printf("token: %s\n", l.token.toChars());
}
while (l.nextToken() != TOKeof);
}
==
Prints:
token: void
token: blah
token: (
token: )
token: {
token: }
(sorry if you get this question too often)
How is DDMD as a whole going? Is it getting closer or are ongoing
DMD changes slowing it down too much?
Anyway, great to have the lexer on DUB, not going to use if for
now because I expect there to be changes, but I guess it may
eventually be useful for writing tools.
Re: DMD's lexer available on code.dlang.org
"Kiith-Sa" wrote in message news:nffxogzwpmayydyom...@forum.dlang.org... (sorry if you get this question too often) How is DDMD as a whole going? Is it getting closer or are ongoing DMD changes slowing it down too much? It's been sitting still for 8 nearly months because of https://github.com/braddr/d-tester/issues/39 and https://github.com/braddr/d-tester/issues/40 I don't mind getting asked, I just wish the answer would change. Anyway, great to have the lexer on DUB, not going to use if for now because I expect there to be changes, but I guess it may eventually be useful for writing tools. Hopefully lots of changes.
Re: DMD's lexer available on code.dlang.org
On 2015-01-04 14:07, Daniel Murphy wrote: I've created a dub package for the D version of DMD's lexer, generated automatically from the C++ source. This is very cool :), keep up to good work. -- /Jacob Carlborg
Re: DMD's lexer available on code.dlang.org
I've created a dub package for the D version of DMD's lexer, generated automatically from the C++ source. very cool. on a related note, have you considered sharing your translation tool (c++ -> D)? I completely understand if you would rather not of course. Laeeth.
Re: DMD's lexer available on code.dlang.org
On Sunday, 4 January 2015 at 17:27:57 UTC, Daniel Murphy wrote: "Kiith-Sa" wrote in message news:nffxogzwpmayydyom...@forum.dlang.org... (sorry if you get this question too often) How is DDMD as a whole going? Is it getting closer or are ongoing DMD changes slowing it down too much? It's been sitting still for 8 nearly months because of https://github.com/braddr/d-tester/issues/39 and https://github.com/braddr/d-tester/issues/40 I don't mind getting asked, I just wish the answer would change. Anyway, great to have the lexer on DUB, not going to use if for now because I expect there to be changes, but I guess it may eventually be useful for writing tools. Hopefully lots of changes. I can't really help with those, but FWIW I bumped the issues.
Re: DMD's lexer available on code.dlang.org
On 1/4/2015 5:07 AM, Daniel Murphy wrote: I've created a dub package for the D version of DMD's lexer, generated automatically from the C++ source. github: https://github.com/yebblies/ddmd dub: http://code.dlang.org/packages/ddmd Great! Thank you!
Re: DMD's lexer available on code.dlang.org
On 5/01/2015 2:39 a.m., Daniel Murphy wrote: "Rikki Cattermole" wrote in message news:m8be2m$1dlp$1...@digitalmars.com... I saw that. I'm really looking forward to getting my teeth into it and doing some good old refactoring. Although that will be a while because of the auto generated thing. There's plenty of refactoring to be done on the C++ side too. What I want to do is D side unfortunately. I just hate to see large lists in e.g. globals with constructors. When they can be dealt with a simple array and a bit of CTFE magic.
Re: DMD's lexer available on code.dlang.org
"Laeeth Isharc" wrote in message news:yzmwemaevaltcmkyw...@forum.dlang.org... on a related note, have you considered sharing your translation tool (c++ -> D)? I completely understand if you would rather not of course. The translation tool is available on github and is boost licensed. This pull request contains the latest version in src/magicport - https://github.com/D-Programming-Language/dmd/pull/3410 Please keep in mind that the tool makes a lot of assumptions about the C++ source that may not be valid for projects other than dmd. It's fairly easy to adapt to other projects, but it won't work on them out of the box.
Re: DMD's lexer available on code.dlang.org
"Daniel Murphy" wrote in message news:m8bdul$1dke$1...@digitalmars.com... I've created a dub package for the D version of DMD's lexer, generated automatically from the C++ source. github: https://github.com/yebblies/ddmd dub: http://code.dlang.org/packages/ddmd I've pushed a new version which should fix the 64-bit build errors.
Re: DMD's lexer available on code.dlang.org
On Monday, 5 January 2015 at 02:51:17 UTC, Daniel Murphy wrote: "Laeeth Isharc" wrote in message news:yzmwemaevaltcmkyw...@forum.dlang.org... on a related note, have you considered sharing your translation tool (c++ -> D)? I completely understand if you would rather not of course. The translation tool is available on github and is boost licensed. This pull request contains the latest version in src/magicport - https://github.com/D-Programming-Language/dmd/pull/3410 Please keep in mind that the tool makes a lot of assumptions about the C++ source that may not be valid for projects other than dmd. It's fairly easy to adapt to other projects, but it won't work on them out of the box. Thanks v much - this will be very helpful indeed.
Re: DMD's lexer available on code.dlang.org
"Laeeth Isharc" wrote in message news:vtgirvyjsalkzjvlz...@forum.dlang.org... Thanks v much - this will be very helpful indeed. Let me know if you have any questions about it.
Re: DMD's lexer available on code.dlang.org
On Sunday, 4 January 2015 at 13:09:42 UTC, Rikki Cattermole wrote:
On 5/01/2015 2:07 a.m., Daniel Murphy wrote:
I've created a dub package for the D version of DMD's lexer,
generated
automatically from the C++ source.
github: https://github.com/yebblies/ddmd
dub: http://code.dlang.org/packages/ddmd
There are a few annoying limitations, such that it uses dmd's
error
printing and allocation functions, and requires configuration
through
'global'.
Here is an example program that uses the lexer:
==
import std.stdio;
import std.file;
import ddmd.tokens;
import ddmd.lexer;
/
void main()
{
Lexer.initLexer();
string data = "void blah() {} // stuff";
auto l = new Lexer("myfile", data.ptr, 0, data.length, 0,
0);
l.nextToken();
do
{
printf("token: %s\n", l.token.toChars());
}
while (l.nextToken() != TOKeof);
}
==
Prints:
token: void
token: blah
token: (
token: )
token: {
token: }
I saw that. I'm really looking forward to getting my teeth into
it and doing some good old refactoring. Although that will be a
while because of the auto generated thing.
I have a bit similar project - lexer for D written in D.
Written just based on "Lexical" documentation page.
https://github.com/buggins/ddc
Trying make it fast and to do as few memory allocations as
possible.
Re: DMD's lexer available on code.dlang.org
It will be really cool when same package will be reused by DMD itself :P
Re: DMD's lexer available on code.dlang.org
On Tuesday, 6 January 2015 at 14:38:21 UTC, Dicebot wrote: It will be really cool when same package will be reused by DMD itself :P I believe ddmd has passed all tests on most platforms for a long time now, so there is nothing stopping those building from source from using ddmd now. :)
Re: DMD's lexer available on code.dlang.org
On 1/6/2015 1:37 AM, Vadim Lopatin wrote: I have a bit similar project - lexer for D written in D. Written just based on "Lexical" documentation page. https://github.com/buggins/ddc Trying make it fast and to do as few memory allocations as possible. Should also make it available on code.dlang.org
