Re: So... let's document dmd

2016-04-07 Thread Nick Sabalausky via Digitalmars-d

On 04/05/2016 05:40 PM, Walter Bright wrote:


I don't really understand why IDE makers don't use an actual programming
language for plugins


Programmer's Notepad 2 uses Python for plugins. Although the syntax 
highlighting is done through Scintilla which has it's own separate 
(non-programming-language IIRC) system for lex/parse.




Re: So... let's document dmd

2016-04-07 Thread Jacob Carlborg via Digitalmars-d

On 2016-04-07 09:28, Walter Bright wrote:


I suppose that being a compiler guy I wouldn't care to build a
half-assed one. It's not *that* hard to get it perfect, but you will
need to use a programming language to do it.


I usually feel the same, but that would require me modify the editor. 
I've many times thought about create a plugin which uses a proper lexer 
and parser but it's a lot of work for something that doesn't have a 
plugin API.


Also, GitHub is using the TextMate grammar to do syntax highlighting for 
D. If TextMate used a plugin for this instead, there would most likely 
be no one maintaining the the grammar GitHub is using.


--
/Jacob Carlborg


Re: So... let's document dmd

2016-04-07 Thread Walter Bright via Digitalmars-d

On 4/6/2016 11:36 PM, Jacob Carlborg wrote:

On 2016-04-07 02:53, Walter Bright wrote:


I doubt any of them could lex C/C++ correctly (trigraphs, macros,
backslash line splicing, wysiwyg string literals).


No. But it doesn't need to be perfect.


I suppose that being a compiler guy I wouldn't care to build a half-assed one. 
It's not *that* hard to get it perfect, but you will need to use a programming 
language to do it.




Re: So... let's document dmd

2016-04-07 Thread Jacob Carlborg via Digitalmars-d

On 2016-04-07 02:53, Walter Bright wrote:


I doubt any of them could lex C/C++ correctly (trigraphs, macros,
backslash line splicing, wysiwyg string literals).


No. But it doesn't need to be perfect. There's also nothing that tells 
what's wrong and what's right. For example, in TextMate the identifier 
in a function declaration is syntax highlighted, but in Eclipse it's not 
syntax highlighted.



Are you talking about an AST?

(Good luck doing an AST for C++ without a real C++ compiler front end!
Not that hard for D, though.)


Yes. In TextMate one describes a grammar and assigns different scopes to 
the different rules. These scopes are then used by the theme engine to 
syntax highlight the code. The scopes are also used to enable/disable 
different bundle commands. For example, pressing Ctrl+B inside a comment 
could make the selected text bold (replace the text with an HTML or 
Markdown tag) while outside comments something else happens.


TextMate also adds top level declarations like classes, structs and 
functions to a jump-list, for easy navigation.


Note that I'm not defending the editors, I would love if one could use a 
proper front end to do the lexing and parsing. I'm just saying it's not 
as easy as it sounded in the post I originally replied to.


--
/Jacob Carlborg


Re: So... let's document dmd

2016-04-06 Thread Walter Bright via Digitalmars-d

On 4/6/2016 1:09 AM, Jacob Carlborg wrote:

TextMate, Sublime and Atom all use the same syntax to describe the grammar for a
language. All of them supports plugins (to various degrees), but none of them
uses plugins to syntax highlight code, as far as I know.

I guess it's easier for most of us, I know you write a lexer in two days :), to
use the custom syntax to describe the grammar than to create a proper lexer and
parser.


I doubt any of them could lex C/C++ correctly (trigraphs, macros, backslash line 
splicing, wysiwyg string literals).


Even D has some issues - token string literals - that would defeat most grammar 
attempts.


A straightforward lexer (i.e. one not heavily optimized for speed) written in a 
reasonable high level language does not look all that different from BNF, and 
has the advantage of not being borked if something is not quite expressible in 
the grammar language.



> Note that a lexer is not enough, these grammars can describe how a
> function (and other constructs) look like.

Are you talking about an AST?

(Good luck doing an AST for C++ without a real C++ compiler front end! Not that 
hard for D, though.)


Re: So... let's document dmd

2016-04-06 Thread Timon Gehr via Digitalmars-d

On 06.04.2016 06:16, Walter Bright wrote:

On 4/5/2016 4:08 PM, Timon Gehr wrote:

Some additional care will need to be taken, e.g.

/+  

or even simply

/+ comment +/


that should not be a problem.



I think it can be for large files if the highlighter implements the 
described strategy in a literal fashion. /+ changes the lexer state at 
each position until the end of the file, and all the soon-to-be reusable 
information is replaced by irrelevant garbage by the reprocessing step. 
I think it is better to sometimes keep multiple lexer states per 
location, and maybe not necessarily run the lexer to completion each time.


Maybe this is one reason why some editors choose to close delimiters for 
you automatically by default.



One idea would be to handle all states simultaneously by keeping an 
efficient summary of the effect certain existing code snippets have on 
the lexer state. Summaries should be efficiently composable to compute 
effects (and hence the lexer state) quickly for any prefix of the code, 
while allowing code updates. I think it should be possible to fill in 
the details.


Re: So... let's document dmd

2016-04-06 Thread Jacob Carlborg via Digitalmars-d

On 2016-04-05 23:40, Walter Bright wrote:


Haha, MicroEmacs is written in D, so just use D code :-)

I don't really understand why IDE makers don't use an actual programming
language for plugins - I'd use javascript as there's a D implementation
of a javascript engine you can use. (Using D as a plugin language would
require making DLLs which is a pain or making a D interpreter, which
would be a major project (CTFE in theory should work for that, but it
consumes too much memory).)


TextMate, Sublime and Atom all use the same syntax to describe the 
grammar for a language. All of them supports plugins (to various 
degrees), but none of them uses plugins to syntax highlight code, as far 
as I know.


I guess it's easier for most of us, I know you write a lexer in two days 
:), to use the custom syntax to describe the grammar than to create a 
proper lexer and parser. Note that a lexer is not enough, these grammars 
can describe how a function (and other constructs) look like.


--
/Jacob Carlborg


Re: So... let's document dmd

2016-04-05 Thread Basile B. via Digitalmars-d

On Tuesday, 5 April 2016 at 13:47:55 UTC, Basile B. wrote:

[...]
by 4 or 5 softwares.


Believe or not, in 2012 I've registered a company with the same 
error (.softwares).

I don't know why...it hasnot been a commercial success.




Re: So... let's document dmd

2016-04-05 Thread Walter Bright via Digitalmars-d

On 4/5/2016 4:08 PM, Timon Gehr wrote:

Some additional care will need to be taken, e.g.

/+  

or even simply

/+ comment +/


that should not be a problem.



Re: So... let's document dmd

2016-04-05 Thread Walter Bright via Digitalmars-d

On 4/5/2016 7:59 PM, Basile B. wrote:

Actually numbers is the only part of the D lexer where errors can be detected.
There's no possible syntax errors otherwise.


D's lexer.d detects many errors. Search lexer.d for "error".



Re: So... let's document dmd

2016-04-05 Thread Basile B. via Digitalmars-d

On Tuesday, 5 April 2016 at 21:37:09 UTC, Walter Bright wrote:

On 4/5/2016 6:47 AM, Basile B. wrote:

Also lexing number doesn't need to be as accurate as the
front-end of the compiler (especially if the HL doesnt have a 
token type for the

illegal "lexem".


That is an interesting design point. If I was doing a 
highlighter, I'd highlight in red tokens that the compiler 
would reject, meaning I'd do the accurate number lexing.


Lexing numbers correctly is not trivial, but since the compiler 
lexer's implementation can be cut/pasted, it is trivial in 
practice.


Even if when the most naive lexer see a number and consumes until 
a blank, a symbol or an operator, it's clear that this can be 
done:


http://i.imgur.com/ehjps04.png

Actually numbers is the only part of the D lexer where errors can 
be detected.

There's no possible syntax errors otherwise.

But one thing I forget to say in my previous post is that lexing 
can be "multi-pass". The D front-end does everything in a single 
pass, for example it direclty detects tokPlusPlus or tokXorEqu, 
but actually a multi pass lexer can work in 3 sub phases:

1/ split words
2/ detects token families in the words; identifier, keyword, 
operator, etc.

3/ specialize tokens: tokOp.data == "++" -> tokPlusPlus



Re: So... let's document dmd

2016-04-05 Thread Timon Gehr via Digitalmars-d

On 05.04.2016 10:46, Walter Bright wrote:

On 1/16/2016 7:13 AM, H. S. Teoh via Digitalmars-d wrote:

I disagree. I think having the dmd itself (lexer, parser, etc.) as a
library (with the dmd executable merely being the default frontend) will
do D a lot of good.

For one thing, IDE's will no longer need to reinvent a D parser for the
purposes of syntax highlighting;


On the other hand, using lexer.d and parse.d as a guide to build your
own is a trivial undertaking. The Boost license is designed so this can
be done without worrying about making a derived work.

I looked into doing syntax highlighting for my editor, MicroEmacs. It
turns out it is not so easy to just use a compiler lexer/parser for it.
For one thing, the one used in the compiler is optimized for speed in a
forward pass through the text.

But a syntax highlighter in a text editor is different. Suppose I change
a character in the middle of a line. All the highlighting from that
point forward may change. And to figure out what that change is, the
parser/lexer has to start over from the beginning of the file! (Think
string literals, nested comments, quoted string literals, etc.) This
would make editing slow.
...


Also, tools might want to parse a more intuitive grammar in order to be 
able to give suggestions on how to adapt the code such that DMDs parser 
will accept it.



I surmised that a solution is to have each line in the editor be tagged
with a state to show the lexing state at the beginning of that line.
Then, when a character in a line changes, the lexer can be restarted
from that state, and it continues forward until the next line state
matches the new computed state. This would make it enormously faster.
...


Some additional care will need to be taken, e.g.

/+  

or even simply

/+ comment +/


But the compiler's lexer is not designed to be restartable in the middle.

Similarly, a source code formatter would be different in a different
way. It would need, for example, extra information about comments, token
start/end positions, etc. Such data collection would be irrelevant to a
compiler, and would slow it down and consume unneeded memory.





Re: So... let's document dmd

2016-04-05 Thread Walter Bright via Digitalmars-d

On 4/5/2016 4:34 AM, Jacob Carlborg wrote:

I agree very much with this. Unfortunately many (most?) editors use their own
weird syntax to describe grammars for syntax highlighting and cannot use a
proper lexer/parser.



Haha, MicroEmacs is written in D, so just use D code :-)

I don't really understand why IDE makers don't use an actual programming 
language for plugins - I'd use javascript as there's a D implementation of a 
javascript engine you can use. (Using D as a plugin language would require 
making DLLs which is a pain or making a D interpreter, which would be a major 
project (CTFE in theory should work for that, but it consumes too much memory).)




Re: So... let's document dmd

2016-04-05 Thread Walter Bright via Digitalmars-d

On 4/5/2016 6:47 AM, Basile B. wrote:

Also lexing number doesn't need to be as accurate as the
front-end of the compiler (especially if the HL doesnt have a token type for the
illegal "lexem".


That is an interesting design point. If I was doing a highlighter, I'd highlight 
in red tokens that the compiler would reject, meaning I'd do the accurate number 
lexing.


Lexing numbers correctly is not trivial, but since the compiler lexer's 
implementation can be cut/pasted, it is trivial in practice.


Re: So... let's document dmd

2016-04-05 Thread Basile B. via Digitalmars-d

On Tuesday, 5 April 2016 at 08:46:30 UTC, Walter Bright wrote:

On 1/16/2016 7:13 AM, H. S. Teoh via Digitalmars-d wrote:
I disagree. I think having the dmd itself (lexer, parser, 
etc.) as a
library (with the dmd executable merely being the default 
frontend) will

do D a lot of good.

For one thing, IDE's will no longer need to reinvent a D 
parser for the

purposes of syntax highlighting;


On the other hand, using lexer.d and parse.d as a guide to 
build your own is a trivial undertaking. The Boost license is 
designed so this can be done without worrying about making a 
derived work.


I looked into doing syntax highlighting for my editor, 
MicroEmacs. It turns out it is not so easy to just use a 
compiler lexer/parser for it. For one thing, the one used in 
the compiler is optimized for speed in a forward pass through 
the text.


But a syntax highlighter in a text editor is different. Suppose 
I change a character in the middle of a line. All the 
highlighting from that point forward may change. And to figure 
out what that change is, the parser/lexer has to start over 
from the beginning of the file! (Think string literals, nested 
comments, quoted string literals, etc.) This would make editing 
slow.


This is how CE highlither works. The lexer used to highlight 
processes line by line. For each line infos about the previous 
line are available (nested comments count, region kind like 
quoted string, raw quoted string, asm, etc).


Also keyword detection might use different dictionaries, up to 3 
(one for the keywords, another for special keywords like 
__FILE__, a third for asm opcodes). Operators doesn't need to 
have a special token (tokxorequ, toplusplus, etc) there's just 
one. Also lexing number doesn't need to be as accurate as the 
front-end of the compiler (especially if the HL doesnt have a 
token type for the illegal "lexem".


Another big difference is that the lexer used by an highlighter 
doesn't store the identifier associated to a token.


Using the front-end (or even libdparse) would require a 
modularisation. But lexing is not hard so I don't think it's 
worse. At last it would only be used by 4 or 5 softwares.




Re: So... let's document dmd

2016-04-05 Thread Jacob Carlborg via Digitalmars-d

On 2016-01-16 16:13, H. S. Teoh via Digitalmars-d wrote:


I disagree. I think having the dmd itself (lexer, parser, etc.) as a
library (with the dmd executable merely being the default frontend) will
do D a lot of good.

For one thing, IDE's will no longer need to reinvent a D parser for the
purposes of syntax highlighting; they can reuse the actual parser the
compiler itself uses, and thus actually be correct (whereas their own
parser may actually only parse a subset of D correctly).  They will also
be able to support instant feedback as the user types in D code
(highlight syntax errors on the fly, etc.).


I agree very much with this. Unfortunately many (most?) editors use 
their own weird syntax to describe grammars for syntax highlighting and 
cannot use a proper lexer/parser.


--
/Jacob Carlborg


Re: So... let's document dmd

2016-04-05 Thread Walter Bright via Digitalmars-d

On 1/16/2016 7:13 AM, H. S. Teoh via Digitalmars-d wrote:

I disagree. I think having the dmd itself (lexer, parser, etc.) as a
library (with the dmd executable merely being the default frontend) will
do D a lot of good.

For one thing, IDE's will no longer need to reinvent a D parser for the
purposes of syntax highlighting;


On the other hand, using lexer.d and parse.d as a guide to build your own is a 
trivial undertaking. The Boost license is designed so this can be done without 
worrying about making a derived work.


I looked into doing syntax highlighting for my editor, MicroEmacs. It turns out 
it is not so easy to just use a compiler lexer/parser for it. For one thing, the 
one used in the compiler is optimized for speed in a forward pass through the text.


But a syntax highlighter in a text editor is different. Suppose I change a 
character in the middle of a line. All the highlighting from that point forward 
may change. And to figure out what that change is, the parser/lexer has to start 
over from the beginning of the file! (Think string literals, nested comments, 
quoted string literals, etc.) This would make editing slow.


I surmised that a solution is to have each line in the editor be tagged with a 
state to show the lexing state at the beginning of that line. Then, when a 
character in a line changes, the lexer can be restarted from that state, and it 
continues forward until the next line state matches the new computed state. This 
would make it enormously faster.


But the compiler's lexer is not designed to be restartable in the middle.

Similarly, a source code formatter would be different in a different way. It 
would need, for example, extra information about comments, token start/end 
positions, etc. Such data collection would be irrelevant to a compiler, and 
would slow it down and consume unneeded memory.




Re: So... let's document dmd

2016-04-01 Thread Andrei Alexandrescu via Digitalmars-d
Alright, we're on! Entry point ain't sexy and the pages need all work, 
but it's a start. http://dlang.org/dmd-prerelease/ -- Andrei


Re: So... let's document dmd

2016-04-01 Thread Andrei Alexandrescu via Digitalmars-d

On 01/14/2016 11:53 PM, Andrei Alexandrescu wrote:

https://github.com/D-Programming-Language/dlang.org/pull/1196
https://github.com/D-Programming-Language/dmd/pull/5352

Destroy!!

Andrei


Finally got around to updating this. To make it work, please review and 
pull https://github.com/D-Programming-Language/dmd/pull/5617 and then 
https://github.com/D-Programming-Language/dlang.org/pull/1196.


BTW I'd forgotten we set up undefined macros to emit  in the generated html. That has been of great help.



Andrei



Re: So... let's document dmd

2016-01-16 Thread tsbockman via Digitalmars-d

On Saturday, 16 January 2016 at 07:52:11 UTC, Jack Stouffer wrote:
dmd is now in D; theoretically that should allow for other 
projects to import from it like a normal D project. So why not 
make all of the ddmd modules available from any code that is 
complied with it, i.e. just like Phobos?


The parser was just an example, there are any number of things 
that one could use from the code base.


That would be cool in theory, but it would likely become a major 
drag on development of the compiler itself.


Once you start promoting the compiler's internals as part of the 
standard library, you can expect sooner or later that pressure 
will mount to bring them up to the same high standards of API 
design and stability as Phobos.


While this process would be beneficial to the overall quality of 
the DMD code being exposed, it would also turn internal 
refactorings into public breaking changes. I really doubt that's 
a good trade-off at this point in DMD's development.


Re: So... let's document dmd

2016-01-16 Thread H. S. Teoh via Digitalmars-d
On Sat, Jan 16, 2016 at 08:35:40AM +, tsbockman via Digitalmars-d wrote:
> On Saturday, 16 January 2016 at 07:52:11 UTC, Jack Stouffer wrote:
> >dmd is now in D; theoretically that should allow for other projects
> >to import from it like a normal D project. So why not make all of the
> >ddmd modules available from any code that is complied with it, i.e.
> >just like Phobos?
> >
> >The parser was just an example, there are any number of things that
> >one could use from the code base.
> 
> That would be cool in theory, but it would likely become a major drag
> on development of the compiler itself.
> 
> Once you start promoting the compiler's internals as part of the
> standard library, you can expect sooner or later that pressure will
> mount to bring them up to the same high standards of API design and
> stability as Phobos.
> 
> While this process would be beneficial to the overall quality of the
> DMD code being exposed, it would also turn internal refactorings into
> public breaking changes. I really doubt that's a good trade-off at
> this point in DMD's development.

I disagree. I think having the dmd itself (lexer, parser, etc.) as a
library (with the dmd executable merely being the default frontend) will
do D a lot of good.

For one thing, IDE's will no longer need to reinvent a D parser for the
purposes of syntax highlighting; they can reuse the actual parser the
compiler itself uses, and thus actually be correct (whereas their own
parser may actually only parse a subset of D correctly).  They will also
be able to support instant feedback as the user types in D code
(highlight syntax errors on the fly, etc.).

This will also help with writing various tools that operate on D code,
e.g., pretty-printers like dfmt won't need to roll its own lexer/parser
anymore. Many other tools also become possible (automatic refactorings,
function call tree analysis, dependency analysis, etc.).

You could even use libdmd in your own D code, thus being able to have
the equivalent of the `eval` available in many scripting languages
(except that it will have native performance(!)).

The problem of dmd refactorings becoming public breaking changes is not
as big a problem if the public API is properly designed. If the compiler
code itself is refactored to interface between its sub-components via
cleanly-designed APIs with proper encapsulations, I don't see that it
will become prohibitively difficult to minimize public API changes
without hampering compiler development efforts. This is the way to go
for better compiler code quality anyway, so even if libdmd never
happens, the code should *still* be refactored in this way.

Besides, I don't think anyone is sanely suggesting that we allow dmd
source files to be free-for-all externally-importable modules. The more
likely scenario is that there will be a clearly-defined external
interface that is unlikely to change much, whereas the implementation
underneath is still free to be changed, refactored, reorganized,
rewritten, etc..


T

-- 
BREAKFAST.COM halted...Cereal Port Not Responding. -- YHL


Re: So... let's document dmd

2016-01-16 Thread tsbockman via Digitalmars-d

On Saturday, 16 January 2016 at 15:13:28 UTC, H. S. Teoh wrote:
I disagree. I think having the dmd itself (lexer, parser, etc.) 
as a
library (with the dmd executable merely being the default 
frontend) will

do D a lot of good.

For one thing, IDE's will no longer need to reinvent a D parser 
for the purposes of syntax highlighting; they can reuse the 
actual parser the compiler itself uses, and thus actually be 
correct (whereas their own parser may actually only parse a 
subset of D correctly).  They will also be able to support 
instant feedback as the user types in D code (highlight syntax 
errors on the fly, etc.).


This will also help with writing various tools that operate on 
D code, e.g., pretty-printers like dfmt won't need to roll its 
own lexer/parser anymore. Many other tools also become possible 
(automatic refactorings, function call tree analysis, 
dependency analysis, etc.).


You could even use libdmd in your own D code, thus being able 
to have
the equivalent of the `eval` available in many scripting 
languages

(except that it will have native performance(!)).

The problem of dmd refactorings becoming public breaking 
changes is not as big a problem if the public API is properly 
designed. If the compiler code itself is refactored to 
interface between its sub-components via cleanly-designed APIs 
with proper encapsulations, I don't see that it will become 
prohibitively difficult to minimize public API changes without 
hampering compiler development efforts. This is the way to go 
for better compiler code quality anyway, so even if libdmd 
never happens, the code should *still* be refactored in this 
way.


Besides, I don't think anyone is sanely suggesting that we 
allow dmd source files to be free-for-all externally-importable 
modules. The more likely scenario is that there will be a 
clearly-defined external interface that is unlikely to change 
much, whereas the implementation underneath is still free to be 
changed, refactored, reorganized, rewritten, etc..



T


You're sort of making my point for me.

I fully agree about all the benefits of making the lexer, parser, 
etc. available as a library.


HOWEVER, as you say, this would first require a large refactoring 
of the code, to make it suitable for use as a library. Time spent 
on that refactoring is time that could have been spent improving 
the compiler in other ways.


D is short of development resources as it is; I think fixing the 
many bugs in the compiler and fleshing out important complex 
features like C++ inter-op is of more value to the language in 
the long run.


Also worth mentioning, is that the SDC project was actually 
designed from the beginning for the use you're advocating. I'm 
sure deadalnix would love to have more help with that...


Re: So... let's document dmd

2016-01-16 Thread Daniel Murphy via Digitalmars-d

On 17/01/2016 2:29 AM, Joakim wrote:


While this would all be nice in principle, the reality is that dmd is
mostly worked on by two people these days
(https://github.com/D-Programming-Language/dmd/graphs/contributors) and
they're unlikely to refactor ddmd to put forth a libdmd.  It will depend
on someone caring enough to take it up, as Daniel did with the port of
the frontend to D.  As tsbockman said, I don't think we're at the stage
where anybody will put in that much effort into this.


I've already spent a huge amount of time on refactoring the frontend, 
there's just so much more to do.  More important than allowing new 
third-party uses of the frontend, is that well defined interfaces will 
make maintaining GDC and LDC less tricky.  Having to maintain a stable 
API would probably hurt more than it helps at this point.


Re: So... let's document dmd

2016-01-16 Thread Joakim via Digitalmars-d

On Saturday, 16 January 2016 at 15:13:28 UTC, H. S. Teoh wrote:
On Sat, Jan 16, 2016 at 08:35:40AM +, tsbockman via 
Digitalmars-d wrote:
On Saturday, 16 January 2016 at 07:52:11 UTC, Jack Stouffer 
wrote:
>dmd is now in D; theoretically that should allow for other 
>projects to import from it like a normal D project. So why 
>not make all of the ddmd modules available from any code that 
>is complied with it, i.e. just like Phobos?

>
>The parser was just an example, there are any number of 
>things that one could use from the code base.


That would be cool in theory, but it would likely become a 
major drag on development of the compiler itself.


Once you start promoting the compiler's internals as part of 
the standard library, you can expect sooner or later that 
pressure will mount to bring them up to the same high 
standards of API design and stability as Phobos.


While this process would be beneficial to the overall quality 
of the DMD code being exposed, it would also turn internal 
refactorings into public breaking changes. I really doubt 
that's a good trade-off at this point in DMD's development.


I disagree. I think having the dmd itself (lexer, parser, etc.) 
as a
library (with the dmd executable merely being the default 
frontend) will

do D a lot of good.

For one thing, IDE's will no longer need to reinvent a D parser 
for the purposes of syntax highlighting; they can reuse the 
actual parser the compiler itself uses, and thus actually be 
correct (whereas their own parser may actually only parse a 
subset of D correctly).  They will also be able to support 
instant feedback as the user types in D code (highlight syntax 
errors on the fly, etc.).


This will also help with writing various tools that operate on 
D code, e.g., pretty-printers like dfmt won't need to roll its 
own lexer/parser anymore. Many other tools also become possible 
(automatic refactorings, function call tree analysis, 
dependency analysis, etc.).


You could even use libdmd in your own D code, thus being able 
to have
the equivalent of the `eval` available in many scripting 
languages

(except that it will have native performance(!)).

The problem of dmd refactorings becoming public breaking 
changes is not as big a problem if the public API is properly 
designed. If the compiler code itself is refactored to 
interface between its sub-components via cleanly-designed APIs 
with proper encapsulations, I don't see that it will become 
prohibitively difficult to minimize public API changes without 
hampering compiler development efforts. This is the way to go 
for better compiler code quality anyway, so even if libdmd 
never happens, the code should *still* be refactored in this 
way.


Besides, I don't think anyone is sanely suggesting that we 
allow dmd source files to be free-for-all externally-importable 
modules. The more likely scenario is that there will be a 
clearly-defined external interface that is unlikely to change 
much, whereas the implementation underneath is still free to be 
changed, refactored, reorganized, rewritten, etc..


While this would all be nice in principle, the reality is that 
dmd is mostly worked on by two people these days 
(https://github.com/D-Programming-Language/dmd/graphs/contributors) and they're unlikely to refactor ddmd to put forth a libdmd.  It will depend on someone caring enough to take it up, as Daniel did with the port of the frontend to D.  As tsbockman said, I don't think we're at the stage where anybody will put in that much effort into this.


Re: So... let's document dmd

2016-01-15 Thread Jack Stouffer via Digitalmars-d

On Saturday, 16 January 2016 at 05:11:57 UTC, Joakim wrote:
The ddmd lexer is already on dub, just have to add the parser 
next:


http://code.dlang.org/packages/ddmd


You missed the main point of my question.


Re: So... let's document dmd

2016-01-15 Thread Joakim via Digitalmars-d

On Friday, 15 January 2016 at 23:58:32 UTC, Jack Stouffer wrote:
On Friday, 15 January 2016 at 04:53:16 UTC, Andrei Alexandrescu 
wrote:

https://github.com/D-Programming-Language/dlang.org/pull/1196
https://github.com/D-Programming-Language/dmd/pull/5352

Destroy!!

Andrei


Since dmd is now in D, is there any reason to not allow people 
to import from it while using dmd? Wouldn't it make anyone's 
life who wanted to parse D code easier by just doing


import ddmd.parser; (or what ever it is)

and use the actual parser?


The ddmd lexer is already on dub, just have to add the parser 
next:


http://code.dlang.org/packages/ddmd


Re: So... let's document dmd

2016-01-15 Thread tsbockman via Digitalmars-d

On Saturday, 16 January 2016 at 05:29:03 UTC, Jack Stouffer wrote:

On Saturday, 16 January 2016 at 05:11:57 UTC, Joakim wrote:
The ddmd lexer is already on dub, just have to add the parser 
next:


http://code.dlang.org/packages/ddmd


You missed the main point of my question.


I'm confused as to your point as well. Perhaps you could rephrase?


Re: So... let's document dmd

2016-01-15 Thread Joakim via Digitalmars-d

On Saturday, 16 January 2016 at 05:36:23 UTC, tsbockman wrote:
On Saturday, 16 January 2016 at 05:29:03 UTC, Jack Stouffer 
wrote:

On Saturday, 16 January 2016 at 05:11:57 UTC, Joakim wrote:
The ddmd lexer is already on dub, just have to add the parser 
next:


http://code.dlang.org/packages/ddmd


You missed the main point of my question.


I'm confused as to your point as well. Perhaps you could 
rephrase?


I think he's trying to emphasize that he'd like it to come with 
dmd.  There was talk of putting the lexer/parser into phobos, but 
I believe dub was chosen as a staging ground for people to play 
with it first.  So if you want to use the lexer/parser, use it 
from dub for now, then push for its inclusion into phobos later.


Re: So... let's document dmd

2016-01-15 Thread Jack Stouffer via Digitalmars-d
On Friday, 15 January 2016 at 04:53:16 UTC, Andrei Alexandrescu 
wrote:

https://github.com/D-Programming-Language/dlang.org/pull/1196
https://github.com/D-Programming-Language/dmd/pull/5352

Destroy!!

Andrei


Since dmd is now in D, is there any reason to not allow people to 
import from it while using dmd? Wouldn't it make anyone's life 
who wanted to parse D code easier by just doing


import ddmd.parser; (or what ever it is)

and use the actual parser?


Re: So... let's document dmd

2016-01-15 Thread Jack Stouffer via Digitalmars-d

On Saturday, 16 January 2016 at 05:36:23 UTC, tsbockman wrote:
On Saturday, 16 January 2016 at 05:29:03 UTC, Jack Stouffer 
wrote:

On Saturday, 16 January 2016 at 05:11:57 UTC, Joakim wrote:
The ddmd lexer is already on dub, just have to add the parser 
next:


http://code.dlang.org/packages/ddmd


You missed the main point of my question.


I'm confused as to your point as well. Perhaps you could 
rephrase?


dmd is now in D; theoretically that should allow for other 
projects to import from it like a normal D project. So why not 
make all of the ddmd modules available from any code that is 
complied with it, i.e. just like Phobos?


The parser was just an example, there are any number of things 
that one could use from the code base.


So... let's document dmd

2016-01-14 Thread Andrei Alexandrescu via Digitalmars-d

https://github.com/D-Programming-Language/dlang.org/pull/1196
https://github.com/D-Programming-Language/dmd/pull/5352

Destroy!!

Andrei