Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-25 Thread Steven Schveighoffer
On Mon, 18 Jun 2012 13:53:43 -0400, Walter Bright newshou...@digitalmars.com wrote: On 6/18/2012 6:07 AM, Don Clugston wrote: On 17/06/12 00:37, Walter Bright wrote: On 6/14/2012 1:03 AM, Don Clugston wrote: It is for debug builds. Iain's data indicates that it's only a few % of the time

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-25 Thread Martin Nowak
On Mon, 18 Jun 2012 19:53:43 +0200, Walter Bright newshou...@digitalmars.com wrote: On 6/18/2012 6:07 AM, Don Clugston wrote: On 17/06/12 00:37, Walter Bright wrote: On 6/14/2012 1:03 AM, Don Clugston wrote: It is for debug builds. Iain's data indicates that it's only a few % of the time

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-19 Thread Kagamin
On Monday, 18 June 2012 at 17:54:40 UTC, Walter Bright wrote: Yeah, but I can't escape that lingering feeling that lexing is slow. I was fairly disappointed that asynchronously reading the source files didn't have a measurable effect most of the time. I don't even understand all this rage

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-19 Thread Kagamin
On Tuesday, 19 June 2012 at 01:47:27 UTC, Timon Gehr wrote: Parsing is not a huge issue. Depending on how powerful the language is, auto-completion may depend on full code analysis. Yep, pegged runs at compile time.

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-19 Thread dennis luehring
Am 19.06.2012 09:43, schrieb Kagamin: On Monday, 18 June 2012 at 17:54:40 UTC, Walter Bright wrote: Yeah, but I can't escape that lingering feeling that lexing is slow. I was fairly disappointed that asynchronously reading the source files didn't have a measurable effect most of the time. I

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-19 Thread dennis luehring
Am 18.06.2012 19:53, schrieb Walter Bright: On 6/18/2012 6:07 AM, Don Clugston wrote: On 17/06/12 00:37, Walter Bright wrote: On 6/14/2012 1:03 AM, Don Clugston wrote: It is for debug builds. Iain's data indicates that it's only a few % of the time taken on semantic1(). Do you have data that

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-19 Thread deadalnix
Le 16/06/2012 11:18, Iain Buclaw a écrit : On 13 June 2012 12:47, Iain Buclawibuc...@ubuntu.com wrote: On 13 June 2012 12:33, Kagamins...@here.lot wrote: On Wednesday, 13 June 2012 at 11:29:45 UTC, Kagamin wrote: The measurements should be done for modules being imported, not the module

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-19 Thread Iain Buclaw
On 16 June 2012 22:17, Guillaume Chatelet chatelet.guilla...@gmail.com wrote: So parsing time has taken quite a hit since I last did any reports on compilation speed of building phobos. So maybe my post about keeping import clean wasn't as irrelevant as I thought.

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-19 Thread deadalnix
Le 17/06/2012 00:41, Walter Bright a écrit : On 6/14/2012 11:58 PM, Don Clugston wrote: And we're well set up for parallel compilation. There's no shortage of things we can do to improve compilation time. The language is carefully designed, so that at least in theory all the passes could be

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-19 Thread deadalnix
Le 18/06/2012 19:53, Walter Bright a écrit : On 6/18/2012 6:07 AM, Don Clugston wrote: On 17/06/12 00:37, Walter Bright wrote: On 6/14/2012 1:03 AM, Don Clugston wrote: It is for debug builds. Iain's data indicates that it's only a few % of the time taken on semantic1(). Do you have data

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-18 Thread Don Clugston
On 17/06/12 00:37, Walter Bright wrote: On 6/14/2012 1:03 AM, Don Clugston wrote: It is for debug builds. Iain's data indicates that it's only a few % of the time taken on semantic1(). Do you have data that shows otherwise? Nothing recent, it's mostly from my C++ compiler testing. But you

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-18 Thread Walter Bright
On 6/18/2012 6:07 AM, Don Clugston wrote: On 17/06/12 00:37, Walter Bright wrote: On 6/14/2012 1:03 AM, Don Clugston wrote: It is for debug builds. Iain's data indicates that it's only a few % of the time taken on semantic1(). Do you have data that shows otherwise? Nothing recent, it's

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-18 Thread Daniel
On Monday, 18 June 2012 at 17:54:40 UTC, Walter Bright wrote: On 6/18/2012 6:07 AM, Don Clugston wrote: On 17/06/12 00:37, Walter Bright wrote: On 6/14/2012 1:03 AM, Don Clugston wrote: It is for debug builds. Iain's data indicates that it's only a few % of the time taken on semantic1(). Do

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-18 Thread Chris Cain
On Monday, 18 June 2012 at 18:05:59 UTC, Daniel wrote: Same here, I wish there were a standardized pre-lexed-token binary file-format, would benefit all text editors also, as they need to lex it anyway to perform color syntax highlighting. If I were to make my own language, I'd forego a

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-18 Thread Timon Gehr
On 06/19/2012 02:47 AM, Chris Cain wrote: On Monday, 18 June 2012 at 18:05:59 UTC, Daniel wrote: Same here, I wish there were a standardized pre-lexed-token binary file-format, would benefit all text editors also, as they need to lex it anyway to perform color syntax highlighting. If I were

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-16 Thread Iain Buclaw
On 13 June 2012 12:47, Iain Buclaw ibuc...@ubuntu.com wrote: On 13 June 2012 12:33, Kagamin s...@here.lot wrote: On Wednesday, 13 June 2012 at 11:29:45 UTC, Kagamin wrote: The measurements should be done for modules being imported, not the module being compiled. Something like this. ---

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-16 Thread Iain Buclaw
On 16 June 2012 10:18, Iain Buclaw ibuc...@ubuntu.com wrote: On 13 June 2012 12:47, Iain Buclaw ibuc...@ubuntu.com wrote: On 13 June 2012 12:33, Kagamin s...@here.lot wrote: On Wednesday, 13 June 2012 at 11:29:45 UTC, Kagamin wrote: The measurements should be done for modules being imported,

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-16 Thread Guillaume Chatelet
So parsing time has taken quite a hit since I last did any reports on compilation speed of building phobos. So maybe my post about keeping import clean wasn't as irrelevant as I thought. http://www.digitalmars.com/d/archives/digitalmars/D/Keeping_imports_clean_162890.html#N162890 -- Guillaume

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-16 Thread Walter Bright
On 6/14/2012 1:03 AM, Don Clugston wrote: It is for debug builds. Iain's data indicates that it's only a few % of the time taken on semantic1(). Do you have data that shows otherwise? Nothing recent, it's mostly from my C++ compiler testing. Yes, it is designed so you could just import a

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-16 Thread Walter Bright
On 6/14/2012 11:58 PM, Don Clugston wrote: And we're well set up for parallel compilation. There's no shortage of things we can do to improve compilation time. The language is carefully designed, so that at least in theory all the passes could be done in parallel. I've got the file reads in

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-15 Thread Don Clugston
On 14/06/12 10:10, Jonathan M Davis wrote: On Thursday, June 14, 2012 10:03:05 Don Clugston wrote: On 13/06/12 16:29, Walter Bright wrote: On 6/13/2012 1:07 AM, Don Clugston wrote: On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-15 Thread Jonathan M Davis
On Friday, June 15, 2012 08:58:55 Don Clugston wrote: I don't think Phobos should use .di files at all. I don't think there are any cases where we want to conceal code. The performance benefit you would get is completely negligible. It doesn't even reduce the number of files that need to be

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-14 Thread Don Clugston
On 13/06/12 16:29, Walter Bright wrote: On 6/13/2012 1:07 AM, Don Clugston wrote: On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-14 Thread Jonathan M Davis
On Thursday, June 14, 2012 10:03:05 Don Clugston wrote: On 13/06/12 16:29, Walter Bright wrote: On 6/13/2012 1:07 AM, Don Clugston wrote: On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull request to improve di file generation

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-14 Thread Kagamin
On Thursday, 14 June 2012 at 08:11:02 UTC, Jonathan M Davis wrote: Or do we still end up paying the semantic cost for importing the .d files such that using .di files would still help with compilation times? Oh, right, the module can use mixins and CTFE, so it should be semantically

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Paulo Pinto
On Tuesday, 12 June 2012 at 12:23:21 UTC, Dmitry Olshansky wrote: On 12.06.2012 16:09, foobar wrote: On Tuesday, 12 June 2012 at 11:09:04 UTC, Don Clugston wrote: On 12/06/12 11:07, timotheecour wrote: There's a current pull request to improve di file generation

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Don Clugston
On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to suggest further ideas. As far as I understand, di interface files try to achieve

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Iain Buclaw
On 13 June 2012 09:07, Don Clugston d...@nospam.com wrote: On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to suggest further

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread deadalnix
Le 13/06/2012 11:37, Iain Buclaw a écrit : On 13 June 2012 09:07, Don Clugstond...@nospam.com wrote: On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull request to improve di file generation

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Dmitry Olshansky
On 13.06.2012 13:37, Iain Buclaw wrote: On 13 June 2012 09:07, Don Clugstond...@nospam.com wrote: On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull request to improve di file generation

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Iain Buclaw
On 13 June 2012 10:45, Dmitry Olshansky dmitry.o...@gmail.com wrote: On 13.06.2012 13:37, Iain Buclaw wrote: On 13 June 2012 09:07, Don Clugstond...@nospam.com  wrote: On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull request to

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Dmitry Olshansky
On 13.06.2012 14:16, Iain Buclaw wrote: On 13 June 2012 10:45, Dmitry Olshanskydmitry.o...@gmail.com wrote: On 13.06.2012 13:37, Iain Buclaw wrote: On 13 June 2012 09:07, Don Clugstond...@nospam.comwrote: On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Kagamin
The measurements should be done for modules being imported, not the module being compiled. Something like this. --- import std.algorithm; import std.stdio; import std.typecons; import std.datetime; int ok; ---

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Kagamin
On Wednesday, 13 June 2012 at 11:29:45 UTC, Kagamin wrote: The measurements should be done for modules being imported, not the module being compiled. Something like this. --- import std.algorithm; import std.stdio; import std.typecons; import std.datetime; int ok; --- Oh and let it import .d

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Iain Buclaw
On 13 June 2012 12:33, Kagamin s...@here.lot wrote: On Wednesday, 13 June 2012 at 11:29:45 UTC, Kagamin wrote: The measurements should be done for modules being imported, not the module being compiled. Something like this. --- import std.algorithm; import std.stdio; import std.typecons;

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Kagamin
On Wednesday, 13 June 2012 at 11:47:31 UTC, Iain Buclaw wrote: std.datetime is one reason for me to run it again. I can imagine that *that* module will have an impact on parse times. But I'm still persistent that the majority of the compile time in the frontend is done in the first semantic

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Walter Bright
On 6/13/2012 1:07 AM, Don Clugston wrote: On 12/06/12 18:46, Walter Bright wrote: On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to suggest further ideas. As far as I

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-13 Thread Jacob Carlborg
On 2012-06-13 13:47, Iain Buclaw wrote: std.datetime is one reason for me to run it again. I can imagine that *that* module will have an impact on parse times. But I'm still persistent that the majority of the compile time in the frontend is done in the first semantic pass, and not the

AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread timotheecour
There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to suggest further ideas. As far as I understand, di interface files try to achieve these conflicting goals: 1) speed up compilation by avoiding having to reparse

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Tobias Pankrath
Currently .di-files are compiler independent. If this should hold for dib-files, too, we'll need a standard ast structure, won't we?

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Don Clugston
On 12/06/12 11:07, timotheecour wrote: There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to suggest further ideas. As far as I understand, di interface files try to achieve these conflicting goals: 1) speed up

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Timon Gehr
On 06/12/2012 12:47 PM, Alex Rønne Petersen wrote: On 12-06-2012 12:23, Tobias Pankrath wrote: Currently .di-files are compiler independent. If this should hold for dib-files, too, we'll need a standard ast structure, won't we? Which is a Good Thing (TM). It would /require/ formalization of

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread foobar
On Tuesday, 12 June 2012 at 11:09:04 UTC, Don Clugston wrote: On 12/06/12 11:07, timotheecour wrote: There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to suggest further ideas. As far as I understand, di interface

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Dmitry Olshansky
On 12.06.2012 16:09, foobar wrote: On Tuesday, 12 June 2012 at 11:09:04 UTC, Don Clugston wrote: On 12/06/12 11:07, timotheecour wrote: There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to suggest further ideas. As

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread foobar
On Tuesday, 12 June 2012 at 11:09:04 UTC, Don Clugston wrote: On 12/06/12 11:07, timotheecour wrote: There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to suggest further ideas. As far as I understand, di interface

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Jacob Carlborg
On 2012-06-12 14:09, foobar wrote: This is a solved problem since the 80's (E.g. Pascal units). Per Adam's post, the issue is tied to DMD's use of OMF/optlink which we all would like to get rid of anyway. Once we're in proper COFF land, couldn't we just store the required metadata (binary AST?)

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread deadalnix
Le 12/06/2012 12:23, Tobias Pankrath a écrit : Currently .di-files are compiler independent. If this should hold for dib-files, too, we'll need a standard ast structure, won't we? We need it anyway at some point. AST macro is another example. It would also greatly simplify compiler writing

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread deadalnix
Le 12/06/2012 14:39, foobar a écrit : Another related question - AFAIK the LLVM folks did/are doing work to make their implementation less platform-depended. Could we leverage this in ldc to store LLVM bit code as D libs which still retain enough info for the compiler to replace header files?

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Adam Wilson
On Tue, 12 Jun 2012 06:46:44 -0700, Jacob Carlborg d...@me.com wrote: On 2012-06-12 14:09, foobar wrote: This is a solved problem since the 80's (E.g. Pascal units). Per Adam's post, the issue is tied to DMD's use of OMF/optlink which we all would like to get rid of anyway. Once we're in

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Timon Gehr
On 06/12/2012 03:54 PM, deadalnix wrote: Le 12/06/2012 12:23, Tobias Pankrath a écrit : Currently .di-files are compiler independent. If this should hold for dib-files, too, we'll need a standard ast structure, won't we? We need it anyway at some point. Plain D code is already a perfectly

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Walter Bright
On 6/12/2012 2:07 AM, timotheecour wrote: There's a current pull request to improve di file generation (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to suggest further ideas. As far as I understand, di interface files try to achieve these conflicting goals: 1) speed up

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Adam Wilson
On Tue, 12 Jun 2012 05:23:16 -0700, Dmitry Olshansky dmitry.o...@gmail.com wrote: On 12.06.2012 16:09, foobar wrote: On Tuesday, 12 June 2012 at 11:09:04 UTC, Don Clugston wrote: On 12/06/12 11:07, timotheecour wrote: There's a current pull request to improve di file generation

Re: AST files instead of DI interface files for faster compilation and easier distribution

2012-06-12 Thread Dmitry Olshansky
On 12.06.2012 22:47, Adam Wilson wrote: On Tue, 12 Jun 2012 05:23:16 -0700, Dmitry Olshansky dmitry.o...@gmail.com wrote: On 12.06.2012 16:09, foobar wrote: On Tuesday, 12 June 2012 at 11:09:04 UTC, Don Clugston wrote: On 12/06/12 11:07, timotheecour wrote: There's a current pull request to