Re: D needs to publicize its speed of compilation
On 01/02/2018 10:13 AM, H. S. Teoh wrote: > As you can see, I made File into a template parameter so that the > unittest can simulate a virtual filesystem with a struct All I'm hearing is "blah blah blog." ;) Ali
Re: D needs to publicize its speed of compilation
On Tue, Jan 02, 2018 at 04:00:11PM +, Atila Neves via Digitalmars-d wrote:
> On Saturday, 30 December 2017 at 01:09:59 UTC, H. S. Teoh wrote:
[...]
> > Yeah... recently I've been resorting to `dmd -unittest -main
> > module.d` on single modules for faster turnaround time, because in
> > spite of D's famed compilation speeds, some things still make it go
> > far too slow, e.g. if you use template-heavy / CTFE-heavy code, or
> > there are just too many files to compile.
> >
> > [...]
>
> This is pretty interesting. Unfortunately I can see it getting messy
> with multiple dependent modules and/or if the function has "real"
> template parameters on top. I have to think about how to use your
> strategy in my own code.
[...]
Yeah, when you have dependent modules it can get pretty messy. Which is
why I've been moving towards modularising my code beyond what people
would usually do, i.e., turn even symbols in dependent modules into
template parameters so that you can make the module completely
independent. Here's one actual code example I'm particularly proud of:
--
/**
* Expands '@'-directives in a range of strings.
*
* Returns: A range of strings with lines that begin with '@' substituted with
* the contents of the file named by the rest of the line.
*/
auto expandFileDirectives(File = std.stdio.File, R)(R args)
if (isInputRange!R && is(ElementType!R : const(char)[]))
{
import std.algorithm.iteration : joiner, map;
import std.algorithm.searching : startsWith;
import std.range : only;
import std.range.interfaces : InputRange, inputRangeObject;
import std.typecons : No;
return args.map!(arg => arg.startsWith('@') ?
cast(InputRange!string) inputRangeObject(
File(arg[1 .. $]).byLineCopy(No.keepTerminator)) :
cast(InputRange!string) inputRangeObject(only(arg)))
.joiner;
}
unittest
{
static struct FakeFile
{
static string[][string] files;
string[] contents;
this(string filename) { contents = files[filename]; }
auto byLineCopy(KeepTerminator dummy) { return contents; }
}
FakeFile.files["file1"] = [
"x=1",
"y=2"
];
FakeFile.files["file2"] = [
"z=3"
];
import std.algorithm.comparison : equal;
assert([
"p=abc",
"@file1",
"q=def",
"@file2",
"e=ghi"
].expandFileDirectives!FakeFile.equal([
"p=abc",
"x=1",
"y=2",
"q=def",
"z=3",
"e=ghi"
]));
}
--
As you can see, I made File into a template parameter so that the
unittest can simulate a virtual filesystem with a struct, without ever
needing to deal with the messiness of actually creating and managing
temporary files on a real filesystem. This is (mostly) invisible to user
code (i.e., other code that uses this function) because if you don't
specify what File should be, it defaults to the usual std.stdio.File.
Note also that not all of File's API needed to be implemented; since the
function only uses .byLineCopy, that's all I needed to implement in my
surrogate FakeFile struct.
Not all code can be cut up in this way, though. I've only achieved this
in a few low-level modules; for other modules I'm still facing the
problem of having to specify dependent modules on the command-line. But
in trying to attain to this goal, it really forces you to think about
structuring your code in a better way than just throwing in haphazard
dependencies that may not strictly be necessary.
T
--
Just because you can, doesn't mean you should.
Re: D needs to publicize its speed of compilation
On Saturday, 30 December 2017 at 01:09:59 UTC, H. S. Teoh wrote: On Fri, Dec 29, 2017 at 05:52:36PM -0700, Jonathan M Davis via Digitalmars-d wrote: [...] [...] [...] Yeah... recently I've been resorting to `dmd -unittest -main module.d` on single modules for faster turnaround time, because in spite of D's famed compilation speeds, some things still make it go far too slow, e.g. if you use template-heavy / CTFE-heavy code, or there are just too many files to compile. [...] This is pretty interesting. Unfortunately I can see it getting messy with multiple dependent modules and/or if the function has "real" template parameters on top. I have to think about how to use your strategy in my own code. Atila
Re: D needs to publicize its speed of compilation
On Fri, Dec 29, 2017 at 05:52:36PM -0700, Jonathan M Davis via Digitalmars-d
wrote:
[...]
> Oh, certainly. But my point is that once it starts taking minutes to
> be able to build and run the unit tests for what you're currently
> doing, it becomes infeasible to edit-test-edit-etc. in any kind of
> reasonable cycle. It doesn't have to get anywhere near the 30 minute
> mark for it to be a serious problem. So, of course, 30+ minutes is an
> utter disaster if you're looking to test stuff as you go.
[...]
Yeah... recently I've been resorting to `dmd -unittest -main module.d`
on single modules for faster turnaround time, because in spite of D's
famed compilation speeds, some things still make it go far too slow,
e.g. if you use template-heavy / CTFE-heavy code, or there are just too
many files to compile.
This has also been encouraging me to write code in a far more modular
fashion than even traditional modularists would do, in the sense that
where possible, even dependent modules won't be imported, but instead
left to a template parameter, to be instantiated by the main program
with the dependent module. Contrived example:
-
void printSomething(alias stdio = selfImport!"std.stdio", Args...)(string fmt,
Args args)
{
stdio.writefln(fmt, args);
}
unittest
{
struct TestHarness {
static void writefln() { /* fake implementation here */ }
}
printSomething!TestHarness("%s", "blah");
}
-
The idea is that importing std.stdio won't even happen when compiling
with -unittest -main. This approach lets you completely detach a module
from its dependencies, so you never have to worry about what else it
depends on when compiling with -unittest -main, and also conveniently
allows you to substitute complicated types with cheaper substitutes that
suffice for testing, e.g., instead of writing to stdout or actually
causing side-effects visible outside the program, have a substitute
module (or just struct!) that redirects output or filesystem changes to
a log that's verified by the unittest to be the correct sequence of
operations. The substitute type doesn't even have to implement the
complete API of the module/object in question; just enough of it for the
unittest to do what it needs to do.
Being able to substitute a struct in place of a module inside a unittest
is one of the awesome things about D that make me love it so. :-D
And a side perk is that you'll get automatic attribute inference for all
such functions, if the original function would otherwise be just a
non-template function. (On that note, nowadays I'm tempted to just stick
empty compile-time parameters on every function I can; it enables
attribute inference and also reduces executable bloat if the function is
never actually called from anywhere. The only bad thing is the compiler
won't run semantic on potentially broken code... but then again, if you
never use it, it doesn't matter, and if it ought to matter, there should
be a unittest for it anyway.)
T
--
"You are a very disagreeable person." "NO."
Re: D needs to publicize its speed of compilation
On Friday, December 29, 2017 15:51:53 H. S. Teoh via Digitalmars-d wrote: > On Fri, Dec 29, 2017 at 03:49:07PM -0700, Jonathan M Davis via > Digitalmars-d wrote: [...] > > > It also depends on your personality. Personally, while I like builds > > being near instantaneous when dealing with unit tests, if they take a > > few seconds, it's not a big deal, but I know folks who get seriously > > annoyed if they can notice much of a difference time-wise between > > hitting enter to run the build command and hitting enter with no > > command. Either way though, having build times be more than a few > > seconds when dealing with unit tests tends to cause serious problems > > even though having a build for an entire project take a few minutes > > usually isn't a big deal unless you're really impatient. Having the > > delay between making changes and testing them be very long can > > _really_ slow down development, and so that's where build times really > > matter. > > [...] > > If you think a few minutes is bad... I've been on the receiving end of > builds that take 30-45 *minutes* for 1-line code changes > (*ahem*cough*C++ header files*cough*ahem*). It put a serious damper on > my ability to make any progress with the code. It's *possible*, yes, but > it's not something I'd wish on anyone. Oh, certainly. But my point is that once it starts taking minutes to be able to build and run the unit tests for what you're currently doing, it becomes infeasible to edit-test-edit-etc. in any kind of reasonable cycle. It doesn't have to get anywhere near the 30 minute mark for it to be a serious problem. So, of course, 30+ minutes is an utter disaster if you're looking to test stuff as you go. - Jonathan M Davis
Re: D needs to publicize its speed of compilation
On Fri, Dec 29, 2017 at 03:49:07PM -0700, Jonathan M Davis via Digitalmars-d wrote: [...] > It also depends on your personality. Personally, while I like builds > being near instantaneous when dealing with unit tests, if they take a > few seconds, it's not a big deal, but I know folks who get seriously > annoyed if they can notice much of a difference time-wise between > hitting enter to run the build command and hitting enter with no > command. Either way though, having build times be more than a few > seconds when dealing with unit tests tends to cause serious problems > even though having a build for an entire project take a few minutes > usually isn't a big deal unless you're really impatient. Having the > delay between making changes and testing them be very long can > _really_ slow down development, and so that's where build times really > matter. [...] If you think a few minutes is bad... I've been on the receiving end of builds that take 30-45 *minutes* for 1-line code changes (*ahem*cough*C++ header files*cough*ahem*). It put a serious damper on my ability to make any progress with the code. It's *possible*, yes, but it's not something I'd wish on anyone. T -- MSDOS = MicroSoft's Denial Of Service
Re: D needs to publicize its speed of compilation
On Thursday, 28 December 2017 at 21:58:07 UTC, Walter Bright wrote: On 12/28/2017 5:22 AM, Atila Neves wrote: On Thursday, 28 December 2017 at 07:21:28 UTC, Walter Bright wrote: On 12/27/2017 1:23 PM, Atila Neves wrote: However, my experience has been that D has fast builds from scratch, but is really really slow for incremental builds. You can do fast incremental builds with D by not putting all the files on the command line. Just put one. I don't build the compiler command-line myself, nor do I want to. Even then, recompiling one file isn't useful to me, what is is to recompile what's needed and run the tests. Emacs/flycheck already highlights compilation errors in just one file. I don't understand. C++ compiles files one at a time. You can do the same in D. How can this model be useful in C++ but not with D? Because: . As Iain pointed out independently, D modules are like C/C++ headers. All dependencies must also be recompiled. If I edit a C++ source file, only that source file has to be recompiled. I can use .di files, and they can even be generated for me (nice!) but there's no build system to do this properly. This is one of the ideas I alluded to earlier - I want to put this sort of thing in reggae. And now that dmd is a dub package I can even just use the actual compiler code. i.e. Generate the .di, if it's the same as before don't overwrite the old one to not mess with the timestamp. . C++ doesn't have built-in unit tests. Which means a framework has to be used and since nobody wants to put their tests in #ifdef "blocks" they live in separate files. Changing the implementation doesn't require the tests to be rebuilt. Yes, I can separate them in D too, but then I lose DDoc benefits and it's actually nicer to see tests next to the code. The fact that all D files are effectively headers means editing a unittest means recompiling all modules that depend on the current one. . Related to the previous point: compiling separately in D is actually slower overall than by package, because of all of the dependencies (I measured). Yes, I can compile by package, but I don't want to do this by hand. To the extent of my knowledge the only way to get per-package compilation by default or otherwise in the D ecosystem is reggae. But: . Until recently, it wasn't possible to use __traits(getUnitTests) with separate compilation. I've run into other issues with when doing so (I can't remember what they were). Even with the bug fix editing a test entails compiling several modules and it's slower than just one C++ file. In C++ editing a test source file always means recompiling exactly one file. . Speaking of dependencies, there's no easy way currently to know what they are. _If_ I want to build per package/module and only build what's needed, then I need to either hand-roll a Makefile (points gun to head) or use reggae. If there's any other alternative, I don't know about it. Even Phobos recompiles the world. That's not a good example to set. . C++ test framework's hacky usage of global variable assignment to register tests seems to be faster to compile than D's compile-time reflection to achieve the same result. I'd have to benchmark to be sure though. . If one has dub dependencies (and in all likelihood one has), then either one is a reggae early adopter (I keep mentioning it to highlight why I wrote it in the first place), or in the 99.9% of other users typing either `dub build` or `dub test`. There is no per-package build option. There's a per-module option, but it's single-threaded, therefore slow. Also, no dependency tracking, so let's recompile the whole dub package! In one thread. Z. What I really want is to hit F5 in emacs, rebuild the current file _and_ all of its dependencies, cache the dependency build per file so that if they haven't change don't rebuild, link and run the unit tests. But build _only_ what's needed to avoid linker errors in order to be able to run the unit tests for _one_ file. I spent an afternoon writing the necessary elisp to get this done and it worked! Then I tried a different project and it failed miserably. So did the 3rd one. I think I just remembered one of the separate compilation bugs I mentioned above. I know, I know, if I don't file it it won't get fixed (even if I file it I have a sneaky suspicion it'll be me fixing it anyway), but figuring out what went wrong takes time and energy and I haven't had enough of either to take this off my TODO list. To summarise: . Dependencies. . There's no standard build system that isn't called dub. . There's only one alternative build system that can do any of what's needed. . That alternative build system hasn't implemented my .di idea yet. . Separate compilation bugs. That nobody ever hits because dub builds everything at once. Atila
Re: D needs to publicize its speed of compilation
On Friday, December 29, 2017 22:37:56 I Love Stuffing via Digitalmars-d wrote: > To be fair, you are basing this off C++. C++ has an obnoxiously > slow build system at times and everyone knows this, have known > this, and have even addressed some parts of it. Yet they still > use it. I don't think build times are that big of a deal. It's a > nice to have, but as long as a fairly sizable project compiles in > a less than a minute, you won't stand out in a bad way. So much about build times and how much they matter seems to be about perception. For instance, I used to work at a company whose main project took over 3 - 4 hours to do a clean build. After the project was redone in Java, the build times were about 10 minutes, and folks complained about how long they were even though they were worlds better than the C++ build times had been. It also depends on your personality. Personally, while I like builds being near instantaneous when dealing with unit tests, if they take a few seconds, it's not a big deal, but I know folks who get seriously annoyed if they can notice much of a difference time-wise between hitting enter to run the build command and hitting enter with no command. Either way though, having build times be more than a few seconds when dealing with unit tests tends to cause serious problems even though having a build for an entire project take a few minutes usually isn't a big deal unless you're really impatient. Having the delay between making changes and testing them be very long can _really_ slow down development, and so that's where build times really matter. - Jonathan M Davis
Re: D needs to publicize its speed of compilation
On Friday, 22 December 2017 at 10:06:18 UTC, Joakim wrote: This one of the main strengths of D, it is what Walter focuses on, yet I have seen almost nothing on the D blog talking about this. What brought me to emphasize this today is this recent post about how long it takes to compile the mostly-C++ Chromium web browser and the reddit discussion about it: https://lobste.rs/s/iri1te/chromium_has_compilation_time_problem https://www.reddit.com/r/programming/comments/7ktzog/chromium_has_a_compilation_time_problem/ I'm tempted to call BS on that 6-7 hour build time, as I used to contribute to the Chromium project, and I think it used to take me 20 minutes for a release build in a FreeBSD jail on a fairly weak, 2008-vintage mini-desktop, a dual-core Celeron with 2 GBs of RAM (don't hold me to that build time, just a vague recollection, but probably in the ballpark). Of course, the last time I built Chromium was more than 5 years ago, and a lot has likely changed since then, such as now using LTO to speed up the browser apparently, and maybe the cross-compilation toolchain for ARM is slower, though others note similar times for native x64 compilation also. That still implies a slowdown of 2-3 orders of magnitude over the last 5 years, given the much more powerful hardware he's using, which is nuts. D really needs the community to write blog posts talking about how fast it is, publicizing that there is an alternative to these glacial build times: who wants to do this? It doesn't need to be on the D blog, could just be on your personal blog, but it is a message that really needs to be spread. To be fair, you are basing this off C++. C++ has an obnoxiously slow build system at times and everyone knows this, have known this, and have even addressed some parts of it. Yet they still use it. I don't think build times are that big of a deal. It's a nice to have, but as long as a fairly sizable project compiles in a less than a minute, you won't stand out in a bad way. If anything, D should leverage the fast compile times to take advantage of the leeway it gives so the compiler can do a bit of extra work to give better error messages.
Re: D needs to publicize its speed of compilation
On Thursday, 28 December 2017 at 13:26:55 UTC, Seb wrote: On Thursday, 28 December 2017 at 13:20:34 UTC, Atila Neves wrote: On Thursday, 28 December 2017 at 07:05:25 UTC, Joakim wrote: On Sunday, 24 December 2017 at 22:55:12 UTC, aberba wrote: On Friday, 22 December 2017 at 10:06:18 UTC, Joakim wrote: What do you plan to do about it? I have an idea or two. I just need time/mental energy to do this. Atila Are you aware of this great work: https://github.com/dlang/dmd/pull/7239? I wasn't, thanks for pointing it out. Looks interesting, but all the compile times were still over a second, which is my pain threshold. Atila
Re: D needs to publicize its speed of compilation
On Thursday, 28 December 2017 at 15:54:03 UTC, H. S. Teoh wrote: On Thu, Dec 28, 2017 at 01:22:13PM +, Atila Neves via Digitalmars-d wrote: On Thursday, 28 December 2017 at 07:21:28 UTC, Walter Bright wrote: > On 12/27/2017 1:23 PM, Atila Neves wrote: > > However, my experience has been that D has fast builds > > from scratch, but is really really slow for incremental > > builds. > You can do fast incremental builds with D by not putting all > the files on the command line. Just put one. I don't build the compiler command-line myself, nor do I want to. Even then, recompiling one file isn't useful to me, what is is to recompile what's needed and run the tests. [...] Isn't that what a build system is for? T Yes, and I wrote one. It turns out that I misunderstood what Walter said though. Atila
Re: D needs to publicize its speed of compilation
On 12/28/2017 2:16 PM, Iain Buclaw wrote: D modules are like C++ headers. Change one, and you need to recompile everything that imports it. That's right. That doesn't mean that D should incrementally compile slower than C++.
Re: D needs to publicize its speed of compilation
On 28 December 2017 at 22:58, Walter Bright via Digitalmars-d wrote: > On 12/28/2017 5:22 AM, Atila Neves wrote: >> >> On Thursday, 28 December 2017 at 07:21:28 UTC, Walter Bright wrote: >>> >>> On 12/27/2017 1:23 PM, Atila Neves wrote: However, my experience has been that D has fast builds from scratch, but is really really slow for incremental builds. >>> >>> You can do fast incremental builds with D by not putting all the files on >>> the command line. Just put one. >> >> >> I don't build the compiler command-line myself, nor do I want to. Even >> then, recompiling one file isn't useful to me, what is is to recompile >> what's needed and run the tests. Emacs/flycheck already highlights >> compilation errors in just one file. > > > I don't understand. C++ compiles files one at a time. You can do the same in > D. How can this model be useful in C++ but not with D? > D modules are like C++ headers. Change one, and you need to recompile everything that imports it.
Re: D needs to publicize its speed of compilation
On 12/28/2017 5:22 AM, Atila Neves wrote: On Thursday, 28 December 2017 at 07:21:28 UTC, Walter Bright wrote: On 12/27/2017 1:23 PM, Atila Neves wrote: However, my experience has been that D has fast builds from scratch, but is really really slow for incremental builds. You can do fast incremental builds with D by not putting all the files on the command line. Just put one. I don't build the compiler command-line myself, nor do I want to. Even then, recompiling one file isn't useful to me, what is is to recompile what's needed and run the tests. Emacs/flycheck already highlights compilation errors in just one file. I don't understand. C++ compiles files one at a time. You can do the same in D. How can this model be useful in C++ but not with D?
Re: D needs to publicize its speed of compilation
On Thu, Dec 28, 2017 at 01:22:13PM +, Atila Neves via Digitalmars-d wrote: > On Thursday, 28 December 2017 at 07:21:28 UTC, Walter Bright wrote: > > On 12/27/2017 1:23 PM, Atila Neves wrote: > > > However, my experience has been that D has fast builds from > > > scratch, but is really really slow for incremental builds. > > You can do fast incremental builds with D by not putting all the > > files on the command line. Just put one. > > I don't build the compiler command-line myself, nor do I want to. Even > then, recompiling one file isn't useful to me, what is is to recompile > what's needed and run the tests. [...] Isn't that what a build system is for? T -- GEEK = Gatherer of Extremely Enlightening Knowledge
Re: D needs to publicize its speed of compilation
On Thursday, 28 December 2017 at 13:20:34 UTC, Atila Neves wrote: On Thursday, 28 December 2017 at 07:05:25 UTC, Joakim wrote: On Sunday, 24 December 2017 at 22:55:12 UTC, aberba wrote: On Friday, 22 December 2017 at 10:06:18 UTC, Joakim wrote: What do you plan to do about it? I have an idea or two. I just need time/mental energy to do this. Atila Are you aware of this great work: https://github.com/dlang/dmd/pull/7239?
Re: D needs to publicize its speed of compilation
On Thursday, 28 December 2017 at 07:21:28 UTC, Walter Bright wrote: On 12/27/2017 1:23 PM, Atila Neves wrote: However, my experience has been that D has fast builds from scratch, but is really really slow for incremental builds. You can do fast incremental builds with D by not putting all the files on the command line. Just put one. I don't build the compiler command-line myself, nor do I want to. Even then, recompiling one file isn't useful to me, what is is to recompile what's needed and run the tests. Emacs/flycheck already highlights compilation errors in just one file. Atila
Re: D needs to publicize its speed of compilation
On Thursday, 28 December 2017 at 07:05:25 UTC, Joakim wrote: On Sunday, 24 December 2017 at 22:55:12 UTC, aberba wrote: On Friday, 22 December 2017 at 10:06:18 UTC, Joakim wrote: What do you plan to do about it? I have an idea or two. I just need time/mental energy to do this. Atila
Re: D needs to publicize its speed of compilation
On 12/27/2017 1:23 PM, Atila Neves wrote: However, my experience has been that D has fast builds from scratch, but is really really slow for incremental builds. You can do fast incremental builds with D by not putting all the files on the command line. Just put one.
Re: D needs to publicize its speed of compilation
On Sunday, 24 December 2017 at 22:55:12 UTC, aberba wrote: On Friday, 22 December 2017 at 10:06:18 UTC, Joakim wrote: This one of the main strengths of D, it is what Walter focuses on, yet I have seen almost nothing on the D blog talking about this. What brought me to emphasize this today is this recent post about how long it takes to compile the mostly-C++ Chromium web browser and the reddit discussion about it: [...] Memory consumption is a deal breaker. The dmd frontend prioritized speed over memory, which can cause problems with large or CTFE-heavy projects on low-memory systems: http://www.drdobbs.com/cpp/increasing-compiler-speed-by-over-75/240158941 On Wednesday, 27 December 2017 at 21:23:15 UTC, Atila Neves wrote: I thought D did market itself that way. It has historically been a selling point, but perhaps it has been taken for granted. I'm saying it needs a marketing push now. I just went and checked the front page of the website, and I see a new marketing slogan that was added a couple weeks ago, "write fast, read fast, and run fast. Fast code, fast:" https://github.com/dlang/dlang.org/pull/1965 However, "build fast" is not included in that list. You could argue it's implied by "write fast" or the last "code, fast," but it's not mentioned in the list under "write fast" below. So even the front page doesn't highlight build speed, one of the core advantages of D. However, my experience has been that D has fast builds from scratch, but is really really slow for incremental builds. Here's the thing: personally I don't care about anything except incremental builds. It's great that Phobos compiles in seconds when I do a git fetch after months. Win! Oh wait, I'm working on a bug, changed one character and it still takes the same amount of time... Given that I write D code every day now, this incremental compilation situation is driving me nuts, and I'm definitely going to do something about it. It's getting so bad that the day at work I had to write C++ I was happy. Because the edit-compile-link-test cycle was _faster_. Think about that for a bit - I enjoyed writing C++ (!!! either it or Scala are the slowest languages to compile ever) because I got faster feedback. Epic fail. It's also possible I use templates and CTFE more that the regular D programmer. But... they're some of the main reasons I use D in the first place! I don't want to write Java in D syntax. What do you plan to do about it?
Re: D needs to publicize its speed of compilation
On Friday, 22 December 2017 at 10:06:18 UTC, Joakim wrote: This one of the main strengths of D, it is what Walter focuses on, yet I have seen almost nothing on the D blog talking about this. What brought me to emphasize this today is this recent post about how long it takes to compile the mostly-C++ Chromium web browser and the reddit discussion about it: https://lobste.rs/s/iri1te/chromium_has_compilation_time_problem https://www.reddit.com/r/programming/comments/7ktzog/chromium_has_a_compilation_time_problem/ I'm tempted to call BS on that 6-7 hour build time, as I used to contribute to the Chromium project, and I think it used to take me 20 minutes for a release build in a FreeBSD jail on a fairly weak, 2008-vintage mini-desktop, a dual-core Celeron with 2 GBs of RAM (don't hold me to that build time, just a vague recollection, but probably in the ballpark). Of course, the last time I built Chromium was more than 5 years ago, and a lot has likely changed since then, such as now using LTO to speed up the browser apparently, and maybe the cross-compilation toolchain for ARM is slower, though others note similar times for native x64 compilation also. That still implies a slowdown of 2-3 orders of magnitude over the last 5 years, given the much more powerful hardware he's using, which is nuts. D really needs the community to write blog posts talking about how fast it is, publicizing that there is an alternative to these glacial build times: who wants to do this? It doesn't need to be on the D blog, could just be on your personal blog, but it is a message that really needs to be spread. I thought D did market itself that way. However, my experience has been that D has fast builds from scratch, but is really really slow for incremental builds. Here's the thing: personally I don't care about anything except incremental builds. It's great that Phobos compiles in seconds when I do a git fetch after months. Win! Oh wait, I'm working on a bug, changed one character and it still takes the same amount of time... Given that I write D code every day now, this incremental compilation situation is driving me nuts, and I'm definitely going to do something about it. It's getting so bad that the day at work I had to write C++ I was happy. Because the edit-compile-link-test cycle was _faster_. Think about that for a bit - I enjoyed writing C++ (!!! either it or Scala are the slowest languages to compile ever) because I got faster feedback. Epic fail. It's also possible I use templates and CTFE more that the regular D programmer. But... they're some of the main reasons I use D in the first place! I don't want to write Java in D syntax. Atila Atila
Re: D needs to publicize its speed of compilation
On Friday, 22 December 2017 at 10:06:18 UTC, Joakim wrote: This one of the main strengths of D, it is what Walter focuses on, yet I have seen almost nothing on the D blog talking about this. What brought me to emphasize this today is this recent post about how long it takes to compile the mostly-C++ Chromium web browser and the reddit discussion about it: [...] Memory consumption is a deal breaker.
Re: D needs to publicize its speed of compilation
On Friday, 22 December 2017 at 11:39:48 UTC, user1234 wrote: If this ever happens, what will be published will have to be based on LDC2. If benchmarks based on DMD are published, the article will be subject to the criticism that is that the shorter build time is due to the optimization pass, since it's known not to be super deep in DMD backend. Sure, ldc is slower but not significantly so, and will still beat the pants off clang, Rust, or Swift. On Friday, 22 December 2017 at 11:46:49 UTC, Chris wrote: D does not do itself any favors when it keeps accepting mediocre results in benchmarks. These are all runtime codegen benchmarks, whereas I was talking about build speed. https://www.techempower.com/benchmarks/previews/round15/ Some years it does not complete the tests, in others it shows results that are below mediocre. Vibe.d ( and other D web frameworks ) benching twice slower then scripting languages like PHP/Ruby. That does not advertise D. Sonke and others have noted that there are issues that need to be worked out with that benchmark submission. How about: http://benchmarksgame.alioth.debian.org/ No D there? Performance must be bad because its not listed at all ( for a language that exist 20 years )? It was on there for many years but was tossed out by a subsequent maintainer, who doesn't want it for some reason, as thedeemon pointed out: http://forum.dlang.org/post/lspybogtpqorauaus...@forum.dlang.org Another one: https://github.com/costajob/app-servers Single threaded ( look at CPU total ), losing to other languages again. I think vibe.d is meant to be run in multiple process instances instead, and it does second-best to Rust on a single core (maybe beat out by Crystal too?), which is pretty good. Impressions are everything when there is a wealth of languages to pick from. Anybody stumbling over these results think: Well, i am better going with Go, Rust, Crystal, ... for a web hosting as they show more consistent high speed results. I wouldn't say they're "everything," but they intrigue people enough to dig deeper into the language or just to move on without bothering. Of course, you may get great benchmarks with Rust but once you try coding with it, you still may not like using it. On Friday, 22 December 2017 at 14:48:22 UTC, thedeemon wrote: On Friday, 22 December 2017 at 11:46:49 UTC, Chris wrote: http://benchmarksgame.alioth.debian.org/ No D there? Performance must be bad because its not listed at all ( for a language that exist 20 years )? D is not there for the only reason of that benchmark maintainer unwilling to include D. Technically you can take any C solution there, translate it to D (mostly by renaming from .c to .d) and with LDC get the same speed as C shows. Better than C actually, if you use D libraries instead: http://forum.dlang.org/post/olelrjiaiwdxhjhil...@forum.dlang.org Getting back to the original topic of build speed, highlighting that the dmd compiler itself now builds in seconds after the C++->D translation would seem to be a layup: http://forum.dlang.org/thread/oknuas$2lfp$1...@digitalmars.com
Re: D needs to publicize its speed of compilation
On Fri, 2017-12-22 at 14:48 +, thedeemon via Digitalmars-d wrote: > […] > > D is not there for the only reason of that benchmark maintainer > unwilling to include D. Technically you can take any C solution > there, translate it to D (mostly by renaming from .c to .d) and > with LDC get the same speed as C shows. For performance related stuff Go people use gccgo, just as D people use gdc (or more usually ldc). -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: D needs to publicize its speed of compilation
On Friday, 22 December 2017 at 11:39:48 UTC, user1234 wrote: If benchmarks based on DMD are published, the article will be subject to the criticism that is that the shorter build time is due to the optimization pass, since it's known not to be super deep in DMD backend. Well, Go folks don't shy away from praising their compilation speed, but the situation is very similar there: Go compiles fast because it doesn't optimize the code. Last time I checked their compiler had zero options mentioning optimization, it's like "always debug build".
Re: D needs to publicize its speed of compilation
On Friday, 22 December 2017 at 11:46:49 UTC, Chris wrote: http://benchmarksgame.alioth.debian.org/ No D there? Performance must be bad because its not listed at all ( for a language that exist 20 years )? D is not there for the only reason of that benchmark maintainer unwilling to include D. Technically you can take any C solution there, translate it to D (mostly by renaming from .c to .d) and with LDC get the same speed as C shows.
Re: D needs to publicize its speed of compilation
On Friday, 22 December 2017 at 10:06:18 UTC, Joakim wrote: This one of the main strengths of D, it is what Walter focuses on, yet I have seen almost nothing on the D blog talking about this. What brought me to emphasize this today is this recent post about how long it takes to compile the mostly-C++ Chromium web browser and the reddit discussion about it: https://lobste.rs/s/iri1te/chromium_has_compilation_time_problem https://www.reddit.com/r/programming/comments/7ktzog/chromium_has_a_compilation_time_problem/ I'm tempted to call BS on that 6-7 hour build time, as I used to contribute to the Chromium project, and I think it used to take me 20 minutes for a release build in a FreeBSD jail on a fairly weak, 2008-vintage mini-desktop, a dual-core Celeron with 2 GBs of RAM (don't hold me to that build time, just a vague recollection, but probably in the ballpark). Of course, the last time I built Chromium was more than 5 years ago, and a lot has likely changed since then, such as now using LTO to speed up the browser apparently, and maybe the cross-compilation toolchain for ARM is slower, though others note similar times for native x64 compilation also. That still implies a slowdown of 2-3 orders of magnitude over the last 5 years, given the much more powerful hardware he's using, which is nuts. D really needs the community to write blog posts talking about how fast it is, publicizing that there is an alternative to these glacial build times: who wants to do this? It doesn't need to be on the D blog, could just be on your personal blog, but it is a message that really needs to be spread. D does not do itself any favors when it keeps accepting mediocre results in benchmarks. https://www.techempower.com/benchmarks/previews/round15/ Some years it does not complete the tests, in others it shows results that are below mediocre. Vibe.d ( and other D web frameworks ) benching twice slower then scripting languages like PHP/Ruby. That does not advertise D. How about: http://benchmarksgame.alioth.debian.org/ No D there? Performance must be bad because its not listed at all ( for a language that exist 20 years )? Another one: https://github.com/costajob/app-servers Single threaded ( look at CPU total ), losing to other languages again. Impressions are everything when there is a wealth of languages to pick from. Anybody stumbling over these results think: Well, i am better going with Go, Rust, Crystal, ... for a web hosting as they show more consistent high speed results.
Re: D needs to publicize its speed of compilation
On Friday, 22 December 2017 at 10:06:18 UTC, Joakim wrote: This one of the main strengths of D, it is what Walter focuses on, yet I have seen almost nothing on the D blog talking about this. What brought me to emphasize this today is this recent post about how long it takes to compile the mostly-C++ Chromium web browser and the reddit discussion about it: https://lobste.rs/s/iri1te/chromium_has_compilation_time_problem https://www.reddit.com/r/programming/comments/7ktzog/chromium_has_a_compilation_time_problem/ I'm tempted to call BS on that 6-7 hour build time, as I used to contribute to the Chromium project, and I think it used to take me 20 minutes for a release build in a FreeBSD jail on a fairly weak, 2008-vintage mini-desktop, a dual-core Celeron with 2 GBs of RAM (don't hold me to that build time, just a vague recollection, but probably in the ballpark). Of course, the last time I built Chromium was more than 5 years ago, and a lot has likely changed since then, such as now using LTO to speed up the browser apparently, and maybe the cross-compilation toolchain for ARM is slower, though others note similar times for native x64 compilation also. That still implies a slowdown of 2-3 orders of magnitude over the last 5 years, given the much more powerful hardware he's using, which is nuts. D really needs the community to write blog posts talking about how fast it is, publicizing that there is an alternative to these glacial build times: who wants to do this? It doesn't need to be on the D blog, could just be on your personal blog, but it is a message that really needs to be spread. If this ever happens, what will be published will have to be based on LDC2. If benchmarks based on DMD are published, the article will be subject to the criticism that is that the shorter build time is due to the optimization pass, since it's known not to be super deep in DMD backend.
