Re: D compilation is too slow and I am forking the compiler
On 11/21/2018 8:48 PM, Vladimir Panteleev wrote: On Wednesday, 21 November 2018 at 20:51:17 UTC, Walter Bright wrote: Unfortunately, you're right. The title will leave the impression "D is slow at compiling". You have to carefully read the article to see otherwise, and few will do that. Sorry about that. I'll have to think of two titles next time, one for the D community and one for everyone else. If it's of any consolation, the top comments in both discussion threads point out that the title is inaccurate on purpose. It has indeed done well. In fact, the article is so good it is probably worth it to have that attention-getting title. It is a risky strategy, though :-)
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 20:51:17 UTC, Walter Bright wrote: Unfortunately, you're right. The title will leave the impression "D is slow at compiling". You have to carefully read the article to see otherwise, and few will do that. Sorry about that. I'll have to think of two titles next time, one for the D community and one for everyone else. If it's of any consolation, the top comments in both discussion threads point out that the title is inaccurate on purpose.
Re: sumtype 0.7.0
On Thursday, 22 November 2018 at 00:02:18 UTC, H. S. Teoh wrote: Any way this could be expanded to arbitrary types like Variant? Or is that not possible without reverting to TypeInfo dependency? T Not possible, unfortunately--for that and other reasons.
Re: termcolor-d - Colors with writeln(...);
On Wednesday, 21 November 2018 at 23:49:29 UTC, Vladimirs Nordholm wrote: However, not sure if it's active anymore, but ConsoleD (by Robik and Adam D. Ruppe) has most Windows specific colors and attributes available. Maybe give that a look? Well, the consoleD part I'm not sure about, but I still work on my terminal.d which includes cross-platform color support (among a lot more like mouse input, line editing, etc). Your api is not compatible with old Windows, but newer versions of Windows support the same esc sequences linux uses - though I think you have to opt into it with a console api call. (I might be adding support to this to terminal.d but eh i already have the features in there)
Re: termcolor-d - Colors with writeln(...);
On Wed, Nov 21, 2018 at 11:49:29PM +, Vladimirs Nordholm via Digitalmars-d-announce wrote: > On Wednesday, 21 November 2018 at 23:46:00 UTC, Dennis wrote: > > On Wednesday, 21 November 2018 at 18:36:06 UTC, Vladimirs Nordholm > > wrote: > > > (hackish, POSIX only) > > > > Windows support coming? :) > > Maybe during another lunch break ;) > > However, not sure if it's active anymore, but ConsoleD (by Robik and > Adam D. Ruppe) has most Windows specific colors and attributes > available. Maybe give that a look? [...] I've used Adam Ruppe's terminal.d to great effect in my CLI programs. Highly recommended: https://github.com/adamdruppe/arsd/blob/master/terminal.d T -- Why did the mathematician reinvent the square wheel? Because he wanted to drive smoothly over an inverted catenary road.
Re: sumtype 0.7.0
On Wed, Nov 21, 2018 at 12:38:25AM +, Paul Backus via Digitalmars-d-announce wrote: > SumType is a generic sum type for modern D. It is meant as an > alternative to `std.variant.Algebraic`. > > Features: > - Pattern matching, including support for structural matching (★) > - Self-referential types, using `This` > - Works with pure, @safe, @nogc, and immutable (★) > - Zero runtime overhead compared to hand-written C > - No heap allocation > - Does not rely on runtime type information (`TypeInfo`) (★) > > Starred features (★) are those that are missing from `Algebraic`. [...] Took a quick look at this. Wow! Excellent job! Very nicely-designed API, much better than std.variant.* IMO. Any way this could be expanded to arbitrary types like Variant? Or is that not possible without reverting to TypeInfo dependency? T -- Why have vacation when you can work?? -- EC
Re: termcolor-d - Colors with writeln(...);
On Wednesday, 21 November 2018 at 23:30:39 UTC, H. S. Teoh wrote: On Wed, Nov 21, 2018 at 06:36:06PM +, Vladimirs Nordholm via Digitalmars-d-announce wrote: [...] Clever idea! Doesn't quite cover all the color features of newer terminals, but good enough for basic coloring on the terminal. Maybe I'll steal your idea next time I'm looking for some terminal colors. :D T Thanks, and feel free to!
Re: termcolor-d - Colors with writeln(...);
On Wednesday, 21 November 2018 at 23:46:00 UTC, Dennis wrote: On Wednesday, 21 November 2018 at 18:36:06 UTC, Vladimirs Nordholm wrote: (hackish, POSIX only) Windows support coming? :) Maybe during another lunch break ;) However, not sure if it's active anymore, but ConsoleD (by Robik and Adam D. Ruppe) has most Windows specific colors and attributes available. Maybe give that a look?
Re: termcolor-d - Colors with writeln(...);
On Wednesday, 21 November 2018 at 18:36:06 UTC, Vladimirs Nordholm wrote: (hackish, POSIX only) Windows support coming? :)
Re: termcolor-d - Colors with writeln(...);
On Wed, Nov 21, 2018 at 06:36:06PM +, Vladimirs Nordholm via Digitalmars-d-announce wrote: > https://github.com/vladdeSV/termcolor-d > > Saw a library recently which allowed you to color text, but it had an > odd syntax. > > Since I already had some code for coloring text in terminals, I made > this (hackish, POSIX only) project during lunch break. It in action: > > import std.stdio : writeln; > import termcolor; > > // Color → Green → Foreground > writeln(C.green.fg, "Green text", resetColor); > > // Color → Red → Background > writeln(C.red.bg, "Red background", resetColor); > > // only tested on macOS running zsh using iTerm2/Hyper.js/Terminal.app > > Hope this helps those who are making CLI applications :^) Clever idea! Doesn't quite cover all the color features of newer terminals, but good enough for basic coloring on the terminal. Maybe I'll steal your idea next time I'm looking for some terminal colors. :D T -- I am not young enough to know everything. -- Oscar Wilde
Re: D compilation is too slow and I am forking the compiler
I would say opposite :) On Wed, Nov 21, 2018 at 9:55 PM Walter Bright via Digitalmars-d-announce < digitalmars-d-announce@puremagic.com> wrote: > On 11/21/2018 5:55 AM, Guillaume Piolat wrote: > > On Wednesday, 21 November 2018 at 11:18:22 UTC, Nicholas Wilson wrote: > >> On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev > wrote: > >>> > https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ > >>> > >> > >> This is #2 on HN at the moment. > > > > I would be wary of such titles. > > > > Whatever will remain in minds will be something like "D compilation is > slow" > > which isn't accurate when compared to the competition. > > > > The article is clever, the title is clever, but most people will only > read the > > title. > > Unfortunately, you're right. The title will leave the impression "D is > slow at > compiling". You have to carefully read the article to see otherwise, and > few > will do that. >
Re: D compilation is too slow and I am forking the compiler
I neglected point out, however, that the article itself is a home run! Thank you, Vladimir!
Re: D compilation is too slow and I am forking the compiler
On 11/21/2018 5:55 AM, Guillaume Piolat wrote: On Wednesday, 21 November 2018 at 11:18:22 UTC, Nicholas Wilson wrote: On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote: https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ This is #2 on HN at the moment. I would be wary of such titles. Whatever will remain in minds will be something like "D compilation is slow" which isn't accurate when compared to the competition. The article is clever, the title is clever, but most people will only read the title. Unfortunately, you're right. The title will leave the impression "D is slow at compiling". You have to carefully read the article to see otherwise, and few will do that.
Re: D compilation is too slow and I am forking the compiler
On 11/21/2018 3:19 AM, Iain Buclaw wrote: On Wednesday, 21 November 2018 at 10:56:02 UTC, Walter Bright wrote: Wouldn't it be awesome to have the lexing/parsing of the imports all done in parallel? The main difficulty in getting that to work is dealing with the shared string table. What about creating a new Fiber for each module needing lexing/parsing/semantic to be ran? Compilation of one module would then get as far as it can until it needs to defer, then calls yield() to continue compilation of the next module. This in hope that when the round trip returns, the AST will be sufficiently complete enough for compilation to continue. Since the lexing/parsing does not do any blocking calls, fibers won't help. It has to be another hardware thread. Trying to parallelize semantic over multiple modules will be very hard to do.
termcolor-d - Colors with writeln(...);
https://github.com/vladdeSV/termcolor-d Saw a library recently which allowed you to color text, but it had an odd syntax. Since I already had some code for coloring text in terminals, I made this (hackish, POSIX only) project during lunch break. It in action: import std.stdio : writeln; import termcolor; // Color → Green → Foreground writeln(C.green.fg, "Green text", resetColor); // Color → Red → Background writeln(C.red.bg, "Red background", resetColor); // only tested on macOS running zsh using iTerm2/Hyper.js/Terminal.app Hope this helps those who are making CLI applications :^)
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 11:18:22 UTC, Nicholas Wilson wrote: This is #2 on HN at the moment. Also on reddit: https://www.reddit.com/r/programming/comments/9z36xg/d_compilation_is_too_slow_and_i_am_forking_the/
Re: smile.amazon.com Promotion
On Monday, 29 October 2018 at 16:01:38 UTC, Mike Parker wrote: One of the easiest ways to support the D Language Foundation is using smile.amazon.com when you make a purchase. Until Nov 2, they're running a special where they're donating 5% (10 times the usual amount) you buy through AmazonSmile. smile.amazon.com/ch/47-5352856 Hmmm... cannot register for the "D Language Foundation" at smile.amazon.<<< DE >>> ... and on __COM__ I'm note getting some things even it shows "qualified for smile.amazon.DE"...
Re: LDC 1.13.0-beta2
On Wed, Nov 21, 2018 at 10:43:55AM +, kinke via Digitalmars-d-announce wrote: > Glad to announce the second beta for LDC 1.13: > > * Based on D 2.083.0+ (yesterday's DMD stable). > * The Windows packages are now fully self-sufficient, i.e., a Visual > Studio/C++ Build Tools installation isn't required anymore. > * Substantial debug info improvements. > * New command-line option `-fvisibility=hidden` to hide functions/globals > not marked as export, to reduce the size of shared libraries. > > Full release log and downloads: > https://github.com/ldc-developers/ldc/releases/tag/v1.13.0-beta2 > > Thanks to all contributors! Awesome work keeping up with the DMD releases! T -- Береги платье снову, а здоровье смолоду.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 11:58:25 UTC, Vladimir Panteleev wrote: On Wednesday, 21 November 2018 at 11:35:02 UTC, Atila Neves wrote: [...] Looking forward to it! [...] That particular problem is in large part due to that the -unittest switch is not namespaced. I ran into the same issue with -allinst - with std.path, it breaks only if -unittest is also specified. I don't want to compile std.path unit tests, just my own! Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)? I'm not sure what the real issue is here or what the solution would be. There was a PR to fix it that was closed without merging.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 11:18:22 UTC, Nicholas Wilson wrote: On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote: https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ This is #2 on HN at the moment. I would be wary of such titles. Whatever will remain in minds will be something like "D compilation is slow" which isn't accurate when compared to the competition. The article is clever, the title is clever, but most people will only read the title.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 13:05:27 UTC, Nicholas Wilson wrote: On Wednesday, 21 November 2018 at 11:58:25 UTC, Vladimir Panteleev wrote: Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)? Not that I know of, thats a great idea! Yes it's sadly a well-known problem e.g. https://github.com/dlang/dmd/pull/8124
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 13:05:27 UTC, Nicholas Wilson wrote: On Wednesday, 21 November 2018 at 11:58:25 UTC, Vladimir Panteleev wrote: Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)? Not that I know of, thats a great idea! Maybe this hack could be developed further into a more generic "compiler server" idea. Wasn't that Robert's Masters thesis (Dconf 2013(?) presentation)? ;) The main problem with this, is the amount of context a compilers needs. And the current design of DMD does not lend itself splitting out the context. If you wanted you could consider the semantic pass of the compiler as a database, which answers queries such as: - which size does this type have. - which arguments does this function have - can the type A be casted to type B - which conversion function should be invoked for (B)A ? - is this function known to be pure? The data-base containing this information needs to be maintained on the compile-nodes, and that possibly leads to many data-dependencies. Which may degrade the performance of the "compiler server" to the point where it is quicker to do it locally. I am currently working (albeit very slowly due to lack of time and focus) to enable programmers to circumvent slow parts in compiler. When completed this should make a compiler-server unnecessary for some time to come.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 11:58:25 UTC, Vladimir Panteleev wrote: Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)? Not that I know of, thats a great idea! Maybe this hack could be developed further into a more generic "compiler server" idea. Wasn't that Robert's Masters thesis (Dconf 2013(?) presentation)? ;)
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 11:35:02 UTC, Atila Neves wrote: I'm also currently working on a project to save my bloodstream from the cortisol drip that happens when anything a computer does takes over a second, which these days means waiting for dmd to compile my code so I can see the result of the tests. I'll share more details when I have time. Looking forward to it! But: one of the things I want to do is a version of this / precompiled headers. I've complained before that compiling std.path with -unittest takes forever (0.5s or so, and most of it due to std.uni). That's a price I pay every time I make a one line change to any file, and the linker hasn't even been invoked yet. Here's the thing: Phobos only changes from one release to the next. Why am I waiting to recompile a read-only file that won't change unless I update the compiler, over and over again? That particular problem is in large part due to that the -unittest switch is not namespaced. I ran into the same issue with -allinst - with std.path, it breaks only if -unittest is also specified. I don't want to compile std.path unit tests, just my own! Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)? I'd love it if I could precompile Phobos and just use the digested information every time I'm iterating. I agree, it would be nice if we could ship some "precompiled module" files along with Phobos .lib / .so files, but it looks like implementing this feature correctly might be non-trivial. Maybe this hack could be developed further into a more generic "compiler server" idea.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote: https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ Very interesting. I'm also currently working on a project to save my bloodstream from the cortisol drip that happens when anything a computer does takes over a second, which these days means waiting for dmd to compile my code so I can see the result of the tests. I'll share more details when I have time. But: one of the things I want to do is a version of this / precompiled headers. I've complained before that compiling std.path with -unittest takes forever (0.5s or so, and most of it due to std.uni). That's a price I pay every time I make a one line change to any file, and the linker hasn't even been invoked yet. Here's the thing: Phobos only changes from one release to the next. Why am I waiting to recompile a read-only file that won't change unless I update the compiler, over and over again? I'd love it if I could precompile Phobos and just use the digested information every time I'm iterating.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote: https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ This is #2 on HN at the moment.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 10:56:02 UTC, Walter Bright wrote: Wouldn't it be awesome to have the lexing/parsing of the imports all done in parallel? The main difficulty in getting that to work is dealing with the shared string table. What about creating a new Fiber for each module needing lexing/parsing/semantic to be ran? Compilation of one module would then get as far as it can until it needs to defer, then calls yield() to continue compilation of the next module. This in hope that when the round trip returns, the AST will be sufficiently complete enough for compilation to continue.
Re: D compilation is too slow and I am forking the compiler
On 11/21/2018 2:16 AM, Vladimir Panteleev wrote: On Wednesday, 21 November 2018 at 09:46:44 UTC, Walter Bright wrote: It works by allocating memory from a memory-mapped file, which serves as the precompiled header. Hey, that's a great idea! Can we do this for DMD? :D On a more serious note: do you think that with D's features (type system / metaprogramming), you could have avoided some of those bugs? For example, one thing we can do in D which is still impossible in C++ is to automatically serialize/deserialize all fields of a struct/class (using tupleof / allMembers). Memory mapped files really were the key to success, because if you could reload the mmf at the same address, the pointers did not have to be patched. In the DMC++ source code, "dehydrating" a pointer meant subtracting a value from it so it was correct for the base address of the mmf, and "hydrating" a pointer was the inverse. The two bug prone problems were: 1. separating out the tangled data structures into what goes into the pch, and what does not. Obviously, nothing in the pch could point outside of it. 2. .h files are simply not compatible with this, so you've got to detect when it won't work. For example, anything like a command line switch or a macro that might cause different code to be generated in the pch had to invalidate it. Maybe I should have done your fork idea? :-) My experience with this drove many design decisions for D modules, for example, D modules are unaffected by where they are imported, the order they are imported, or the number of times they are imported. (Yes, I know about https://digitalmars.com/d/archives/digitalmars/D/D_needs_to_be_honest_320976.html) Anyhow, what I've thought about doing since the beginning was make DMD multithreaded. The language is designed to support multithreaded compilation. For example, lexing, parsing, semantic analysis, optimization, and code generation can all be done concurrently. DMD 1.0 would read imports in a separate thread. This would speed things up if you were using a slow filesystem, like NAS or a USB stick, but it was eventually disabled because there wasn't a perceptible speedup with current filesystems. Wouldn't it be awesome to have the lexing/parsing of the imports all done in parallel? The main difficulty in getting that to work is dealing with the shared string table.
LDC 1.13.0-beta2
Glad to announce the second beta for LDC 1.13: * Based on D 2.083.0+ (yesterday's DMD stable). * The Windows packages are now fully self-sufficient, i.e., a Visual Studio/C++ Build Tools installation isn't required anymore. * Substantial debug info improvements. * New command-line option `-fvisibility=hidden` to hide functions/globals not marked as export, to reduce the size of shared libraries. Full release log and downloads: https://github.com/ldc-developers/ldc/releases/tag/v1.13.0-beta2 Thanks to all contributors!
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 09:46:44 UTC, Walter Bright wrote: It works by allocating memory from a memory-mapped file, which serves as the precompiled header. Hey, that's a great idea! Can we do this for DMD? :D On a more serious note: do you think that with D's features (type system / metaprogramming), you could have avoided some of those bugs? For example, one thing we can do in D which is still impossible in C++ is to automatically serialize/deserialize all fields of a struct/class (using tupleof / allMembers).
Re: D compilation is too slow and I am forking the compiler
On 11/21/2018 12:07 AM, Vladimir Panteleev wrote: https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ I implemented precompiled headers for Digital Mars C++. It took a lng time to work all the bugs out of it. It's also a brittle system. It works by allocating memory from a memory-mapped file, which serves as the precompiled header.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote: https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ You might want to have a brush up on which direction C++ modules are heading in. Notable talks would be those given at the GNU Cauldron for both 2017 and 2018. The general run-down as I understand it. === Problem to solve: Compiler asks an Oracle about module A. Phrased this way, Compiler is a client, Oracle is a server. Oracle could be a file, socket, remote server, anything that can be read from or written to. Communication can be done via a standard format (such as json). This means that the Oracle (the implementation of) that keeps track of compilation and dependencies of the build is now someone else's problem as far as the Compiler is concerned. === I think what you've already started would fit well into this. Iain.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote: https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ Not only an interesting read, but also interesting research!
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 08:32:39 UTC, Nicholas Wilson wrote: You gave me a fright there with the title there for a moment. :) Awesome stuff though. Not sure how easy it will be to upstream considering this needs to not wreck Windows and needs to work with LDC/GDC (at least we have inlining in the backend). All the DMD-side logic is all encapsulated in one function: https://github.com/CyberShadow/dmd/blob/dmdforker/src/dmd/mars.d#L501-L673 Its body can be versioned out in incompatible platforms/implementations.
Re: D compilation is too slow and I am forking the compiler
On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote: https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ You gave me a fright there with the title there for a moment. Awesome stuff though. Not sure how easy it will be to upstream considering this needs to not wreck Windows and needs to work with LDC/GDC (at least we have inlining in the backend).
D compilation is too slow and I am forking the compiler
https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/
