Re: Usability of D on windows?
On Thursday, 25 August 2016 at 20:31:59 UTC, John Burton wrote: I'll try to get a self contained report tomorrow. Please do – the -deps switch is certainly not the most well-tested part of LDC. It mostly inherits this part of the code from DMD, but there might be a subtle, unintentional difference hiding somewhere. (There is also https://github.com/ldc-developers/ldc/issues/1625.) — David
Re: Why don't we switch to C like floating pointed arithmetic instead of automatic expansion to reals?
On Saturday, 6 August 2016 at 21:56:06 UTC, Walter Bright wrote: Let me rephrase the question - how does fusing them alter the result? There is just one rounding operation instead of two. Of course, if floating point values are strictly defined as having only a minimum precision, then folding away the rounding after the multiplication is always legal. — David
Re: Why don't we switch to C like floating pointed arithmetic instead of automatic expansion to reals?
On Saturday, 6 August 2016 at 09:35:32 UTC, Walter Bright wrote:
The LDC fastmath bothers me a lot. It throws away proper NaN
and infinity handling, and throws away precision by allowing
reciprocal and algebraic transformations.
This is true – and precisely the reason why it is actually
defined (ldc.attributes) as
---
alias fastmath = AliasSeq!(llvmAttr("unsafe-fp-math", "true"),
llvmFastMathFlag("fast"));
---
This way, users can actually combine different optimisations in a
more tasteful manner as appropriate for their particular
application.
Experience has shown that people – even those intimately familiar
with FP semantics – expect a catch-all kitchen-sink switch for
all natural optimisations (natural when equating FP values with
real numbers). This is why the shorthand exists.
— David
Re: Why don't we switch to C like floating pointed arithmetic instead of automatic expansion to reals?
On Saturday, 6 August 2016 at 12:48:26 UTC, Iain Buclaw wrote: There are compiler switches for that. Maybe there should be one pragma to tweak these compiler switches on a per-function basis, rather than separately named pragmas. This might be a solution for inherently compiler-specific settings (although for LDC we would probably go for "type-safe" UDAs/pragmas instead of parsing faux command-line strings). Floating point transformation semantics aren't compiler-specific, though. The corresponding options are used commonly enough in certain kinds of code that it doesn't seem prudent to require users to resort to compiler-specific ways of expressing them. — David
Re: Why don't we switch to C like floating pointed arithmetic instead of automatic expansion to reals?
On Saturday, 6 August 2016 at 10:02:25 UTC, Iain Buclaw wrote: No pragmas tied to a specific architecture should be allowed in the language spec, please. I wholeheartedly agree. However, it's not like FP optimisation pragmas would be specific to any particular architecture. They just describe classes of transformations that are allowed on top of the standard semantics. For example, whether transforming `a + (b * c)` into a single operation is allowed is not a question of the target architecture at all, but rather whether the implicit rounding after evaluating (b * c) can be skipped or not. While this in turn of course enables the compiler to use FMA instructions on x86/AVX, ARM/NEON, PPC, …, it is not architecture-specific at all on a conceptual level. — David
Re: Please rid me of this goto
On Friday, 24 June 2016 at 00:26:52 UTC, John Colvin wrote: My biggest bugbear is actually the opposite of what you are point out here: people doing careful benchmarking and asm-inspection of small code-fragments in isolation when in reality it is always going to be inlined and optimised in context. Yep. This is why I was so adamant on using real-world code for my performance tracking project, in case anybody has heard me going on about that. — David
Re: Please rid me of this goto
On Thursday, 23 June 2016 at 23:34:54 UTC, David Nadlinger wrote: I get the following results: (This is of course not intended to be a comprehensive analysis. For example, the codegen for the two functions is actually identical on GDC and LDC, the relative differences are due to measurement noise. — David)
Re: Please rid me of this goto
On Thursday, 23 June 2016 at 22:08:20 UTC, Seb wrote:
[1] https://github.com/wilzbach/perf-d/blob/master/test_pow.d
[2] https://github.com/wilzbach/perf-d/blob/master/test_powi.d
This is a bad way to benchmark. You are essentially testing the
compiler's ability to propagate your constants into the
benchmarking loop/hoisting the code to be benchmarked out of it.
For cross-compiler tests, you should define the candidate
functions in a separate compilation units with an extern(C)
interface to inhibit any optimisations. In this case, your code
could e.g. be altered to:
---
import std.math : pow;
extern(C) long useStd(long a, long b) { return pow(a, b); }
extern(C) long useOp(long a, long b) { return a ^^ b; }
---
extern(C) long useStd(long a, long b);
extern(C) long useOp(long a, long b);
void main(string[] args)
{
import std.datetime: benchmark, Duration;
import std.stdio: writeln, writefln;
import std.conv: to;
long a = 5;
long b = 25;
long l = 0;
void f0() { l += useStd(a, b); }
void f1() { l += useOp(a, b); }
auto rs = benchmark!(f0, f1)(100_000_000);
foreach(j,r;rs)
{
version(GNU)
writefln("%d %d secs %d ms", j, r.seconds(),
r.msecs());
else
writeln(j, " ", r.to!Duration);
}
// prevent any optimization
writeln(l);
}
---
(Keeping track of the sum is of course no longer really
necessary.)
I get the following results:
---
$ gdc -finline-functions -frelease -O3 -c test1.d
$ gdc -finline-functions -frelease -O3 test.d test1.o
$ ./a.out
0 0 secs 620 ms
1 0 secs 647 ms
4939766238266722816
---
---
$ ldc2 -O3 -release -c test1.d
$ ldc2 -O3 -release test.d test1.o
$ ./test
0 418 ms, 895 μs, and 3 hnsecs
1 409 ms, 776 μs, and 1 hnsec
4939766238266722816
---
---
$ dmd -O -release -inline -c test1.d
$ dmd -O -release -inline test.d test1.o
0 637 ms, 19 μs, and 9 hnsecs
1 722 ms, 57 μs, and 8 hnsecs
4939766238266722816
---
— David
Re: Is dmd fast?
On Thursday, 23 June 2016 at 17:24:34 UTC, Andrei Alexandrescu wrote: On my Ubuntu, /usr/bin/ld -> x86_64-linux-gnu-ld. What does that mean? -- Andrei `ld --version` should clear that up. ;) — David
Re: Phobo's migration
On Thursday, 23 June 2016 at 12:57:47 UTC, Steven Schveighoffer wrote: […] I believe Weka.io does not [use Phobos] either. Weka actually uses Phobos quite liberally – `fgrep -rI "import std." weka` yields several thousand results (I'm sure Liran will be happy to share more details). It's just that there are also quite a few areas where they need more than Phobos currently offers. Examples would be advanced data structures/containers or workarounds allowing the Phobos algorithms to be used without GC-allocating closures for the predicate alias parameters. — David
Re: problem using ldc 1.0.0 as D compiler
On Tuesday, 21 June 2016 at 21:44:59 UTC, Dicebot wrote: Would you mind making a point release with it if fix is confirmed? That's Kai's territory, but I'm sure he'll agree that this a sensible thing to do. — David
Re: problem using ldc 1.0.0 as D compiler
On Tuesday, 21 June 2016 at 16:26:14 UTC, Satoshi wrote: On Monday, 20 June 2016 at 11:28:58 UTC, Antonio Corbi wrote: Hi folks! I'm using ldc version: LDC - the LLVM D compiler (1.0.0): based on DMD v2.070.2 and LLVM 3.8.0 built with DMD64 D Compiler v2.071.0 Default target: x86_64-unknown-linux-gnu Host CPU: core2 [...] I have the same problem with LDC 1.1.0 on Arch Linux A fix for this just landed in Git master: https://github.com/ldc-developers/ldc/commit/6de981af757b00a91b36ae94d9ce9a8a81cfabf9 It would be great if you could build LDC yourself to verify that the issue is gone – be sure to open another ticket if the issue persists! — David
Re: Fiber implementation questions
On Tuesday, 21 June 2016 at 16:12:13 UTC, Steven Schveighoffer wrote: On 6/21/16 11:43 AM, Dicebot wrote: On Tuesday, 21 June 2016 at 12:18:37 UTC, Steven Schveighoffer wrote: On 6/21/16 6:14 AM, Dicebot wrote: httpthub.com/dlang/druntime/blob/master/src/core/thread.d#L1611 I suspected something like this, it explains a lot. Do you know why this approach was chosen instead of simply registering allocated memory for GC scan? Does GC has special approach for stack scanning which differs from normal memory? There must be a reason, I think maybe Martin or Sean would know more. You don't want to be scanning the unused part of the stack region. -David
Re: Phobos: __FILE__ as template default parameter
On Monday, 20 June 2016 at 08:10:19 UTC, Dicebot wrote: How about defining semantics like "try inlining if possible, fallback to always emitting symbol to object file otherwise"? That would also allow compatible implementation in dmd. This would get rid of the undefined symbols, but there is a much more subtle problem here that nobody has brought up yet: If the template ends up being emitted twice with different mangled names, then, well, it ends up existing twice in the final executable. This is not really an issue for functions, but very much so for data symbols (e.g. a static variable inside a function), where you could end up with the same alias referring to two different pieces of data depending on where it is used from. The root of this issue is that __FILE__ introduces incidental environmental state into the otherwise pure module system. — David
Re: problem using ldc 1.0.0 as D compiler
Hi Antonio, On Monday, 20 June 2016 at 11:28:58 UTC, Antonio Corbi wrote: ./ldcfail Fatal Error while loading '/usr/lib/libphobos2-ldc.so.70': The module 'std.regex.internal.parser' is already defined in './ldcfail'. […] Is this a bug in ldc or is it somehow related to my code? Thanks for your help. This is a bug in LDC, or rather its interplay with certain system linker versions. This issue has also been reported at https://github.com/ldc-developers/ldc/issues/1534, but we are not sure how to reproduce it yet. Could you please add your test case and system details there? (Of particular interest are the version of the Linux distribution you are using, and `ld --version`.) As a workaround, you can always use Phobos as a static library in the meantime (e.g. from our the binary release packages). — David
Re: Parameterized inheritence issues, bug or ignorance?
On Monday, 20 June 2016 at 00:34:18 UTC, Joerg Joergonson wrote: Can I just force the cast in some way if I know good an well it works for ever? Or do I have to write extra code to get around axiom: "We did this to save you trouble because we know better than you what you are doing. Go forth and write more code!"? If you want to convert between class references by means of pointer repainting, you can do so by casting to void* in between. As a side note, you might want to revisit your OOP terminology – in both this and the dm.D.learn thread, you are referring to down/upcasting in the reverse sense compared to the common definition. — David
Re: Parameterized inheritence issues, bug or ignorance?
On Monday, 20 June 2016 at 00:01:58 UTC, Joerg Joergonson wrote:
class a { }
class b : a { }
class A(T : a)
{
T x;
}
void main(string[] argv)
{
auto y = new A!a();
auto z = new A!b();
y.x = new a();
z.x = new b();
auto p1 = cast(A!a)y;
auto p2 = cast(A!b)z;
auto p3 = cast(A!a)z; // Fail! Why? z.x is of type b which can
be down cast to type a without a problem ->
As has been pointed out in dm.D.learn already (which is where
this discussion belongs), there isn't any inheritance
relationship between A!b and A!a.
In your example, it's easy to see that there can't be if you ask
yourself whether the purported relation should be co- or
contravariant in nature. To put it in simple terms, if your cast
succeeded, you could do this:
---
p3.x = new a;
// Now typeof(z.x) == b, but it points to an instance of a!
---
— David
Re: Phobos: __FILE__ as template default parameter
On Sunday, 19 June 2016 at 21:13:00 UTC, Johan Engelen wrote: On Sunday, 19 June 2016 at 21:11:59 UTC, Johan Engelen wrote: (I think we can pretty much inline anything the user throws at us) (as long as it is not a naked asm function) Another example is `alloca`, which you might not want to inline. — David
Re: GPGPU work and Identifiers
On Sunday, 19 June 2016 at 20:20:38 UTC, David Nadlinger wrote: Such a separate driver could then also be used as part of the regular distribution to improve the performance on memory-constrained all-at-once builds, by freeing all the frontend/LLVM memory (i.e., terminating the process) before invoking the linker, which can take a substantial amount of memory for huge binaries too. (This is not likely to be worth the trouble on Windows, where – partly depending on the user's configuration – process creation can take ages. — David)
Re: GPGPU work and Identifiers
On Sunday, 19 June 2016 at 18:23:06 UTC, Jakob Bornecrantz wrote: That will be annoying, the LDC that works with SPIR-V wont be able to produce MSVC compatible exes. Lets hope they mainline it soon. One possible solution for this would be a thin driver executable that parses just enough of the command line to detect the target triple used, and then forwards to the correct "real" compiler executable. Such a separate driver could then also be used as part of the regular distribution to improve the performance on memory-constrained all-at-once builds, by freeing all the frontend/LLVM memory (i.e., terminating the process) before invoking the linker, which can take a substantial amount of memory for huge binaries too. — David
Re: Weird compiler
On Sunday, 19 June 2016 at 14:05:22 UTC, mogu wrote: It's awful that I compile a little 64bit program(or -m32mscoff) in windows must have visual studio which has tremendous size installed even though I only need a linker. It's weird that a compiler compiles to binary targets needs another compiler or tools. Sigh. Microsoft now releases the Visual C++ Build Tools separately, which contain their C++ toolchain without the IDE parts (https://blogs.msdn.microsoft.com/vcblog/2016/03/31/announcing-the-official-release-of-the-visual-c-build-tools-2015/). — David
Re: Version identifier for PS4
On Friday, 10 June 2016 at 14:37:23 UTC, Markus Pursche wrote: If this is the case we would want to use Orbis. As Seb also pointed out on GitHub, I've seen LLVM use "PS4" internally, and it always seemed rather natural to me. On the other hand, with Orbis I would have had no idea what the code was about, and it's not like you can really install another operating system on a PS4 anyway. (Yes, I'm aware of the Jailbreak/Homebrew community, but that's rather besides the point.) — David
Re: Let's avoid range specializations - make ranges great again!
On Saturday, 4 June 2016 at 17:35:23 UTC, Seb wrote: ldc does the job (nearly) correctly. Scratch the "nearly" – at least you can't infer that from the performance results alone, the differences are well within the run-to-run noise. dmd -release -O -boundscheck=off test_looping.d && ./test_looping You seem to be missing `-inline` here, although it doesn't seem to influence the results (DMD 2.071.0/OS X x86_64). 2) Should it matter whether I use r.save or r[i..$]? Answer: it does for dmd and ldc. […] Open question: how can we make DMD/LDC smarter, so that they inline range primitives […]? Is this really the conclusion to draw from this benchmark? First of all, unless I'm mistaken, the three implementations are not equivalent – f_for seems to ignore the first element by doing an extra `popFront()`. Fixing that, the implementations seem to be essentially equivalent in terms of execution time (LDC master, i7-4980HQ@2.8GHz): ``` 0 17 secs, 840 ms, 770 μs, and 9 hnsecs 1 16 secs, 680 ms, 80 μs, and 6 hnsecs 2 17 secs, 635 ms, 548 μs, and 1 hnsec ``` Even though this is using a larger number of iterations for illustrative purposes, the difference still is to be in the run-to-run noise. Comparing the x86_64 assembly produced for 1 and 2, everything is inlined correctly in both cases. The "reversed" loop structure does lead to a slightly higher number of instructions in the loop, though, which might or might not be measurable: --- __D9test_save17__T9f_foreachTAmZ9f_foreachFNaNbNiNfAmmZAm: xor eax, eax testrsi, rsi je LBB11_4 mov rcx, rdx .align 4, 0x90 LBB11_2: cmp qword ptr [rcx], rdi je LBB11_5 inc rax add rcx, 8 cmp rsi, rax ja LBB11_2 LBB11_4: mov rax, rsi ret LBB11_5: sub rsi, rax mov rax, rsi mov rdx, rcx ret __D9test_save13__T5f_forTAmZ5f_forFNaNbNiNfAmmZAm: testrsi, rsi je LBB13_4 mov rax, rsi dec rax mov rcx, rdx add rcx, 8 .align 4, 0x90 LBB13_2: cmp qword ptr [rcx - 8], rdi je LBB13_4 mov rdx, rcx mov rsi, rax dec rax add rcx, 8 cmp rax, -1 jne LBB13_2 LBB13_4: mov rax, rsi ret --- — David
Re: faster splitter
On Tuesday, 31 May 2016 at 21:29:34 UTC, Andrei Alexandrescu wrote: You may want to then try https://dpaste.dzfl.pl/392710b765a9, which generates inline code on all compilers. -- Andrei In general, it might be beneficial to use ldc.intrinsics.llvm_expect (cf. __builtin_expect) for things like that in order to optimise basic block placement. (We should probably have a compiler-independent API for that in core.*, by the way.) In this case, the skip computation path is probably small enough for that not to matter much, though. Another thing that might be interesting to do (now that you have a "clever" baseline) is to start counting cycles and make some comparisons against manual asm/intrinsics implementations. For short(-ish) needles, PCMPESTRI is probably the most promising candidate, although I suspect that for \r\n scanning in long strings in particular, an optimised AVX2 solution might have higher throughput. Of course these observations are not very valuable without backing them up with measurements, but it seems like before optimising a generic search algorithm for short-needle test cases, having one's eyes on a solid SIMD baseline would be a prudent thing to do. — David
Re: The Case Against Autodecode
On Friday, 27 May 2016 at 22:12:57 UTC, Minas Mina wrote: Those should be the same though, i.e compare the same. In order to do that, there is normalization. What is does is to _expand_ the single codepoint Ä into A + ¨ Unless I'm mistaken, this depends on the form used. For example, in NFKC you'd get the single codepoint Ä. — David
Re: faster splitter
On Friday, 27 May 2016 at 14:06:09 UTC, Chris wrote: I have to correct myself. A manual loop is faster, I couldn't believe it either, so I benchmarked the same function with `foreach` and `for`. Turns out that `for` is _way_ faster. Just about the only reason I could think of for this to happen is if the compiler fails to inline the range primitives from std.array. Otherwise, the loops should be pretty much equivalent to LLVM's optimiser. This is so fundamental to D's strategic promise that we can't afford to get it wrong. — David
Re: Why doesn't dlang-bot use the GitHub conventions?
On Wednesday, 18 May 2016 at 14:27:22 UTC, Jacob Carlborg wrote: Apparently dlang-bot doesn't recognize the GitHub syntax/conventions [1] to link and close issues from pull requests. Instead one have to use "Fix issue ...". I don't see a point in inventing new conventions for this. Can we please have dlang-bot recognize the GitHub syntax as well? [1] https://help.github.com/articles/closing-issues-via-commit-messages/ dlang-bot is not related to the Bugzilla integration proper. Martin wrote it to have links back to the respective issues automatically available somewhere on the PR page, but it's a separate tool. - David
Re: File size of exe: DMD vs LDC
On Thursday, 12 May 2016 at 17:27:37 UTC, Nick Sabalausky wrote: FWIW, keep in mind that doesn't necessarily imply the same 2x scale will still hold for larger programs. Could be as much as 2x, could be as little as a constant +3MB. In typical programs, it is indeed a factor, and can be considerably more than 2x too. I've seen 400 MiB vs. 1.3 GiB, for example. — David
Re: File size of exe: DMD vs LDC
On Thursday, 12 May 2016 at 21:35:12 UTC, David Nadlinger wrote: In typical programs it is indeed a factor, and can be considerably more than 2x too. […] To expand on that a bit, if you write template- and CTFE-less programs with no unused code and no inlined functions, you would obviously expect much less gain from linker-level dead code/data elimination than for a typical D2 program. — David
Re: File size of exe: DMD vs LDC
On Monday, 9 May 2016 at 13:53:45 UTC, Dicebot wrote: On Monday, 9 May 2016 at 13:48:50 UTC, Walter Bright wrote: On 5/9/2016 5:24 AM, Jacob Carlborg wrote: LDC can strip sections which DMD cannot. Why not? AFAIR, because of how ModuleInfo/TypeInfo is emitted which makes linker never collect sections because they are referenced at least from there. LDC was intentionally modified to change code gen in that regard to make --gc-sections work - David will surely tell more :) It's something along those lines. What is certainly true is that I spent quite some time trying to work around "interesting" linker behaviour to make this work. You can find more details on my solution at https://issues.dlang.org/show_bug.cgi?id=879. I haven't looked at DMD's related codegen in a while, so I don't know what exactly remains to be done there. — David
Re: [OT] Swift removing minor features to piss me off
On Thursday, 28 April 2016 at 18:49:54 UTC, Steven Schveighoffer wrote: Please, D, don't ever do this kind of stuff! I just gained about 45 warnings in my iOS project. … and what? This statement alone is hardly an argument. Both those warnings are trivial to fix in an automated fashion; in fact, you pointed out the possible rewrites yourself. Fixing the warnings and verifying the changes takes way less time than tracing down and addressing even a simple application bug, yet the language is substantially cleaner as a result. This is exactly the sort of changes D needs more of. — David
Re: Checking if an Integer is an Exact Binary Power
On Sunday, 24 April 2016 at 23:00:56 UTC, Temtaime wrote:
Please no cmp.
Just
bool isPowerOf2(uint x) { return x && !(x & (x - 1)); }
You do realise that this will (typically) emit a branch?
— David
Re: Checking if an Integer is an Exact Binary Power
On Saturday, 23 April 2016 at 20:34:52 UTC, Nordlöw wrote: So is there a way to check if popcnt builtin is available on current platform? As Andrei pointed out, it's not only popcnt being available, it's also it being fast. A function implementing the classic hand-written version compiles down to something like lea eax, [rdi - 1] testeax, edi seteal ret which is already quite good. There is also blsi to consider on Haswell and above. — David
Re: DMD internal: appendToModuleMember performance
On Saturday, 23 April 2016 at 09:13:01 UTC, Johan Engelen wrote: Well, this 140k list has a removal very early on, so the linear search is going to happen >100k times from then on ;) No – only for searching the particular removed item (or other false positives due to hash collisions). — David
Re: DMD internal: appendToModuleMember performance
On Friday, 22 April 2016 at 23:49:22 UTC, Walter Bright wrote: BTW, this looks like a particularly bad piece of engineering. The trouble is, it saves an index to the member, then does a bunch of semantic processing, then deletes what is on that index. But what if the members[] in the meantime shifts around? There is an assert for that case, so it at least won't cause corruption, but it looks bad. IIRC I was involved in this somehow. It has been some time, but I think the deal is that it only does so when all the other members that might have been added in the meantime have also been removed. I suppose it seemed like a good idea to save that local information locally instead of increasing the size of TemplateInstance by a pointer, of which there might be a bajillion instances in a typical "modern" D program. — David
Re: DMD internal: appendToModuleMember performance
On Friday, 22 April 2016 at 21:38:41 UTC, Johan Engelen wrote: I don't understand exactly what you mean; do you propose to […] do a linear search when the shadow data structure says the item is present? That's the idea. As long as you can reduce the need to do a full linear search by, say, 1 : 1000, it would be enough to make the quadratic behaviour disappear into the noise. That being said, I did some estimates, and since the keys are only 8 bytes in size, Bloom filters in particular might not be worth it in this case, unfortunately. Quite sad, actually. I like Bloom filters. — David
Re: DMD internal: appendToModuleMember performance
On Friday, 22 April 2016 at 22:10:47 UTC, Walter Bright wrote: On 4/22/2016 2:38 PM, Johan Engelen wrote: I don't understand exactly what you mean; do you propose to resort to linear search after a removal happened? Or only do a linear search when the shadow data structure says the item is present? I don't know how often removals happen, but for the 140k elements list removals happens _very_ often. While compiling phobos, removals happen not for all modules, but for quite a few of them. I did a grep for "members.remove" and got no hits. Where are the removes happening? One of them is https://github.com/dlang/dmd/blob/5ea445c68451152d43595c9de4797b6ec1e4f57d/src/dtemplate.d#L6503, I think. — David
Re: DMD internal: appendToModuleMember performance
On Friday, 22 April 2016 at 21:48:09 UTC, Walter Bright wrote: Why not just use a hash table? D's builtin one? Apparently, some parts rely on the insertion order, although I'm not convinced they should. Module::importAll is one of them, but in a way that's trivial to avoid. I didn't check any other locations. If order is not important, a hash set should be fine. — David
Re: DMD internal: appendToModuleMember performance
On Friday, 15 April 2016 at 18:33:44 UTC, Johan Engelen wrote: I'm thinking about removing the old array all-together. My question is: is it essential to keep an ordered list? (I see a `.shift(...)` call on the array, to put something in first position. If so, could that be solved by having *two* trees, where "in-order" iteration first iterates over the first one and then the second. The "high priority" symbols can then be put in the first tree, normal ones in the second tree (if that is how order matters...). Did you have a closer look at where this is done? One place where order matters is for `object` in `importAll`, but that's trivially rewritten a different way. — David
Re: DMD internal: appendToModuleMember performance
On Friday, 22 April 2016 at 18:51:57 UTC, David Nadlinger wrote: As long as elements are not removed too frequently (what do your numbers say?), the performance impact of doing a full linear search in those cases shouldn't be too bad. Note that a (properly tuned) probabilistic data structure like a Bloom filter allows you to avoid having to keep a full second copy of the list around, hopefully making it possible to keep the optimisation on by default without regrets. (For tiny modules with only a small number of members, you can still do the linear search to skip the hash lookup.) — David
Re: DMD internal: appendToModuleMember performance
On Saturday, 16 April 2016 at 13:58:28 UTC, Johan Engelen wrote: On Friday, 15 April 2016 at 19:32:46 UTC, David Nadlinger wrote: Another "quick fix" if we have to keep the order would be to add a Bloom filter/… on the side to eliminate most array searches. In rare cases, symbols are removed from the members list, so the shadow data structure needs the ability to delete elements. Bloom filters can have false positives anyway. As long as elements are not removed too frequently (what do your numbers say?), the performance impact of doing a full linear search in those cases shouldn't be too bad. — David
Re: Distributor's whishlist and questions for D
On Friday, 22 April 2016 at 09:18:48 UTC, Kagamin wrote: RMW operations on shared data are deprecated. The template filter on atomicOp is a little different: https://dlang.org/phobos/core_atomic.html#.atomicOp Looks like an LDC bug. Yes, this was indeed an LDC issue. We have our own implementation, and apparently missed that the template constraint on the DMD version was updated: https://github.com/ldc-developers/ldc/pull/1456 — David
Re: DMD internal: appendToModuleMember performance
On Friday, 15 April 2016 at 18:33:44 UTC, Johan Engelen wrote: Hi all, When building a unittest case in Weka.io's codebase, I measure that appendToModuleMember takes about 10% of the total compile time, and about 25% of total semantic analysis time. "appendToModuleMember" is the only function that pops up in profiling with such a large time fraction spent in it. Yep, see also: https://issues.dlang.org/show_bug.cgi?id=15323. When I tried to get rid of the ordered array, I ran into several interesting issues, but I don't remember many details. I'm thinking about removing the old array all-together. My question is: is it essential to keep an ordered list? (I see a `.shift(...)` call on the array, to put something in first position. If so, could that be solved by having *two* trees, where "in-order" iteration first iterates over the first one and then the second. The "high priority" symbols can then be put in the first tree, normal ones in the second tree (if that is how order matters...). Another "quick fix" if we have to keep the order would be to add a Bloom filter/… on the side to eliminate most array searches. — David
Re: Policy for exposing range structs
On Saturday, 26 March 2016 at 05:22:56 UTC, Andrei Alexandrescu wrote: Compression of template names. -- Andrei Compression in the usual sense won't help. Sure, it might reduce the object file size, but the full string will again have to be generated first, still requiring absurd amounts time and space. The latter is definitely not negligible for symbol names of several hundreds of kilobytes; it shows up prominently in compiler profiles of affected Weka builds. — David
Re: RFC: Units of measurement for D (Phobos?)
On Wednesday, 16 March 2016 at 17:12:42 UTC, ZombineDev wrote: Sometimes a more gradual approach may be useful: […] How would this be any more than an informational annotation? If anything, you could build a custom lint tool to propagate and verify the unit annotations, but I don't quite see how the type system/compiler would make use of it. — David
Re: D could catch this wave: web assembly
On Saturday, 19 March 2016 at 16:18:00 UTC, Adi wrote: I need help to write my proposal for gsoc. What exactly do you need help with? You can find some general hints at the GSoC student guide: http://write.flossmanuals.net/gsocstudentguide/writing-a-proposal/. There are currently no extra hard rules for the D foundation, at least not this year. As a general note, though – I'm not involved with GSoC this year, neither as a student nor a mentor –, your proposal should demonstrate that you have researched the project at hand somewhat carefully (including the formal aspect!) and should convey that you will be able to pull your own weight over the course of the summer. — David
Re: RFC: Units of measurement for D (Phobos?)
On Wednesday, 16 March 2016 at 22:16:20 UTC, Nordlöw wrote: How should then sin, cos, tan, expi be redesigned to support `ScaledUnit` aswell as `Quantity`? What you can do is to take an arbitrary quantity with an (angle.canConvert!radian) template constraint. Inside the function, you then simply use cos(angle.convert!radian / radian) or whatever. — David
Re: RFC: Units of measurement for D (Phobos?)
Hi Per, Many thanks for working on the code! Could you put it up as a separate Dub package to make it easier for people to try out? `justd` is an 68 MiB-heavy clone, which makes it somewhat cumbersome to play around with and contribute to it. You'd probably also want to add your name to the Authors list. On Wednesday, 16 March 2016 at 14:27:16 UTC, Nordlöw wrote: - UFCS (!) Done There might be additional design possibilities now that didn't previously exist. The way of constructing a quantity is one such example, although I still like the approach using `*`. - UDAs Could you elaborate on possible usage in this case? For example, to replace SuperSecretAliasToMarkThisAsUnit. It could conceivably also be used for the extra metadata like the base linear unit for affine units, etc. - Extracting template symbol/arguments from template instances Could you elaborate on possible usage in this case? IIRC there are various `isXyz` templates where I went with a duck typing approach out of necessity (which might or might not be a better design choice than just checking for a certain template instance). - Eponymous templates with extra members Could you elaborate on possible usage in this case? The code in and around GetConversion can probably be simplified. It currently explicitly uses `.Result` members and so on. Another thing I forgot to mention in the previous post are documented unit tests. Pretty much all of the example blocks should probably be converted to it. The CTFE-related code could also use a closer look, for example makeIndexCtfe() is probably not needed any longer. Best, David
Re: RFC: Units of measurement for D (Phobos?)
On Thursday, 17 March 2016 at 12:51:28 UTC, Nordlöw wrote:
I believe `isConvertableTo` is a more Phobos-compliant naming,
right?
I went for canConvert since it is a bit shorter than
isConvertible and there is precedent for it in Phobos as well
(e.g. find/canFind). It's a matter of taste, though. If I were to
work on the library again today, I might just as well go for
isConvertible.
Is there are reason why `convert` wasn't named the shorter and
Phobos-compliant `to` instead?
One thing to consider is that convert!U differs from to!T in that
the latter returns something of type T, whereas the former
returns a Quantity!(U, …). This seems enough of a difference that
choosing the same name as the widely used std.conv.to might be
somewhat misleading (even though I usually advocate for using the
module system for disambiguating names).
function, you then simply use cos(angle.convert!radian /
radian) or whatever.
Why not simply add a Quantity member […] toValue()
This is again a design decision that can go either way. The
choice I made was deliberate, although of course not above debate:
Since I chose to have dimensionless quantities implicitly convert
to "plain" value types, it seemed superfluous to add another
primitive (`a / a.unit` is simple enough to write). Furthermore,
it underlines the point that `*` and `/` between quantities and
units do not actually change the numeric value, but merely modify
its units. Making use of that for the "core functionality" seemed
like a cute way to drive the point home to users. Explicitly
specifying the unit also helps avoiding mistakes in generic code.
If you want to go the other route, I would suggest carefully
deliberating the naming scheme, since discarding unit information
is a lossy and potentially "unsafe" (as in code correctness). In
an attempt to make this obvious in the current implementation
(where it is only enabled for dimensionless quantities), I named
the function rawValue(). For example, renaming (fromValue,
rawValue) to {from, to}Raw() might be clearer than {from,
to}Value().
By the way, I highly doubt that "puts less stress on the
compiler" is a valid argument for that decision. In that regard,
the big elephant in the room is GetConversion, which you might
want to re-do using a breadth-first search anyway (see TODO
comments; I couldn't be bothered to implement it back then due to
the numerous DMD bugs I had to fight).
— David
Re: RFC: Units of measurement for D (Phobos?)
On Tuesday, 15 March 2016 at 09:07:05 UTC, Nordlöw wrote:
On Monday, 14 March 2016 at 18:51:45 UTC, Chris Wright wrote:
Ohm my, that's awesome. Watt needs to happen to get this into
Phobos?
I'm cleaning up David's work right and will put up a PR today.
Please make a really, really thorough pass through the source
before proposing it for Phobos.
The code was written in a dark age where CTFE bugs were very
common (e.g. trying to use std.algorithm.sort would give weird
results), there was a number of issues with template parameters,
and many relevant features simply did not exist yet:
- UFCS (!)
- UDAs
- Extracting template symbol/arguments from template instances
- Natural alias syntax ("=")
- alias/enum/… templates
- Eponymous templates with extra members
- …
Back then, getting this to work with a half-reasonable API in
spite of all the compiler bugs and limitations was quite the tour
de force (IIRC some of the issues are marked using @@BUG@@ in the
code, but that was by far not all of them). With the current
compilers, it should be much more a question of design than of
implementation.
The basic design should still be sound, though. The main
decisions to make concern not so much the implementation (or
whether unit string parsing is implemented, etc.) but the level
of flexibility in the unit and conversion structure. One such
decision would be whether to store
a) the numerical value verbatim in the units specified by the
user, or
b) in some distinguished base unit for the given dimension.
Another such question is whether to use
1) a pre-defined, closed system of dimensions (e.g. the SI
units),
2) an explicit, but extensible structure of dimensions (to
which units can be assigned), or
3) units can arbitrarily be created and the concept of
dimensions emerges implicitly from conversions between them.
The less flexible solutions often have the advantage of allowing
a simpler implementation, being easier to understand for the
novice user (less types/templates) and possibly being quicker to
compile.
For my original library, the general motif was the to follow the
most expressive and flexible solution (3) and a) in the above).
For instance, on top of the obvious consequences regarding
precision, going with a) allows you to seamlessly interface with
non-unit-aware code and to define runtime conversions that are
only applied when actually requested by the user. Similar
considerations apply to other aspects of the design.
— David
Re: RFC: Units of measurement for D (Phobos?)
On Tuesday, 15 March 2016 at 14:46:30 UTC, D.Enguiyen wrote: Is it able to manage quantities with a non-linear transformations, for example when dealing with octet and byte ? (1024)!"byte" == 1!"kbyte"; There are no non-linear transformations going on here, following usual math terminology. If your question is whether unit conversions are possible with non-power-of-ten factors, the answer is an empathic "yes" – the scale factors could even be determined at run-time! The most basic solution would be to define a scaled unit like this: enum kibyte = scale!(byte, 1024, "kibibyte", "kiB"); You would probably want to define a power-of-two PrefixSystem though. (http://klickverbot.at/code/units/std_units.html#PrefixSystem) — David
Re: RFC: Units of measurement for D (Phobos?)
On Monday, 14 March 2016 at 19:08:18 UTC, Robert burner Schadek wrote: have a look at this! https://github.com/biozic/quantities This has been brought up (and subsequently discussed) on the first page of this thread, more than two years ago. — David
Re: D could catch this wave: web assembly
On Tuesday, 15 March 2016 at 20:18:40 UTC, deadalnix wrote: I can. I know LLVM fairly well (I'm not a committer), but I do not have that much experience with WebAssembly. Yes, please! I'd volunteer myself, but this summer will be too busy for me academically. — David
Re: dmd-built dmd is faster that ldc-built dmd or gdc-built dmd?
On Friday, 11 March 2016 at 15:09:26 UTC, Iain Buclaw wrote: Yes, I know. And if I were to say which behaviour is desired. I'd say GDC is doing the right thing. Marking all functions as 'weak' is a terrible idea on both DMD's and LDC's part. I distinctly remember Daniel and me having to remove those functions from DDMD when we first got it to build with LDC during/after DConf 2015. LDC definitely does not mark symbols as weak/comdat/etc. by default. However, the linker might not pull in an object file from a static library if the respective symbols have already been resolved using other files. — David
Re: Official compiler
On Friday, 26 February 2016 at 18:53:21 UTC, Andrei Alexandrescu wrote: My understanding is the main problem is the _same_ templates are repeatedly instantiated with the same exact parameters - the epitome of redundant work. -- Andrei Within one compiler execution, there might be some optimization potential in the way semantically equivalent template instantiations are merged, yes – it's been a while since I have looked at the related code (see e.g. TemplateDeclaration.findExistingInstance). Another area matching your description would be that of the same template being instantiated from multiple compilation units, where it can be omitted from some of the compilation units (i.e. object files). Our current logic for that is broken anyway, see e.g. https://issues.dlang.org/show_bug.cgi?id=15318. I was referring to something different in my post, though, as the question concerned "low-hanging fruit". The problem there is really just that template names sometimes grow unreasonably long pretty quickly. As an example, without wanting to divulge any internals, some of the mangled symbols (!) in the Weka codebase are several hundred kilobytes in size. core.demangle gives up on them anyway, and they appear to be extremely repetitive. Note that just like in Steven's post which I linked earlier, the code in question does not involve any crazy recursive meta-templates, but IIRC makes use of Voldemort types. Tracking down and fixing this – one would almost be tempted to just use standard data compression – would lead to a noticeable decrease in compile and link times for affected code. — David
Re: Official compiler
On Friday, 26 February 2016 at 18:19:57 UTC, Steven Schveighoffer wrote: The idea is that ldc and gdc will get plenty of warning if something breaks. As stated, this in itself would be utterly useless. Right now, you can be absolutely certain that the AST semantics will change in between each DMD release. Sometimes in obvious ways because fields are removed and so on, but much more often silently and in a hard-to-track-down fashion because the structure of the AST or the interpretation of certain node properties changes. In other words, we don't need any warning that something breaks, because we already know it will. The people that need the warning are the authors of the breaking front-end commits, so that they can properly document the changes and make sure they are acceptable for the other backends (right now, you typically have to reverse-engineer that from the DMD glue layer changes). Ideally, of course, no such changes would be merged without making sure that all the backends have already been adapted for them first. — David
Re: Official compiler
On Thursday, 25 February 2016 at 23:06:43 UTC, H. S. Teoh wrote: Are there any low-hanging fruit left that could make dmd faster? A big one would be overhauling the template mangling scheme so it does not generate mangled names a few hundred kilo (!) bytes in size anymore for code that uses templates and voldemort types. For an example, see http://forum.dlang.org/post/n96k3g$ka5$1...@digitalmars.com, although the problem can get much worse in big code bases. I've seen just the handling of the mangle strings (generation, ...) making up a significant part of the time profile. — David
Re: Official compiler
On Friday, 26 February 2016 at 11:50:27 UTC, Russel Winder wrote: On Fri, 2016-02-26 at 11:12 +, BBasile via Digitalmars-d wrote: […] BTW Malicious people can cheat and commit in the past, according to https://github.com/gelstudios/gitfiti commitment date is not reliable. Indeed, which is why Mercurial is a much better system, though it is far from perfect. "hg commit" knows the "--date" option just as well. Can we please keep this out of here? — David
Re: Official compiler
On Thursday, 25 February 2016 at 22:03:47 UTC, Walter Bright wrote: DMD did slow down because it was now being compiled by DMD instead of g++. You can compile it using LDC just fine now. ;) Also, dmd was doing multithreaded file I/O, but that was removed because speed didn't matter . Did we ever have any numbers showing that this in particular produced a tangible performance benefit (even a single barnacle)? As I said, keeping the compiler speed up is a constant battle. And this leaves me wondering why nobody ever wrote a comprehensive compiler performance tracking tool. There is Dash, my half-finished CI-style project (that a couple of people were interested in picking up after DConf, but which never really happened), and Vladimir's quite limited TrenD adaption of Mozilla's areweslimyet, but nobody really came up with something that would be part of our day-to-day development workflow. Currently, dmd makes zero user of multicore. I didn't know that ldc did. LDC doesn't do so either. I think what rsw0x referred to is doing a normal "C-style" parallel compilation of several compilation unit. I'm not sure why this couldn't also be done with DMD, though. — David
Re: Official compiler
On Thursday, 25 February 2016 at 03:05:21 UTC, Walter Bright wrote: On 2/18/2016 9:52 AM, Kai Nacke wrote: I really like the compiler diversity. Me too. Having 3 major implementations is a great source of strength for D. I like it too. I just think that we can't afford it at this point, and that this is a major impediment for improving the quality of the D ecosystem. — David
Re: Official compiler
On Thursday, 25 February 2016 at 02:58:08 UTC, Walter Bright wrote: A big chunk of that was getting D to catch C++ exceptions. And before I did this work, neither GDC nor LDC did, either. It's not a simple matter of just turning it on given Dwarf EH. That's beside the point, the C++ interop needed to be worked out either way and is not specific to the DMD backend. In that stupid example I gave, I was referring to the DWARF EH implementation itself, which will have taken you a non-negligible amount of time due to all the barely documented details, unless you are even more of a super-human compiler implementation expert than I already know you are. ;) Don't get me wrong, I couldn't care less about the details of how long it took whom to implement C++ EH interop (or the fact that it did exist before in LDC/Calypso, and in the form of prototypes for vanilla GDC/LDC, etc.). I'm only playing devil's advocate because many people here make it seem as if there was no cost to supporting multiple compilers, while there most definitely is. This ranges from the blatant duplication of work over PR issues to the fact that big language/compiler features are all but impossible to implement for anybody but you, since you are the only one who knows how to implement them on DMD (and in the current situation, not having them available in DMD would be a deal-breaker). Sure, the fact that you know all the nitty-gritty details of one backend might make implementing certain changes easier for you, as you pointed out. But the fact that this one backend is obscure compared to the usual suspects, poorly documented and license-encumbered pretty much ensures that you will remain the only person to tackle such projects in the future. — David
Re: Official compiler
On Thursday, 25 February 2016 at 09:04:17 UTC, Russel Winder wrote: I wonder if anyone has noticed, or appreciated that, all the trendy, hipster cloud based CI servers support Go, sometimes C++ and C (sort of), but not Rust, or D. Travis CI, which is probably the one "trendy, hipster" service most would think of, has been supporting D for quite some while now due to Martin Nowak's great work. (He put Iain's name and mine on it too, but we didn't really contribute at all.) Of course, there is always room for improving the integration with this and similar services. When I'm saying that dividing the attention between three compilers is a strategic mistake, it's not because I doubt that having multiple compilers is not a nice thing to have. It certainly is. But I'm convinced that expending the same amount of effort on the wider ecosystem would get us much farther. — David
Re: Swift deprecate i++ and c-style for loop
On Wednesday, 24 February 2016 at 16:37:04 UTC, Suliman wrote:
func slices() {
var array = ["First", "Second", "Third", "Fourth"]
array.removeLast()
array.removeFirst()
}
also look very intuitive.
Have a look at http://dlang.org/phobos/std_range.html#dropOne.
— David
Re: C++ UFCS update
On Friday, 19 February 2016 at 16:18:12 UTC, Dejan Lekic wrote: I have to go through each piece separated by dots to understand what it is... Let me play devil's advocate here: How would this be any different if UFCS were not used? — David
Re: Official compiler
On Thursday, 18 February 2016 at 17:56:32 UTC, Jonathan M Davis wrote: […] if you want to be writing scripts in D (which is really useful), you need rdmd, which means using dmd You can use rdmd with ldmd2 just as well (and presumably gdmd too). New users are frequently impressed by how fast dmd compiles code, and it's a big selling point for us. It's only later that benchmarking comes into play, and if want to do that, then use gdc or ldc. The download page already says to use gdc or ldc if you want better optimization. I'd claim that an equal number of people is put off by the sometimes abysmal performance of optimized DMD output in their initial tests and first toy projects. dmd is a clear winner as far as development goes. Clear only to somebody with x86-centric vision. I'm not claiming that the somewhat lower compile times aren't good for productivity. But being able to easily tap into the rich LLVM ecosystem or even just targeting the most widely used CPU architecture (in terms of units) is also something not to be forgotten when considering the development process. — David
Re: Official compiler
On Thursday, 18 February 2016 at 11:12:57 UTC, Jonathan M Davis wrote: And actually, he'd risk legal problems if he did, because he doesn't want anyone to be able to accuse him of taking code from gcc or llvm. That's a silly strawman, and you should know better than putting that forward as an argument by now. Walter is of course free to do whatever he pleases, and I would totally understand if his reason was just that it's hard to give something up you've worked on for a long time. But please don't make up argument trying to rationalize whatever personal decision somebody else made. You could literally copy LLVM source code into your application and sell it as a closed-source product without risking any copyright problems (if you comply with the very modest attribution clause of the license). If anything, the problem is probably that the gdc and ldc folks could use more help, but dmd and Phobos suffer from that problem on some level as well, albeit probably not as acutely. The problem that many of us are seeing is that D development is unnecessarily defocussed by spreading out the effort between three different compilers. Of course, ideally we would have infinite manpower. A "special-case" compiler that boasts lower compile times for x86 development would definitely be nice to have then. But our resources aren't limitless, and as such the question whether we can afford to maintain such a "nice to have"-compiler is very relevant. Don't get me wrong, I understand that there is an argument to be made for the current situation. And, by the way, let me make very clear that even if I argue that sticking to DMD is a strategic mistake, this is not about personal things. I highly respect Walter as a compiler developer and like him as a person. But perpetuating ill-informed arguments really doesn't do this debate any good. — David
[OT] Re: Official compiler
On Wednesday, 17 February 2016 at 22:57:20 UTC, Márcio Martins wrote: […] On a completely unrelated note, you aren't by any chance the Márcio Martins who is giving a talk at ETH in a couple of days, are you? — David
Re: Official compiler
On Thursday, 18 February 2016 at 11:41:26 UTC, Kai Nacke wrote: LLVM has about 2.5 million code lines. I am anything than sure if it is easy to improve compilation speed. I think you are a tad too pessimistic here. First, don't forget that there are some big LLVM customers for which low compile times are important too (remember all the buzz from when Clang first hit the scene?). Second, when was the last time you focussed on optimizing LDC -O0 compiler performance? There is currently a lot of low-hanging fruit, and then there are still the more involved options (such as making sure we use FastISel as much as possible). It might not end up quite as fast as DMD is right now. But imagine that Walter would have invested all the time he spent e.g. on implementing DWARF EH into optimizing the LDC frontend/glue layer/backend pass structure instead. Who knows, we might have an LDC-based compiler today that is faster than the DMD we currently have. — David
Re: Official compiler
On Thursday, 18 February 2016 at 12:23:18 UTC, Jonathan M Davis wrote: if the dmd backend were dropped, […] that would further slow down the development of the frontend and not necessarily improve things overall. How would that be? — David
Re: [suggestion] Automated one-stop compiler version chart
On Sunday, 7 February 2016 at 21:26:36 UTC, Xinok wrote: On Sunday, 7 February 2016 at 18:46:48 UTC, Nick Sabalausky wrote: I was just updating a project's .travis.yml file and noticed: It doesn't seem we have any one-stop-shop location to check all the versions of DMD/LDC/GDC currently available on travis-ci. […] The GDC downloads page has this info: http://gdcproject.org/downloads The page doesn't have the history of versions available at Travis though, which is what Nick asked for. I agree, by the way, something like that would definitely be nice to have. Somebody just needs to write a little scrapper set up to gather everything into a pretty table… — David
Re: D's equivalent to C++'s std::move?
On Wednesday, 3 February 2016 at 19:18:12 UTC, Minas Mina wrote: If this becomes the case, please make destructors nothrow so that people won't screw up (like they can very easily do in C++). This would be way too much of a restriction. Throwing in destructors is perfectly fine in D, there is nothing comparable to C++ where you can "screw up" in the sense that your program gets terminated when you throw an exception while another is already being unwound. — David
Literate programming tool (written in D) on Reddit
Just noticed this on the /r/programming front page: https://www.reddit.com/r/programming/comments/43fvbv/literate_programming_tool_made_with_d/ — David
Re: extern(C++, ns)
On Friday, 22 January 2016 at 17:04:00 UTC, Jacob Carlborg wrote: Unfortunately that's not how it works with D. If Andrei or Walter have a proposal it's up to the community to provide compelling evidence why it _should not_ be added. If the community has a proposal it's up to the community to provide compelling evidence why is _should_ be added. Just to make sure there are no misunderstandings: This is not at all the point I was trying to make. My arguments apply just as well to just a single person deliberating the design: Adding a new feature requires justification, especially if the language is quite large already. Not adding one does not to the same extent, because it is the natural, less risky choice. — David
Re: reduce -> fold?
On Saturday, 30 January 2016 at 17:40:38 UTC, Andrei Alexandrescu wrote: I forgot the distinction between currying and partial application. Can we also define currying in current D? -- Andrei Currying is turning (A, B, C) -> D into A -> (B -> (C -> D)), i.e. a function with multiple arguments into a sequence of functions that each take a single argument to apply each. I think I've implemented something like that for fun once, but never really found much use for it. In the few places where I could have used it (mostly binary functions), just using a lambda and partial application seemed to be much more idiomatic. I guess D lacks any idioms that would make its use come naturally. - David
Re: Idea: limited template expansion
On Thursday, 21 January 2016 at 13:36:28 UTC, Steven Schveighoffer wrote: On 1/20/16 6:01 PM, David Nadlinger wrote: How would the client code be simpler with a built-in language feature? The use case I have in mind is a parser, let's say an xml parser. […] I still don't see how a language feature as described in your first post would make this any easier than using a template for that exact purpose (switching between methods to call). If what you are trying to say is that you want the different template function instantiations to return incompatible types in addition to that, the feature from your initial post won't help you there either. Values can't have different types in non-template code, so you'd necessarily need to make the client code a template too. Streamlining the runtime -> compile time value dispatch process (as shown in your initial example) doesn't change the fact that the type of a given value cannot be influenced by runtime decisions. — David
Re: extern(C++, ns)
On Wednesday, 20 January 2016 at 23:21:49 UTC, Walter Bright wrote: On 1/20/2016 3:12 PM, Martin Drašar via Digitalmars-d wrote: The "serious" problem is that the decision to have a namespace introduce a new scope needlessly complicates writing D-like interfaces to C++ code, while catering for one specific use-case that is considered niche. Adding a qualifier here and there does not count as serious. But, to put this statement in the context of my other post, it further raises the bar which the arguments in favor of having extern(C++) introduce D-side scopes have to clear. If we had extern(C++, "ns") just affect the mangling, the need for adding extra qualifiers wouldn't exist in the first place. — David
Re: Request for new Deimos repository
On Thursday, 21 January 2016 at 00:33:34 UTC, Brian Schott wrote: If somebody creates a repository for this I can get these bindings into Deimos pretty quickly. IIRC only Walter, Andrei and Brad have the necessary permissions to create new repositories. — David
Re: extern(C++, ns)
On Tuesday, 19 January 2016 at 22:14:42 UTC, Walter Bright wrote: 1. C++ has namespaces. They went and invented a whole 'nother thing called modules. Evidently not even they think that modules and namespaces are the same thing. C++ modules (as per N4465, etc.) are a whole different story and entirely unrelated to the discussion. Their existence is neither an argument for or against having extern(C++) create named scopes for namespaces. 2. Multiple modules cannot have the same name in D. C++ practice is to litter code with namespaces using the same name. If you want to split up the interface to a C++ namespace among different modules, then the different symbols will *always* reside in different fully qualified scopes – with or without additional scopes being introduced by extern(C++)! The fact that modules are "sealed" scopes is an integral part of the D module system, and while your statement is true, I'm not sure how it constitutes an argument for having extern(C++) introduce more scopes in addition to that. 3. Modules have ModuleInfos, static constructors/destructors, etc. These do not make obvious sense for namespaces. How does this apply to the discussion? Again, extern(C++) code lives inside a D module either way. […] Actually, I'll skip commenting on the other points for now, because reading your last paragraph makes me suspect that there is a misunderstanding: I.e. if modules were bludgeoned into being namespaces, there would be such a list of special cases and confusions and exceptions and awkward crap you'd be far, far better off having namespace as a SEPARATE feature, because that's what they'd wind up being anyway, even if they were named "modules". From this statement, it seems as if you think we are arguing for modules somehow being mapped directly to C++ namespaces, in the vein of some hypothetical "module(C++, ns) foo;" declaration. However, this is not the case at all. Quite the contrary, my proposal (and, as far as I understand also that of Manu, Daniel and others who have agued against your design) is that C++ namespaces should have no semantic representation on the D side at all. We like our D modules as they are [1], and we are certainly not trying to bludgeon them into any other shape. Given the vast semantical differences between them and C++ namespaces – and you haven't even really mentioned the extensibility aspect and the new inline namespaces in your list – this would be a foolish thing to do. Modules only enter the picture in so far as they provide a natural way for client code of two C++ library wrappers to disambiguate between two symbols that have the same name, as it might happen if they correspond to C++ symbols from different namespaces. In other words, modules would likely quite often happen to provide the necessary means of disambiguation, because one would put the wrappers for two different C++ namespaces into different D modules anyway. But if you wanted to have, say, two declarations for interfacing with both A::foo() and B::foo() within the same D module, you could always use another of D's scoping mechanism for that, like (mixin) templates or whatever else tickles your fancy [2]. — David [1] That is, except for issue 314 and the visibility vs. accessibility problem. [2] Those scopes might of course currently leak into the C++ mangled name – I didn't check –, but this can be fixed.
Re: extern(C++, ns)
On Tuesday, 19 January 2016 at 21:46:07 UTC, Walter Bright wrote: On 1/19/2016 12:35 PM, David Nadlinger wrote: You'd run into all kinds of weird issues, as Manu pointed out – many of which have hopefully been fixed by Walter in the meantime. Please read the thread. I fixed them and posted here that they were fixed. Please do not conflate implementation bugs with the design. Before making such statements, it might be prudent to somewhat thoroughly read the message you are referring to. The sentence immediately before the one you quoted reads: "At least when this thread was started, it was not at all possible to just blissfully ignore them." I am not conflating implementation and design – in fact I was precisely pointing out to to Chris that part of the troubles Manu was having is due to implementation issues! — David
Re: extern(C++, ns)
On Tuesday, 19 January 2016 at 21:40:01 UTC, Walter Bright wrote: Please do not conflate not agreeing with my answers with not responding. I am not claiming that you weren't participating in the discussion threads or something like that. But in all your arguments you seem to presume that (badly) imitating the scoping semantics of C++ namespaces in D is somehow a given, that it is natural and does not need justification. However, my position is (Manu and Daniel would probably agree) that this is not a valid assumption. To add a new feature to the language, the onus is on you to provide compelling evidence why it is necessary to have. You need to show that there is a "serious problem" with just using the native D namespacing scheme (modules) for the purpose, not the other way round. — David
Re: Idea: limited template expansion
On Wednesday, 20 January 2016 at 22:00:45 UTC, Steven Schveighoffer wrote: Imagine if you such a call returned something, and you simply used it throughout the rest of your function. The compiler could rewrite this as a template and compile all three versions and branch to the appropriate one, but your code would simply read as a straightforward procedural function. But if you abstract away the dispatch part into a (meta)template function like I suggested, the caller would still be just as linear, right? How would the client code be simpler with a built-in language feature? — David
Re: Idea: limited template expansion
On Wednesday, 20 January 2016 at 21:27:15 UTC, Steven Schveighoffer wrote: It would be cool if the compiler could "expand" a finitely instantiable template (one with a finite number of ways it can be instantiated) based on a runtime value, and do the switch for me. Can't you just write a wrapper function that takes the template function as a compile-time argument and generates the switch for you by iterating over the std.traits.EnumMembers of said compile-time parameter? Something like `enumTemplateDispatch!foo(detectBOM(buffer), ))`. Depending on what template signatures you want to support, you might need to make enumTemplateDispatch explicitly take the enum type to use or the position of the respective parameter in the template argument list of the target. — David
Re: extern(C++, ns)
On Tuesday, 19 January 2016 at 20:03:47 UTC, Chris Wright wrote: I'm kind of surprised that people are insisting on complaining that there's an additional namespace that's trivial to ignore […] That's not what's happening here. At least when this thread was started, it was not at all possible to just blissfully ignore them. You'd run into all kinds of weird issues, as Manu pointed out – many of which have hopefully been fixed by Walter in the meantime. Apart from this, the argument still stands that the whole concept of having named scopes for C++ namespaces increases the size of the language for no good reason (yes, I'm aware that this increase might be trivial in terms of internal implementation, but it's not from a user's perspective). — David
Re: extern(C++, ns)
On Tuesday, 19 January 2016 at 13:56:54 UTC, Manu wrote: On 19 January 2016 at 18:54, Walter Bright via Digitalmars-d As the length of this thread testifies, this has been discussed at length already. No it hasn't. I don't feel like it's been discussed at all. I've explicitly asked both you and Andrei to justify the design several times, no less than 5, and you haven't responded a single time other than repeating these same points without providing any support or evidence. Andrei deliberately dodged the request, replying with something like "the best way to move forwards is to present code that demonstrates bugs". Neither of you appear to be willing to engage in discussion relating to the design, or how it's flawed and pointless. It looks like you understand there's no objective justification for the design. I suspect you just don't want to change it now it's in. While I am not in the mood for mudslinging or making a heated discussion out of this, I have to agree with Daniel and Manu here. If I remember correctly, you never really provided any justification (including during the original discussion back when the feature was introduced) as to why just using the normal means of name resolution and disambiguation in D – the module system – is not good enough for this. It's fine if you just say "I had a hunch that the added complexity would be worth it by making some situations more convenient, even though I can't provide a concrete example". Of course that might not be particularly persuasive, but it's your call in the end. But if you keep dodging the question, this discussion will never come to an end. — David
Re: Wait-free thread communication
On Saturday, 9 January 2016 at 22:44:53 UTC, Ola Fosheim Grøstad wrote: Yes. But my experience from writing custom multi-single queues is that it can end up harder than it looks to get it working and efficient. […] (Intuition is often wrong in this area...) I wholeheartedly agree with that statement. However, if one doesn't understand atomics well enough to correctly implement even a simple single-single queue, I'm not sure whether one would be able to correctly port an existing implementation either. Then again, the primitives might be similar enough between C++11 and D so that a literal translation is possible without understanding what is going on. — David
Re: Wait-free thread communication
On Saturday, 9 January 2016 at 17:44:26 UTC, Andy Smith wrote: Unfortunately I'm not well versed enough on the internals of the three main compiler versions to give an opinion on whether this will be a problem or not :-( It might work on DMD, but I know from experience (one-to-many queues, i.e. single-consumer-multi-producer or the other way round) that the LDC optimizer will ruthlessly break code written in that "I know that it would work in assembly, let's just hope the compiler doesn't touch it" style. — David
Re: Wait-free thread communication
On Saturday, 9 January 2016 at 15:16:43 UTC, Ola Fosheim Grøstad wrote: On Saturday, 9 January 2016 at 14:20:18 UTC, Andy Smith wrote: I'm a little worried you have no volatile writes or fences around your code when you 'publish' an event using head/tail etc. It looks like it's working but how are you ensuring no compiler/CPU reordering is ocurring. Does x86_64 actually allow you to get away with this? I know its memory model is stricter than others... But not on ARM, so he should use atomic acquire-release semantics on indices for push/pull. Not only that. It's a problem on x86 as well because advanced optimizers like those in GDC or LDC will happily assume that the members are not written to by another thread (because there would be a race otherwise) and cache the loads or even eliminate some stores. Any guarantees the processor would offer are irrelevant if the compiler has already reordered your operations. It should be quite easy to see such effects. Just compile a simple test case on GDC or LDC with optimizations on. I suggest porting over spsc_queue from Boost. That would certainly be a starting point, although a high-performance single-producer single-consumer queue is trivial to implement once you understand atomics. Then again, I couldn't convince Andrei that a well-defined memory model (in which it even makes sense to talk about atomics in the first place) is important for D yet. Right now, you basically have to hope that everything works as in C++ – which is not a bad bet on GDC and LDC, and the weaker optimizer in DMD hides a lot of potential issues there – and stay away from things like `consume` loads that would depend on language semantics. — David
Re: Wait-free thread communication
On Saturday, 9 January 2016 at 16:22:12 UTC, Jin wrote: So i simple use atomicFence. Performance does not degrade. Either you are not calling it in the way you think you are, then, or your code is otherwise very unoptimized. You can definitely measure the impact of a full mfence on otherwise tight lock-free code. — David
Re: GC buckets in 2.067
On Tuesday, 1 December 2015 at 08:47:03 UTC, Iain Buclaw wrote: David, did you get anything like this when moving to 2.067? It has been almost a year now, but I don't remember anything like that – are you sure that your GC root ranges (.data/stacks/TLS) are set correctly? — David
Re: I hate new DUB config format
On Wednesday, 25 November 2015 at 20:24:36 UTC, Andrei Alexandrescu wrote: If it offers no leverage, entails boring work, requires a manual, or doesn't look good on the page - it's probably wrong. Interestingly, this applies to the JSON-based DUB configuration format much more so than to the SDL-based one. — David
Re: I hate new DUB config format
On Wednesday, 25 November 2015 at 19:04:19 UTC, Walter Bright wrote: Sure it does: "comment" : "This is a comment", That only works in contexts where a dictionary is used, and also neither stands out visually by itself nor can be rendered differently by syntax highlighting. — David
Re: Dwarf Exception Handling question
On Monday, 23 November 2015 at 21:05:29 UTC, Walter Bright wrote:
The code looks quite good. I've been trying to adjust things,
however, so there are no pushes and pops in the code, trying to
preallocate everything needed in the function prolog.
Wouldn't you still need to restore the stack before leaving the
function (tail call in this case)?
For a single register, push/pop is probably still cheaper than
setting up RBP and having the extra mov/sub.
dmd rewrites try-catch-finally into try-{try-catch}-finally,
which makes it easier to generate code, because fewer special
cases and fewer bugs. I've become a big fan of that technique
Except that we actually need to flatten all the nesting into a
single landing pad anyway. How would you do this in DMD? I didn't
realize you could even have multiple EH table entries attached to
a single code location.
— David
Re: Dwarf Exception Handling question
On Monday, 23 November 2015 at 17:31:51 UTC, Walter Bright wrote: The calls to _d_dynamic cast appear to be redundant - shouldn't the value in RDX be sufficient? This is indeed the case, but of course you need to know in which order the compiler backend wrote the type info entries to the tables. Maybe at the time there didn't exist an intrinsic for that in GDC or something like that. This is how the function looks in LDC (on OS X, but the libunwind "client"-side code is very similar): --- 00010a40 <__D5test24foo1FZv>: 10a40: 53 push rbx 10a41: e8 ca fe ff ff call 10910 <__D4test3abcFZv> 10a46: e8 d5 fe ff ff call 10920 <__D4test3defFZv> 10a4b: 5b poprbx 10a4c: e9 ef fe ff ff jmp10940 <__D4test3jklFZv> 10a51: 48 89 c3movrbx,rax 10a54: 83 fa 03cmpedx,0x3 10a57: 74 05 je 10a5e <__D5test24foo1FZv+0x1e> 10a59: 83 fa 02cmpedx,0x2 10a5c: 75 0f jne10a6d <__D5test24foo1FZv+0x2d> 10a5e: e8 5d ff 00 00 call 1000109c0 <__d_eh_enter_catch> 10a63: 48 8b 3bmovrdi,QWORD PTR [rbx] 10a66: e8 c5 fe ff ff call 10930 <__D4test3ghiFC4test2CCZv> 10a6b: eb de jmp10a4b <__D5test24foo1FZv+0xb> 10a6d: 83 fa 01cmpedx,0x1 10a70: 75 10 jne10a82 <__D5test24foo1FZv+0x42> 10a72: e8 49 ff 00 00 call 1000109c0 <__d_eh_enter_catch> 10a77: e8 d4 fe ff ff call 10950 <__D4test3mnoFZv> 10a7c: 5b poprbx 10a7d: e9 be fe ff ff jmp10940 <__D4test3jklFZv> 10a82: 48 89 dfmovrdi,rbx 10a85: e8 16 ff 00 00 call 1000109a0 <__d_eh_resume_unwind> --- And why is there a 'default' call to _Unwind_Resume if RDX isn't an expected value? Shouldn't the personality routine simply not jump to the landing pad if none of the catch types are satisfied? The reason LDC emits the __d_eh_resume_unwind call all the time is because it is needed as soon as there is a cleanup involved anyway, and so far I was just too lazy to optimize the extra branch away in the special case that there are no cleanups. — David
Re: Catching C++ std::exception in D
On Friday, 13 November 2015 at 18:40:06 UTC, Iain Buclaw wrote: There may be a few other holes between how Fibers and EH interact. https://github.com/D-Programming-Language/druntime/commit/f6633abb43ea1f2464d3a772b8f8fe78216ffd8e The SJLJ stack switching should probably be added to this mechanism. — David
Re: Signed integer overflow undefined behavior or not?
On Friday, 13 November 2015 at 06:00:08 UTC, Walter Bright wrote: It's worth checking how LDC and GDC deal with this deep in their optimizer - is it considering it undefined behavior? Signed types will wrap around correctly for LDC. — David
Re: Our template emission strategy is broken
On Thursday, 12 November 2015 at 02:13:04 UTC, Martin Nowak wrote: There shouldn't be anything quadratic left for template instantiation in the frontend. Maybe not for a single instantiation, but constructing the module member list is overall still quadratic in the number of members (which might be quite a lot if you heavily use templates, because all the instances get added first, whether they make it to codegen or not). See: https://issues.dlang.org/show_bug.cgi?id=15323. — David
Re: DMD is faster than LDC and GDC
On Thursday, 12 November 2015 at 21:16:25 UTC, Walter Bright wrote: It's more than that - dmd's optimizer is designed to make use of the guarantees of a pure function. Since C/C++ do not have pure functions, ldc/gdc's optimizer may not have that capability. Oh, GCC has had similar notions as a non-standard attribute for ages, and LLVM since its inception. At least for LDC, the reason why we do not currently lower many of the qualifiers like pure, nothrow, immutable, etc. is that LLVM will ruthlessly consider your code to exhibit undefined behavior if you try to be clever and violate them, subsequently optimizing based on that. In other words, if you cast away const/immutable and modify a variable, for instance, you might find that the entire function body magically disappears under your feet. Maybe it is time to revisit this, though, but last time I tried it broke druntime/Phobos in a couple of places. — David
Re: Catching C++ std::exception in D
On Thursday, 12 November 2015 at 06:50:31 UTC, Walter Bright wrote: The tricky part with the personality function will likely be recognizing std::exception* exceptions. I wonder if forwarding the call to __gxx_personality_v0 will work. This should be rather simple; you know it is C++ because of the metadata, and if you have access to RTTI (can just be hacked together with pragma(mangle, …)), you just do the type comparison then. What has me quite a bit more worried is the lifetime handling involved in the usual "throw std::exception subclass by value, catch base class by reference" idiom. If I remember correctly, both GCC and Clang handle this by injecting a runtime function call into the end of the catch blocks. Their implementation must also support keeping the value around for rethrowing. At this point, I'm not aware of any fundamental incompatibilities, but it will certainly add to the complexity of the solution. — David
Re: Catching C++ std::exception in D
On Thursday, 12 November 2015 at 16:55:09 UTC, Johannes Pfau wrote: To expand on this: I think we'd prefer one __d_personality_v0 which is implemented in upstream druntime and identical for all compilers. Making the compilers ABI compatible is probably not a high priority, but OTOH we shouldn't introduce new ABI incompatibilities if possible. It should probably be based on the LDC implementation then, as it supports the most target platforms (Posix/libunwind, Win64/MSVC, iOS/SJLJ, and, as work in progress, Win32/SEH) and GDC's implementation is probably GPL-licensed (with runtime exceptions, but still definitely not Boost-compatible). Think of it this way: At some point it would be nice to have one, system-wide installed druntime shared library (from any compiler) and programs compiled with any other compiler should be able to use it. We've got well-defined name mangling primarily to allow this - but because of many small differences mixing compilers is not possible yet. The biggest of which is probably that GDC and DMD disagree on parameter passing orders. — David
Re: Our template emission strategy is broken
On Wednesday, 11 November 2015 at 15:04:01 UTC, Andrei Alexandrescu wrote: Liran Zvibel […] mentioned the package-at-a-time solved essentially all of their build problems. They were stuck at 2.066 until very recently, though, because of issues very similar to the one discussed here. Right now, they are using LDC 2.068.2 with a couple of local workarounds for e.g. the issue from the original post that are not really general enough to go into upstream. The idea would be to return to an unpatched frontend as soon as possible, though. — David
