Evaluation order of index expressions
import core.stdc.stdio; static int[] _array = [ 0, 1, 2, 3 ]; int[] array() @property { printf("array()\n"); return _array; } int start() @property { printf("start()\n"); return 0; } int end() @property { printf("end()\n"); return 1; } void main() { array[start..end] = 666; printf("---\n"); array[start] = end; } array() start() end() --- start() array() end() So for the 2nd assignment's left-hand-side, the index is evaluated before evaluating the container! Please don't tell me that's by design. :> [origin: https://github.com/D-Programming-Language/phobos/pull/3311]
Re: Evaluation order of index expressions
On Sunday, 24 May 2015 at 19:30:54 UTC, kinke wrote: import core.stdc.stdio; static int[] _array = [ 0, 1, 2, 3 ]; int[] array() @property { printf("array()\n"); return _array; } int start() @property { printf("start()\n"); return 0; } int end() @property { printf("end()\n"); return 1; } void main() { array[start..end] = 666; printf("---\n"); array[start] = end; } array() start() end() --- start() array() end() So for the 2nd assignment's left-hand-side, the index is evaluated before evaluating the container! Please don't tell me that's by design. :> [origin: https://github.com/D-Programming-Language/phobos/pull/3311] Why would you expect the order to even be defined? There's no operator precedence involved, so the compiler is free to order the evaluations however it likes. And code like was originally in the PR is just plain error-prone. It's like doing foo(++i, ++i); only worse, because the fact that array is a property and count is used inside it is not immediately obvious when looking at array[count++] = 666; The original code is clearly wrong. And forcing the order of evaluation so that it's one way or the other just changes under which cases you end up with bugs. Mutating in an expression while using it multiple times in that expression or mutating a variable in an expression while using a variable that depends on it is just plain error-prone and is a serious code smell. I really don't see anything wrong with what the compiler is doing in this case. The problem is that the code was making bad assumptions. - Jonathan M Davis
Re: Evaluation order of index expressions
On Sunday, 24 May 2015 at 19:48:05 UTC, Jonathan M Davis wrote: The original code is clearly wrong. And forcing the order of evaluation so that it's one way or the other just changes under which cases you end up with bugs. Mutating in an expression while using it multiple times in that expression or mutating a variable in an expression while using a variable that depends on it is just plain error-prone and is a serious code smell. I really don't see anything wrong with what the compiler is doing in this case. The problem is that the code was making bad assumptions. We agree on the original code smell. I think the evaluation order should be well-defined by the language though, following the intuitive left-to-right order for cases like this. Left-hand-side before right-hand-side in assign statements, container before its index/index range, for the latter start before end etc. Then at least all compilers of that language exhibit the same behavior and we don't end up with cases like this, where LDC complains and DMD compiles. Even worse would be not-so-obvious side effects caused by differing evaluation orders of different compilers, with a fair potential for nasty bugs.
Re: Evaluation order of index expressions
On 05/24/2015 09:48 PM, Jonathan M Davis wrote: On Sunday, 24 May 2015 at 19:30:54 UTC, kinke wrote: import core.stdc.stdio; static int[] _array = [ 0, 1, 2, 3 ]; int[] array() @property { printf("array()\n"); return _array; } int start() @property { printf("start()\n"); return 0; } int end() @property { printf("end()\n"); return 1; } void main() { array[start..end] = 666; printf("---\n"); array[start] = end; } array() start() end() --- start() array() end() So for the 2nd assignment's left-hand-side, the index is evaluated before evaluating the container! Please don't tell me that's by design. :> [origin: https://github.com/D-Programming-Language/phobos/pull/3311] Why would you expect the order to even be defined? Because this is not C. BTW, the documentation contradicts itself on evaluation order: http://dlang.org/expression.html
Re: Evaluation order of index expressions
On Sunday, 24 May 2015 at 20:30:00 UTC, Timon Gehr wrote: On 05/24/2015 09:48 PM, Jonathan M Davis wrote: On Sunday, 24 May 2015 at 19:30:54 UTC, kinke wrote: import core.stdc.stdio; static int[] _array = [ 0, 1, 2, 3 ]; int[] array() @property { printf("array()\n"); return _array; } int start() @property { printf("start()\n"); return 0; } int end() @property { printf("end()\n"); return 1; } void main() { array[start..end] = 666; printf("---\n"); array[start] = end; } array() start() end() --- start() array() end() So for the 2nd assignment's left-hand-side, the index is evaluated before evaluating the container! Please don't tell me that's by design. :> [origin: https://github.com/D-Programming-Language/phobos/pull/3311] Why would you expect the order to even be defined? Because this is not C. BTW, the documentation contradicts itself on evaluation order: http://dlang.org/expression.html There have been discussions on defining the order of evaluation from left-to-right such that it may happen, but there have been issues raised with it as well (particularly from an optimization standpoint, though IIRC, it causes some havoc for the gdc folks as well given how the gcc backend works). Regardless, having an expression where you're mutating a variable and using either it or something that depends on it within the same expression is just plain bug-prone, and even if the compiler wants to makes all such cases a compilation error, it's trivial to move some of the changes into a function call and hide it such that the compiler can't catch it. So, the reality of the matter is that even if we get more restrictive about the order of evaluation in expressions, we can't actually prevent the programmer from shooting themselves in the foot due to issues with the order of evaluation. At most, we can reduce the problem, but that just pushes it from common, relatively easy to catch cases, to more complex ones, so on some level, it's simply providing a false sense of security. So, I don't know if it's better to define the order of evaluation as being left-to-right or not, but it is _not_ a silver bullet. - Jonathan M Davis
Re: Evaluation order of index expressions
On 05/24/2015 10:35 PM, Jonathan M Davis wrote: On Sunday, 24 May 2015 at 20:30:00 UTC, Timon Gehr wrote: On 05/24/2015 09:48 PM, Jonathan M Davis wrote: On Sunday, 24 May 2015 at 19:30:54 UTC, kinke wrote: import core.stdc.stdio; static int[] _array = [ 0, 1, 2, 3 ]; int[] array() @property { printf("array()\n"); return _array; } int start() @property { printf("start()\n"); return 0; } int end() @property { printf("end()\n"); return 1; } void main() { array[start..end] = 666; printf("---\n"); array[start] = end; } array() start() end() --- start() array() end() So for the 2nd assignment's left-hand-side, the index is evaluated before evaluating the container! Please don't tell me that's by design. :> [origin: https://github.com/D-Programming-Language/phobos/pull/3311] Why would you expect the order to even be defined? Because this is not C. BTW, the documentation contradicts itself on evaluation order: http://dlang.org/expression.html There have been discussions on defining the order of evaluation from left-to-right such that it may happen, but there have been issues raised with it as well (particularly from an optimization standpoint, though IIRC, it causes some havoc for the gdc folks as well given how the gcc backend works). ... Optimizations can reorder operations when the compiler is able to prove equivalence, and there actually are annotations to help. Yes, occasionally, reorderings that the compiler won't be able to do on its own might lead to non-trivial performance improvements. In such cases, do them manually. The gcc backend obviously supports ordered operations, because some operations are ordered today. Regardless, having an expression where you're mutating a variable and using either it or something that depends on it within the same expression is just plain bug-prone, So what? If such a bug occurs, you want to be the one who sees it, not the guy who tries to use your code with a different compiler. and even if the compiler wants to makes all such cases a compilation error, it's trivial to move some of the changes into a function call and hide it such that the compiler can't catch it. 'pure','const',... Even then, the compiler should not want to make all such cases a compilation error; it is not necessary if evaluation order is defined. So, the reality of the matter is that even if we get more restrictive about the order of evaluation in expressions, we can't actually prevent the programmer from shooting themselves in the foot due to issues with the order of evaluation. We can at least go as far as to not artificially add additional foot-shooting failure modes to the weapon. At most, we can reduce the problem, but that just pushes it from common, relatively easy to catch cases, to more complex ones, so on some level, it's simply providing a false sense of security. ... No. Seriously. Under the current semantics, running an exhaustive input-output test on a fully @safe program will not ensure that the code is actually correct. Talk about providing a false sense of security.
Re: Evaluation order of index expressions
On Sunday, 24 May 2015 at 21:18:54 UTC, Timon Gehr wrote: The gcc backend obviously supports ordered operations, because some operations are ordered today. Iain has talked in the past about how they're forced to work around the backend to force the order of operations for those cases, and it's definitely ugly. No. Seriously. Under the current semantics, running an exhaustive input-output test on a fully @safe program will not ensure that the code is actually correct. Talk about providing a false sense of security. @safe definitely has issues. We went about it the wrong way by effectively implementing it via a blacklist instead of a whitelist. And it needs to be fixed. But as far as the code actually being correct goes, @safe isn't guaranteed to prove that anyway. All it's supposed to guarantee is that you can't corrupt memory. There's really no way to have the compiler guarantee that a program is correct. - Jonathan M Davis
Re: Evaluation order of index expressions
On 05/24/2015 11:26 PM, Jonathan M Davis wrote: On Sunday, 24 May 2015 at 21:18:54 UTC, Timon Gehr wrote: The gcc backend obviously supports ordered operations, because some operations are ordered today. Iain has talked in the past about how they're forced to work around the backend to force the order of operations for those cases, and it's definitely ugly. ... Given that it is/should be already there for OrExpression, XorExpression, AndExpression, CmpExpression, ShiftExpression, AddExpression, CatExpression, MulExpression, PowExpression, OrOrExpression, AndAndExpression, it would seem that doing the few remaining cases left-to-right shouldn't be that much of an obstacle, no? No. Seriously. Under the current semantics, running an exhaustive input-output test on a fully @safe program will not ensure that the code is actually correct. Talk about providing a false sense of security. @safe definitely has issues. We went about it the wrong way by effectively implementing it via a blacklist instead of a whitelist. And it needs to be fixed. But as far as the code actually being correct goes, @safe isn't guaranteed to prove that anyway. All it's supposed to guarantee is that you can't corrupt memory. You missed the more relevant exhaustive input-output test part. @safe was there simply to ensure there is no UB.
Re: Evaluation order of index expressions
On 24 May 2015 23:30, "Jonathan M Davis via Digitalmars-d" < digitalmars-d@puremagic.com> wrote: > > On Sunday, 24 May 2015 at 21:18:54 UTC, Timon Gehr wrote: >> >> The gcc backend obviously supports ordered operations, because some operations are ordered today. > > > Iain has talked in the past about how they're forced to work around the backend to force the order of operations for those cases, and it's definitely ugly. > There was a point in time when I started out that I took 'gdc does X, dmd does Y' as serious bugs that need fixing. I was so naïve back then. ;-)
Re: Evaluation order of index expressions
On 5/24/15 1:29 PM, Timon Gehr wrote: BTW, the documentation contradicts itself on evaluation order: http://dlang.org/expression.html This comes up once in a while. We should stick with left to right through and through. It's a "simple" matter of getting somebody on the compiler team to find the time for it. -- Andrei
Re: Evaluation order of index expressions
On 25 May 2015 00:20, "Andrei Alexandrescu via Digitalmars-d" < digitalmars-d@puremagic.com> wrote: > > On 5/24/15 1:29 PM, Timon Gehr wrote: >> >> BTW, the documentation contradicts itself on evaluation order: >> http://dlang.org/expression.html > > > This comes up once in a while. We should stick with left to right through and through. It's a "simple" matter of getting somebody on the compiler team to find the time for it. -- Andrei > I find it is not as clear cut as that. In gdc, there is a compiler flag that tells the optimizer to honour left to right evaluation, and because of both what you say and the agreement of others, it seems natural to have this turned on by default. However, this has an interesting side effect with operations with side effects. Ie: foo += bar() could either produce expected or surprising results. Hint, the LTR order - foo = foo + bar() - gives the most surprise in my experience from users.
Re: Evaluation order of index expressions
On 05/25/2015 12:15 AM, Andrei Alexandrescu wrote: On 5/24/15 1:29 PM, Timon Gehr wrote: BTW, the documentation contradicts itself on evaluation order: http://dlang.org/expression.html This comes up once in a while. We should stick with left to right through and through. It's a "simple" matter of getting somebody on the compiler team to find the time for it. -- Andrei https://github.com/D-Programming-Language/dlang.org/pull/999
Re: Evaluation order of index expressions
On 05/25/2015 12:36 AM, Iain Buclaw via Digitalmars-d wrote: > This comes up once in a while. We should stick with left to right through and through. It's a "simple" matter of getting somebody on the compiler team to find the time for it. -- Andrei > I find it is not as clear cut as that. In gdc, there is a compiler flag that tells the optimizer to honour left to right evaluation, and because of both what you say and the agreement of others, it seems natural to have this turned on by default. However, this has an interesting side effect with operations with side effects. Ie: foo += bar() could either produce expected or surprising results. Hint, the LTR order - foo = foo + bar() - gives the most surprise in my experience from users. I think I still don't get it. What is the surprise? That bar() is evaluated before the value is written to foo?
Re: Evaluation order of index expressions
On 5/24/15 3:36 PM, Iain Buclaw via Digitalmars-d wrote: On 25 May 2015 00:20, "Andrei Alexandrescu via Digitalmars-d" mailto:digitalmars-d@puremagic.com>> wrote: > > On 5/24/15 1:29 PM, Timon Gehr wrote: >> >> BTW, the documentation contradicts itself on evaluation order: >> http://dlang.org/expression.html > > > This comes up once in a while. We should stick with left to right through and through. It's a "simple" matter of getting somebody on the compiler team to find the time for it. -- Andrei > I find it is not as clear cut as that. In gdc, there is a compiler flag that tells the optimizer to honour left to right evaluation, and because of both what you say and the agreement of others, it seems natural to have this turned on by default. Even better - the front end could force the sequencing. However, this has an interesting side effect with operations with side effects. Ie: foo += bar() could either produce expected or surprising results. Hint, the LTR order - foo = foo + bar() - gives the most surprise in my experience from users. I think LTR is the most sensible in all cases. -- Andrei
Re: Evaluation order of index expressions
On 05/25/2015 01:49 AM, Andrei Alexandrescu wrote: I think LTR is the most sensible in all cases. -- Andrei It is also what Java and C# do.
Re: Evaluation order of index expressions
On 25 May 2015 01:10, "Timon Gehr via Digitalmars-d" < digitalmars-d@puremagic.com> wrote: > > On 05/25/2015 12:36 AM, Iain Buclaw via Digitalmars-d wrote: >> >> >> > This comes up once in a while. We should stick with left to right >> through and through. It's a "simple" matter of getting somebody on the >> compiler team to find the time for it. -- Andrei >> > >> >> I find it is not as clear cut as that. In gdc, there is a compiler flag >> that tells the optimizer to honour left to right evaluation, and because >> of both what you say and the agreement of others, it seems natural to >> have this turned on by default. >> >> However, this has an interesting side effect with operations with side >> effects. Ie: foo += bar() could either produce expected or surprising >> results. >> >> Hint, the LTR order - foo = foo + bar() - gives the most surprise in my >> experience from users. >> > > I think I still don't get it. What is the surprise? That bar() is evaluated before the value is written to foo? That foo is cached before bar() is evaluated. The context here involves concurrency where bar() calls yield and makes changes to foo before returning to assign the updated results.
Re: Evaluation order of index expressions
On Sun, 24 May 2015 19:30:52 +, kinke wrote: > So for the 2nd assignment's left-hand-side, the index is evaluated > before evaluating the container! Please don't tell me that's by design. > :> it is. at least this is what i was told when i faced the similar issue. "WONTIFX, STFU". signature.asc Description: PGP signature
Re: Evaluation order of index expressions
On Monday, 25 May 2015 at 07:33:49 UTC, ketmar wrote: On Sun, 24 May 2015 19:30:52 +, kinke wrote: So for the 2nd assignment's left-hand-side, the index is evaluated before evaluating the container! Please don't tell me that's by design. :> it is. at least this is what i was told when i faced the similar issue. "WONTIFX, STFU". To be fair, the example that the OP gave is almost the same thing as foo(++i, ++i); whereas what you came up with had a lot more layers to it, with whole chains of function calls affecting each other. With the kind of example you came up with, even with defining the evaluation as strictly left-to-right, you would _still_ run into screwy problems with stuff easily being mutated in a different order than you expected. Defining the order of evaluation as being strictly left-to-right will avoid some of the common bugs cause by folks foolishly doing something like foo(++i, ++i); but the reality of the matter is, if you start doing stuff like mutating the arguments inside of the function inside of the function when the same arguments are being passed to other functions in the same expression, you _will_ have weird and unexpected stuff happening. It might be completely well-defined and consistent, but it may not be what you expect, and even if it is, a slight change to the code could change the order. So, the kind of stuff that you're complaining about not being able to do really shouldn't be done regardless of how well-defined the order of evaluation is. It's just begging for trouble. - Jonathan M Davis
Re: Evaluation order of index expressions
On Monday, 25 May 2015 at 08:00:15 UTC, Jonathan M Davis wrote: it is, a slight change to the code could change the order. So, the kind of stuff that you're complaining about not being able to do really shouldn't be done regardless of how well-defined the order of evaluation is. It's just begging for trouble. The the compiler should complain about it...
Re: Evaluation order of index expressions
On Monday, 25 May 2015 at 08:00:15 UTC, Jonathan M Davis wrote: It might be completely well-defined and consistent, but it may not be what you expect, and even if it is, a slight change to the code could change the order. If the behavior isn't what I expect (and I don't think that's often case for left-to-right order), then the language should force me to express the intention differently. If it's not well-defined, I may not be aware of such issues until I use a different compiler. Allowing implementation-dependent evaluation order is just begging for additional bugs and portability issues. Another example: b = 0; ((++b *= 5) *= 2) += ++b * (b -= 6); DMD yields b=60, LDC the intuitively correct b=65. If it's well defined, one may argue about the form of such a statement, but it's not silly code with different results depending on the used D compiler anymore. +1 for Timon's PR being merged asap.
Re: Evaluation order of index expressions
On Monday, 25 May 2015 at 12:38:29 UTC, kinke wrote: On Monday, 25 May 2015 at 08:00:15 UTC, Jonathan M Davis wrote: It might be completely well-defined and consistent, but it may not be what you expect, and even if it is, a slight change to the code could change the order. If the behavior isn't what I expect (and I don't think that's often case for left-to-right order), then the language should force me to express the intention differently. If it's not well-defined, I may not be aware of such issues until I use a different compiler. It seems like you still don't understand. Yes, defining the order of evaluation within an expression as being left-to-right makes it easier to deal with what is directly inside the expression, and the compiler could be make to do that (and from the sounds of it likely will). It could also be made to not be fixed about the order of evaluation but give an error when it detects that the order of evaluation matters, so that expressions like foo(++i, ++i); give an error. But even so, the compiler _cannot_ be made to catch all such problems for you, because all it takes is starting to bury the problem within other function calls within the expression, and while the order of evaluation in such cases may very well be defined, whether it's going to do what you expect is a completely different matter. For instance, what if you had int bar() { return ++i; } foo(bar(), bar()); Now, because ++i is inside a call to another function, the compiler can no longer see that the arguments that you're using depend on one another. The results _are_ well-defined, but whether it's what you expect is another matter. With one extra layer like this, you'll probably see it, and it'll be doing what you want, but what if you have an expression like auto f = foo(bar() + baz(bop(), beep() + boop())); and 4 levels down into the call stack bar() and beep() both mutate a static or global variable - or some other shared resource. Then the result of this expression ends up depending on an order of evaluation issue that you can't see without really digging into the code, and the compiler sure isn't going to see for you. You might see that swapping the arguments around in the expression results in a different result when you think that it shouldn't, but just as likely, you wouldn't catch that, and a small change to the code later could change the results unexpectedly, which you might or might not notice. Now, that sort of thing is all the more reason to avoid using static or global variables, and it's the sort of thing that I would hope good code would avoid. But defining the order of evaluation as left-to-right, doesn't make those problems go away. At best, it makes them consistent, and that may be worth it, but it's not a silver bullet. And it takes optimization opportunities away from the compiler, since in many cases, it can reorder how the expression is evaluated to make better use of the registers and whatnot. So, forcing the order of evaluation is not without its cons, even if it did solve all order of evaluation issues, and it really doesn't - especially when that often depends on what you expect. Ketmar had a screwy example with arrays a while back that he was complaining bitterly about not working due to order of evaluation issues, but IIRC he had recursive function calls which affected each other and was having all kinds of problems just because he insisted on doing multiple things in an expression rather than splitting them out. And the results he was getting were completely consistent; they just weren't what he wanted. The order of evaluation mattered too much in the expressions that he was writing. Ultimately, if you want to reduce the risk of bugs, you really should be writing expressions where the order of evaluation doesn't matter, or where it only matters based on operator precedence directly within the expression so that it really doesn't matter what other code is doing. And forcing left-to-right evaluation doesn't change that. All it really does is make what what's happening consistent. It doesn't mean that relying on it is a good idea or that it's going to fix anything but the some of the most basic order of evaluation issues. Personally, I don't think that it's worth the cost in lost optimizations, but even if it is, my point here is really that at best it only fixes part of the problem. - Jonathan M Davis
Re: Evaluation order of index expressions
On 5/25/15 1:00 AM, Jonathan M Davis wrote: foo(++i, ++i); More complete example: table[++i] = objTable[++i].funcTable[++i](++i, ++i); should be well defined and evaluate left to right. Andrei
Re: Evaluation order of index expressions
On 5/24/15 11:13 PM, Iain Buclaw via Digitalmars-d wrote: The context here involves concurrency where bar() calls yield and makes changes to foo before returning to assign the updated results. We're not addressing that. += is not supposed to do concurrency magic. -- Andrei
Re: Evaluation order of index expressions
Am Mon, 25 May 2015 09:40:34 -0700 schrieb Andrei Alexandrescu : > On 5/24/15 11:13 PM, Iain Buclaw via Digitalmars-d wrote: > > The context here involves concurrency where bar() calls yield and > > makes changes to foo before returning to assign the updated results. > > We're not addressing that. += is not supposed to do concurrency > magic. -- Andrei It's not += doing the magic, it's bar(). And it's not limited to concurrency, it happens with every side effect: import std.stdio; void main() { int a = 0; int bar() { a++; return a; } a += bar(); // => a = a + bar() writeln(a); } DMD: 2 GDC: 1 which one is correct?
Re: Evaluation order of index expressions
On 5/25/15 10:21 AM, Johannes Pfau wrote: Am Mon, 25 May 2015 09:40:34 -0700 schrieb Andrei Alexandrescu : On 5/24/15 11:13 PM, Iain Buclaw via Digitalmars-d wrote: The context here involves concurrency where bar() calls yield and makes changes to foo before returning to assign the updated results. We're not addressing that. += is not supposed to do concurrency magic. -- Andrei It's not += doing the magic, it's bar(). And it's not limited to concurrency, it happens with every side effect: import std.stdio; void main() { int a = 0; int bar() { a++; return a; } a += bar(); // => a = a + bar() writeln(a); } DMD: 2 GDC: 1 which one is correct? GDC. -- Andrei
Re: Evaluation order of index expressions
On 05/25/2015 07:21 PM, Johannes Pfau wrote: Am Mon, 25 May 2015 09:40:34 -0700 schrieb Andrei Alexandrescu : On 5/24/15 11:13 PM, Iain Buclaw via Digitalmars-d wrote: The context here involves concurrency where bar() calls yield and makes changes to foo before returning to assign the updated results. We're not addressing that. += is not supposed to do concurrency magic. -- Andrei It's not += doing the magic, it's bar(). And it's not limited to concurrency, it happens with every side effect: import std.stdio; void main() { int a = 0; int bar() { a++; return a; } a += bar(); // => a = a + bar() writeln(a); } DMD: 2 GDC: 1 which one is correct? With left-to-right evaluation, 1 is correct. Java and C# also give 1.
Re: Evaluation order of index expressions
"Andrei Alexandrescu" wrote in message news:mjvlv5$vch$1...@digitalmars.com... > which one is correct? GDC. -- Andrei I don't think it matters too much if we pick strict LTR, or keep dmd's existing exception for assign expressions. IIRC Walter is in favour of keeping the exception[1]. Could you and Walter please come to an agreement and confirm here? It should be fairly straightforward to get this fixed once it's clear which way it should go. [1] https://github.com/D-Programming-Language/dmd/pull/4035#issuecomment-58861231
Re: Evaluation order of index expressions
On Monday, 25 May 2015 at 17:21:05 UTC, Johannes Pfau wrote: import std.stdio; void main() { int a = 0; int bar() { a++; return a; } a += bar(); // => a = a + bar() writeln(a); } DMD: 2 GDC: 1 which one is correct? So what about my previous example? int b = 0; ((++b *= 5) *= 2) += ++b * (b -= 6); DMD 2.067.1: 60, latest LDC: 65, GDC: ? This divergence probably doesn't have anything to do with the evaluation order, which seems to be identical (LTR), but rather how the lhs expression is treated (a double-nature as nested lvalue to be assigned to and rvalue result of a binAssign expression). For more context, see https://github.com/ldc-developers/ldc/pull/873.
Re: Evaluation order of index expressions
On 25 May 2015 21:00, "Daniel Murphy via Digitalmars-d" < digitalmars-d@puremagic.com> wrote: > > "Andrei Alexandrescu" wrote in message news:mjvlv5$vch$1...@digitalmars.com. .. > > >> > which one is correct? >> >> GDC. -- Andrei > > > I don't think it matters too much if we pick strict LTR, or keep dmd's existing exception for assign expressions. IIRC Walter is in favour of keeping the exception[1]. > > Could you and Walter please come to an agreement and confirm here? It should be fairly straightforward to get this fixed once it's clear which way it should go. > > [1] https://github.com/D-Programming-Language/dmd/pull/4035#issuecomment-58861231 Yeah, but his reasoning only applies to x86. This makes it void in my books.
Re: Evaluation order of index expressions
On 25 May 2015 at 21:02, kinke via Digitalmars-d < digitalmars-d@puremagic.com> wrote: > On Monday, 25 May 2015 at 17:21:05 UTC, Johannes Pfau wrote: > >> import std.stdio; >> void main() >> { >> int a = 0; >> int bar() >> { >> a++; >> return a; >> } >> a += bar(); // => a = a + bar() >> writeln(a); >> } >> >> DMD: 2 >> GDC: 1 >> >> which one is correct? >> > > So what about my previous example? > > int b = 0; > ((++b *= 5) *= 2) += ++b * (b -= 6); > > DMD 2.067.1: 60, latest LDC: 65, GDC: ? > > If the litmus test is "What does GDC do?", then LDC is doing it the correct way. :-)
Re: Evaluation order of index expressions
On 05/25/2015 09:14 PM, Iain Buclaw via Digitalmars-d wrote: So what about my previous example? int b = 0; ((++b *= 5) *= 2) += ++b * (b -= 6); DMD 2.067.1: 60, latest LDC: 65, GDC: ? If the litmus test is "What does GDC do?", then LDC is doing it the correct way. :-) Even if it isn't. ;)
Re: Evaluation order of index expressions
On 05/25/2015 09:14 PM, Iain Buclaw via Digitalmars-d wrote: If the litmus test is "What does GDC do?", then LDC is doing it the correct way. :-) Perfect. :) On Monday, 25 May 2015 at 19:17:48 UTC, Timon Gehr wrote: Even if it isn't. ;) It is - on its merge-2.067 branch. ;)
Re: Evaluation order of index expressions
On 05/25/2015 09:28 PM, kinke wrote: On 05/25/2015 09:14 PM, Iain Buclaw via Digitalmars-d wrote: If the litmus test is "What does GDC do?", then LDC is doing it the correct way. :-) Perfect. :) On Monday, 25 May 2015 at 19:17:48 UTC, Timon Gehr wrote: Even if it isn't. ;) It is - on its merge-2.067 branch. ;) LDC is doing it the correct way even if "What does GDC do?" is not the litmus test.
Re: Evaluation order of index expressions
On 25 May 2015 21:35, "Timon Gehr via Digitalmars-d" < digitalmars-d@puremagic.com> wrote: > > On 05/25/2015 09:28 PM, kinke wrote: >>> >>> On 05/25/2015 09:14 PM, Iain Buclaw via Digitalmars-d wrote: If the litmus test is "What does GDC do?", then LDC is doing it the correct way. :-) >> >> >> Perfect. :) >> >> On Monday, 25 May 2015 at 19:17:48 UTC, Timon Gehr wrote: >>> >>> Even if it isn't. ;) >> >> >> It is - on its merge-2.067 branch. ;) > > > LDC is doing it the correct way even if "What does GDC do?" is not the litmus test. I am not a fan of this dictatorship. I vote for democracy, if two compilers do 'X', then the odd one out is wrong. ;-)
Re: Evaluation order of index expressions
On 05/24/2015 09:30 PM, kinke wrote: import core.stdc.stdio; static int[] _array = [ 0, 1, 2, 3 ]; int[] array() @property { printf("array()\n"); return _array; } int start() @property { printf("start()\n"); return 0; } int end() @property { printf("end()\n"); return 1; } void main() { array[start..end] = 666; printf("---\n"); array[start] = end; } array() start() end() --- start() array() end() So for the 2nd assignment's left-hand-side, the index is evaluated before evaluating the container! Please don't tell me that's by design. :> [origin: https://github.com/D-Programming-Language/phobos/pull/3311] A related issue is that the rewrites documented at http://dlang.org/operatoroverloading.html don't all preserve the order of subexpressions. However, ideally, the order of evaluation would be preserved.
Re: Evaluation order of index expressions
"Timon Gehr" wrote in message news:mjvtqm$17d8$1...@digitalmars.com... A related issue is that the rewrites documented at http://dlang.org/operatoroverloading.html don't all preserve the order of subexpressions. However, ideally, the order of evaluation would be preserved. As operator overloading is defined in terms of lowering to function calls, I think it's reasonable to decide the order of evaluation after the lowering. This will still be consistent across compilers and platforms. Preserving the original order would require added complexity that I don't think is warranted.
Re: Evaluation order of index expressions
On 05/25/2015 10:02 PM, Daniel Murphy wrote: "Timon Gehr" wrote in message news:mjvtqm$17d8$1...@digitalmars.com... A related issue is that the rewrites documented at http://dlang.org/operatoroverloading.html don't all preserve the order of subexpressions. However, ideally, the order of evaluation would be preserved. As operator overloading is defined in terms of lowering to function calls, I think it's reasonable to decide the order of evaluation after the lowering. This will still be consistent across compilers and platforms. But almost entirely arbitrary. Preserving the original order would require added complexity that I don't think is warranted. The compiler would just need to introduce some temporary variables for the two lowerings. Why wouldn't this be warranted to make overloaded operators consistent with built-in ones? If anything, I think it is desirable to have opBinary(B) on type A and opBinaryRight(A) on type B interchangeable. What complexity are you worried about?
Re: Evaluation order of index expressions
"Timon Gehr" wrote in message news:mjvvq2$19hd$1...@digitalmars.com... > As operator overloading is defined in terms of lowering to function > calls, I think it's reasonable to decide the order of evaluation after > the lowering. This will still be consistent across compilers and > platforms. But almost entirely arbitrary. Yes. I don't think this is particularly important, as depending on evaluation order is highly discouraged. > Preserving the original order would require added complexity > that I don't think is warranted. The compiler would just need to introduce some temporary variables for the two lowerings. Why wouldn't this be warranted to make overloaded operators consistent with built-in ones? If anything, I think it is desirable to have opBinary(B) on type A and opBinaryRight(A) on type B interchangeable. What complexity are you worried about? Introducing temporary variables is added complexity. It affects all sorts of other parts of the compiler.
Re: Evaluation order of index expressions
On 05/25/2015 10:30 PM, Daniel Murphy wrote: "Timon Gehr" wrote in message news:mjvvq2$19hd$1...@digitalmars.com... > As operator overloading is defined in terms of lowering to function > calls, I think it's reasonable to decide the order of evaluation after > the lowering. This will still be consistent across compilers and > platforms. But almost entirely arbitrary. Yes. I don't think this is particularly important, Those small ugly corners of the language do add up, and they do cause real problems. For issues like this one, which are not considered important enough, I think it is fine to fix the spec and let the compiler catch up later (with a warning in the spec). I'm not saying this is urgent, just that it is obvious how it ought to be. as depending on evaluation order is highly discouraged. ... Doesn't mean it won't happen. Having different evaluation order for expressions that look identical is just asking for really funny problems in generic code, of the sort that will summon more threads like this one. > Preserving the original order would require added complexity > that I don't think is warranted. The compiler would just need to introduce some temporary variables for the two lowerings. Why wouldn't this be warranted to make overloaded operators consistent with built-in ones? If anything, I think it is desirable to have opBinary(B) on type A and opBinaryRight(A) on type B interchangeable. What complexity are you worried about? Introducing temporary variables is added complexity. It affects all sorts of other parts of the compiler. This ought to be a matter of changing a few lines in one place. Took me a couple of minutes to implement for opBinaryRight: -r=New!CallExp(opoverloadR,[e1]); -r.loc=loc; +auto tmpe=New!TmpVarExp(e1); +tmpe.loc=loc; +tmpe.semantic(sc); +version(assert) assert(!!tmpe.sym); +auto c=New!CallExp(opoverloadR,[tmpe.sym]); +r=New!(BinaryExp!(Tok!","))(tmpe,c); +r.loc=c.loc=loc; What makes this different for DMD?
Re: Evaluation order of index expressions
On 5/25/15 11:58 AM, Daniel Murphy wrote: "Andrei Alexandrescu" wrote in message news:mjvlv5$vch$1...@digitalmars.com... > which one is correct? GDC. -- Andrei I don't think it matters too much if we pick strict LTR, or keep dmd's existing exception for assign expressions. IIRC Walter is in favour of keeping the exception[1]. Could you and Walter please come to an agreement and confirm here? It should be fairly straightforward to get this fixed once it's clear which way it should go. [1] https://github.com/D-Programming-Language/dmd/pull/4035#issuecomment-58861231 I'm fine with RTL for assignment expressions, and LTR everywhere else. Daniel, if you could work this out at front end level so it goes the same way for all backends, that would be fantastic. -- Andrei
Re: Evaluation order of index expressions
On 05/26/2015 01:45 AM, Andrei Alexandrescu wrote: On 5/25/15 11:58 AM, Daniel Murphy wrote: "Andrei Alexandrescu" wrote in message news:mjvlv5$vch$1...@digitalmars.com... > which one is correct? GDC. -- Andrei I don't think it matters too much if we pick strict LTR, or keep dmd's existing exception for assign expressions. IIRC Walter is in favour of keeping the exception[1]. Could you and Walter please come to an agreement and confirm here? It should be fairly straightforward to get this fixed once it's clear which way it should go. [1] https://github.com/D-Programming-Language/dmd/pull/4035#issuecomment-58861231 I'm fine with RTL for assignment expressions, and LTR everywhere else. Daniel, if you could work this out at front end level so it goes the same way for all backends, that would be fantastic. -- Andrei Why? Strictly left-to-right is the simplest thing.
Re: Evaluation order of index expressions
On Monday, 25 May 2015 at 17:25:57 UTC, Andrei Alexandrescu wrote: It's not += doing the magic, it's bar(). And it's not limited to concurrency, it happens with every side effect: import std.stdio; void main() { int a = 0; int bar() { a++; return a; } a += bar(); // => a = a + bar() writeln(a); } DMD: 2 GDC: 1 which one is correct? GDC. -- Andrei You made me change SDC to return 2 recently using the following as an argument (one lwoering per line): a += foo(); ((ref X, Y) => X = X + Y)(a, foo()); http://33.media.tumblr.com/31bb0136f46468417bd3ccac1c52c769/tumblr_inline_nix5v8WXLd1t7oi6g.gif
Re: Evaluation order of index expressions
On Monday, 25 May 2015 at 23:44:57 UTC, Andrei Alexandrescu wrote: On 5/25/15 11:58 AM, Daniel Murphy wrote: "Andrei Alexandrescu" wrote in message news:mjvlv5$vch$1...@digitalmars.com... > which one is correct? GDC. -- Andrei I don't think it matters too much if we pick strict LTR, or keep dmd's existing exception for assign expressions. IIRC Walter is in favour of keeping the exception[1]. Could you and Walter please come to an agreement and confirm here? It should be fairly straightforward to get this fixed once it's clear which way it should go. [1] https://github.com/D-Programming-Language/dmd/pull/4035#issuecomment-58861231 I'm fine with RTL for assignment expressions, and LTR everywhere else. Daniel, if you could work this out at front end level so it goes the same way for all backends, that would be fantastic. -- Andrei It seems to me that a += b - c; should always be the same as a.opOpAssign!"+"(b - c); because otherwise it's just totally confusing.
Re: Evaluation order of index expressions
On Tuesday, 26 May 2015 at 00:07:33 UTC, Timon Gehr wrote: I'm fine with RTL for assignment expressions, and LTR everywhere else. Daniel, if you could work this out at front end level so it goes the same way for all backends, that would be fantastic. -- Andrei Why? Strictly left-to-right is the simplest thing. In case of opAssign kind of thing, LTR is not doable as operator overloading, at least not in a backward compatible manner.
Re: Evaluation order of index expressions
On Monday, 25 May 2015 at 15:35:02 UTC, Jonathan M Davis wrote: would hope good code would avoid. But defining the order of evaluation as left-to-right, doesn't make those problems go away. At best, it makes them consistent, and that may be worth it, but it's not a silver bullet. And it takes optimization opportunities away from the compiler, since in many cases, it can reorder how the expression is evaluated to make better use of the registers and whatnot. So, forcing the order of One of C's design mistakes is to make assignments expressions and not statements. Favouring terseness over correctness, an issue the C follow-up language Go partially recognized by turning "++" and "--" into statements. I agree with you that if an expression depends on evaluation order it is most likely either buggy or prone to become buggy when the program is modified later on. So it is reasonable to define it as illegal and leave it to a sanitizer. (Of course, the exception is shortcut operators like "&&" and "||", which already are have strict evaluation order). Being able to change evaluation order can provide optimization opportunities that cannot be fixed by having the compiler infer it due to aliasing issues. Not only due to registers, but also because of barriers, aliasing, lookup-tables, sub-expressions etc… The downside to not having a strict evaluation order is contexts where you want to use multiple generator calls in a single expression (like a numeric range or random number generator), so it goes against the whole "range iterators" approach which will lead to lengthy expressions that do contain side effects. So essentially D does not have any other reasonable option than strict LTR evaluation, since it is making "gigantic" expressions with generators in them a selling point. You have to support what you market as a major feature whether that is done by having dedicated "range-operators" that have strict evaluation order or by making all expressions strict..
Re: Evaluation order of index expressions
On 05/26/2015 02:55 AM, deadalnix wrote: On Tuesday, 26 May 2015 at 00:07:33 UTC, Timon Gehr wrote: I'm fine with RTL for assignment expressions, and LTR everywhere else. Daniel, if you could work this out at front end level so it goes the same way for all backends, that would be fantastic. -- Andrei Why? Strictly left-to-right is the simplest thing. In case of opAssign kind of thing, LTR is not doable as operator overloading, at least not in a backward compatible manner. Not caching the value of the left hand side is not the same thing as right-to-left evaluation: int a=0,b=0; (b++,a)=b; // ltr gives a==1, rtl gives a==0, caching irrelevant int a=0,b=0; ((ref a,b)=>a=b)((b++,a),b) // operator overloading lowering gives a==1 However, this is a more general problem with operator overloading: the first argument is always passed by reference, hence it is not cached: int[] foo(){ int a=1; int[] r; a=(a=a*2)+(a=a+2); // with l-t-r and caching: 6 r~=a; alias string=immutable(char)[]; static struct S{ int a; this(int a){ this.a=a; } S opBinary(string op)(S t){ return S(mixin("a "~op~" t.a")); } ref S opUnary(string op:"++")(){ ++a; return this; } } static struct T{ int a; this(int a){ this.a=a; } T opBinaryRight(string op)(T s){ return T(mixin("s.a "~op~" a")); } ref T opUnary(string op:"++")(){ ++a; return this; } } auto s=S(1); auto t=T(1); s=(s=s*S(2))+(s=s+S(2)); // with l-t-r and lowering: 8 t=(t=t*T(2))+(t=t+T(2)); // either 8 or 12, depending on whether evaluation order is preserved during lowering. r~=s.a,r~=t.a; return r; } I guess overloaded operators could be made to cache the old value. (As they do in CTFE, apparently. :o)) However, this seems like overkill. Any other ideas?
Re: Evaluation order of index expressions
On 05/26/2015 06:35 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= " wrote: One of C's design mistakes is to make assignments expressions and not statements. I think it is more about returning void vs. returning the lvalue. The expression/statement distinction is unnecessary.
Re: Evaluation order of index expressions
On 05/26/2015 02:51 PM, Timon Gehr wrote: int a=0,b=0; (b++,a)=b; // ltr gives a==1, rtl gives a==0, caching irrelevant This should have said that caching _on the lhs_ is irrelevant.
Re: Evaluation order of index expressions
On Tuesday, 26 May 2015 at 12:54:27 UTC, Timon Gehr wrote: On 05/26/2015 06:35 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= " wrote: One of C's design mistakes is to make assignments expressions and not statements. I think it is more about returning void vs. returning the lvalue. The expression/statement distinction is unnecessary. Not sure what you mean, the ideal for writing maintainable code is that expressions are either free of side effects or that side effects at least are independent and encapsulated in a robust manner. Everything is unnecessary beyond the bare minimum (e.g. a Turing Machine), but for a sensible imperative language the distinction between statements and expressions is necessary, due to control flow, which is why SSA needs the phi function: http://en.wikipedia.org/wiki/Dominator_(graph_theory) . Unless you are doing something completely different, like some weird non-deterministic language. That said, another C design-flaw is that you cannot prevent return values from being ignored, but I think that is another issue more related to resource management and ownership. In terms of describing intent, the distinction between functions, procedures and constructors have a lot of value. And actually, also visually distinguishing between ownership transfer, referencing of objects and value assignment…
Re: Evaluation order of index expressions
On 05/26/15 14:54, Timon Gehr via Digitalmars-d wrote: > On 05/26/2015 06:35 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= > " wrote: >> >> One of C's design mistakes is to make assignments expressions and not >> statements. > > I think it is more about returning void vs. returning the lvalue. The > expression/statement distinction is unnecessary. int a, b, c; void f(); f(a=b); void g(T...)(T) {} g(a=b); // and, even when 'void' is not a first class type: void h(int); h(((a=b), c)); artur
Re: Evaluation order of index expressions
On 05/26/2015 06:13 PM, Artur Skawina via Digitalmars-d wrote: On 05/26/15 14:54, Timon Gehr via Digitalmars-d wrote: On 05/26/2015 06:35 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= " wrote: One of C's design mistakes is to make assignments expressions and not statements. I think it is more about returning void vs. returning the lvalue. The expression/statement distinction is unnecessary. int a, b, c; void f(); f(a=b); void g(T...)(T) {} g(a=b); // and, even when 'void' is not a first class type: void h(int); h(((a=b), c)); artur Sure, but there is no incentive to do this. a[i=j+1]=3; makes the code shorter.
Re: Evaluation order of index expressions
On 05/26/15 18:16, Timon Gehr via Digitalmars-d wrote: > On 05/26/2015 06:13 PM, Artur Skawina via Digitalmars-d wrote: >> On 05/26/15 14:54, Timon Gehr via Digitalmars-d wrote: >>> On 05/26/2015 06:35 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= >>> " wrote: One of C's design mistakes is to make assignments expressions and not statements. >>> >>> I think it is more about returning void vs. returning the lvalue. The >>> expression/statement distinction is unnecessary. >> >> int a, b, c; >> >> void f(); >> f(a=b); >> >> void g(T...)(T) {} >> g(a=b); >> >> // and, even when 'void' is not a first class type: >> void h(int); >> h(((a=b), c)); > Sure, but there is no incentive to do this. a[i=j+1]=3; makes the code > shorter. But does it really make sense to allow it?... Simple errors and typos would result in valid but nonsensical code. In a language with proper 'void', type propagation and generics, the compiler wouldn't be able to catch them. Also: auto i() { return a=b; } (a,b) => a=b To get back to the topic; I can only think of two ways: a) "absolute" LTR b) LTR, except assignments and function calls (the latter is necessary for op overloads; fortunately 'this' is already magic in D, and UFCS could be special cased too). artur
Re: Evaluation order of index expressions
On Tue, 26 May 2015 18:16:57 +0200, Timon Gehr wrote: > Sure, but there is no incentive to do this. a[i=j+1]=3; makes the code > shorter. and harder to read. it is considered bad practice anyway, and will hardly pass any serious code review. signature.asc Description: PGP signature
Re: Evaluation order of index expressions
On Tuesday, 26 May 2015 at 12:51:20 UTC, Timon Gehr wrote: I guess overloaded operators could be made to cache the old value. (As they do in CTFE, apparently. :o)) However, this seems like overkill. Any other ideas? They can but it wouldn't fix anything. The rvalue is already evaluated by then.
Re: Evaluation order of index expressions
On 05/26/2015 07:48 PM, deadalnix wrote: On Tuesday, 26 May 2015 at 12:51:20 UTC, Timon Gehr wrote: I guess overloaded operators could be made to cache the old value. (As they do in CTFE, apparently. :o)) However, this seems like overkill. Any other ideas? They can but it wouldn't fix anything. The rvalue is already evaluated by then. I.e. they can't.
Re: Evaluation order of index expressions
On Tuesday, 26 May 2015 at 22:54:55 UTC, Timon Gehr wrote: On 05/26/2015 07:48 PM, deadalnix wrote: On Tuesday, 26 May 2015 at 12:51:20 UTC, Timon Gehr wrote: I guess overloaded operators could be made to cache the old value. (As they do in CTFE, apparently. :o)) However, this seems like overkill. Any other ideas? They can but it wouldn't fix anything. The rvalue is already evaluated by then. I.e. they can't. You could make the right hand side lazy or something, but yeah, overkill.