rvalues -> ref (yup... again!)
Forked from the x^^y thread... On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d wrote: > On 3/23/2018 11:09 AM, Manu wrote: >> >> [...] > > Rvalue references are not trivial and can have major unintended > consequences. They're a rather ugly feature in C++, with weirdities. I doubt > D will ever have them. Can you please explain these 'weirdities'? What are said "major unintended consequences"? Explain how the situation if implemented would be any different than the workaround? This seems even simpler than the pow thing to me. Rewrite: func(f()); as: { auto __t0 = f(); func(__t0); } How is that worse than the code you have to write: T temp = f(); T zero = 0; func(temp, zero); This 'workaround' is really upsetting. It's ugly, visually noisy, very annoying to implement over and over, and litters the local namespace with rubbish. There's no good name for most of this stuff, they just get named t0, t1, t2... Surely you can see how this objectively makes code worse...? This one more than anything has made my work trying to convince people that D is cool very hard. Most things I can explain around, or say it's a known issue and it'll be better soon. This one is kinda harder; "yeah... that's like, working exactly as intended! I know, isn't D cool!" ;) > But at some level, D cannot replace C++ on a line-by-line basis. There's > always going to be something different. If not in the core language, in the > way the standard library works. If you're heavily using templates and stuff > in C++, you're likely going to have to rethink how the code works to get it > in D (or any other language). I haven't experienced much friction on that front, or had it expressed by new users. They're generally willing to humour that D has differences in it's meta, probably because they understand C++ sucks and drastic changes must be made. They generally complain about choice of '!' for a few minutes and then move on. By contrast, people will NOT forgive the fact that they have to change: func(f(x), f(y), f(z)); to: T temp = f(x); T temp2 = f(y); T temp3 = f(z); func(temp, temp2, temp3); That's just hideous and in-defensible. A better story would be: func(f(x), f(y), f(z)); => func(x.f, y.f, z.f); Instead of being outraged, they'd be further seduced. Sadly, that's not our reality (yet). What a missed opportunity! ;) > For example, in my efforts translating C to D, the clumsy part is the > metaprogramming in the C preprocessor. There's nothing there D cannot do, > but it has to be redesigned. The result is much better, but translating by > rote is simply impossible. I don't often work with C, and I think it's been considered pretty unsavoury to lean heavily on the preprocessor for a few decades now. Even if I did, I probably wouldn't 'port' C, so much as interact with it, and extern(C) works well. I'm more focused on C++, and as such, my experiences differ significantly ;) Reworking a field of preprocessor cruft is in quite a different space than "reworking all calls to func()". People are forgiving of a thing if they appreciate and understand it. > Also, just try translating some of the code in Phobos to C++. It was tried > to do ranges for C++, and the result was terrifying. (It worked, but that's > about all that could be said for it.) I've done comprehensive slices and ranges in C++. It works reasonably well. Certainly, the worst outcome was that I significantly diminished any arguments I had to switch to D... Not sure what your point is though? You'll find no argument from me that slice, ranges, algorithms are awesome, and one of D's main events... but new users need to get far enough into the woods to reach that point. We need to make sure we're the least amount likely to repel them prior to reaching that point as possible. I had a lunch discussion with a colleague today; he believes my claims about ranges/algorithms and stuff, but he has no feel for it, and can't really grok just how ground breaking that stuff is... he admits he needs some experience to comment, and he's actively checking it out. It's critical that he doesn't try to call a function that takes a ref before he reaches that moment, because if he does, chances are he'll get angry at me for wasting his time.
Re: rvalues -> ref (yup... again!)
It never made any sense to me that there could be any problem with the compiler automatically creating a temporary hidden lvalue so a ref could be taken. If there IS any problem, I can only imagine it would be symptomatic of a different, larger problem.
Re: rvalues -> ref (yup... again!)
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: ... By contrast, people will NOT forgive the fact that they have to change: func(f(x), f(y), f(z)); to: T temp = f(x); T temp2 = f(y); T temp3 = f(z); func(temp, temp2, temp3); ... Or you could do this: func(tx=f(x), ty=f(y), tz=f(z)); I know it will not solve your problem, but except for the explicit variables, it's almost like your first example. Well, to be honest I still can't understand why would you want to pass a RValue as reference. Matt.
Re: rvalues -> ref (yup... again!)
On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote: > It never made any sense to me that there could be any problem > with the compiler automatically creating a temporary hidden > lvalue so a ref could be taken. If there IS any problem, I can > only imagine it would be symptomatic of a different, larger > problem. It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used). So, having ref in general accept rvalues would be a huge problem. However, that could be solved by having a different attribute indicate that the idea is that the compiler will accept rvalues and create temporaries for them if they're passed instead of an lvalue. Then, the situation is clear. As for rvalue references in general, I can never remember what exactly the problem is. IIRC, part of it relates to the fact that it makes it impossible for the compiler to know whether it's dealing with an actual lvalue or not, which has serious @safety implications, and in the case of C++, complicates things considerably. Andrei has plenty of nasty things to say about how rvalue references in C++ were a huge mistake. It's just that I can never remember the arguments very well. The use of scope with DIP 1000 may reduce the problem such that scope ref could be made to work @safely (I don't know), but that by itself isn't enough if you want it to be clear whether a function is supposed to be mutating the argument or if it's just trying to avoid copying it. C++ solves that problem by requiring const with rvalue references, but given how D's const works, that really wouldn't make sense. So, we'd have to do something else. I get the impression that this issue is one where it seems like a small one on the surface but that when you get into the guts of what it actually means to implement it, it gets nasty. I think that most of us have just take the approach of passing everything by value unless the type is clearly too expensive to copy for that to make sense, in which case, it's then passed by ref or auto ref, or we make it a reference type. I don't know how much the problem with the lack of rvalue references in D is a PR issue, because C++ programmers get annoyed about it and deem D to be subpar as a result, and how much it's an actual performance problem. I don't work in Manu's world, so I don't what really makes sense there. For me, passing by value is usually a non-issue, and it's easy enough to work around the problem in the rare cases where it is a problem. Clearly, it's a PR issue in Manu's world, but it may also be a performance problem. I don't know. Either way, it's clear that Manu and others like him think that the fact that D doesn't have rvalue references is a serious deficiency. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote: > Well, to be honest I still can't understand why would you want to > pass a RValue as reference. Well, it's frequently the case that you don't want to copy an object if you don't have to - especially in the gaming world, where every ounce of performance matters. If a function accepts an argument by ref, then no copy is made, but then you have to pass an lvalue, making it really annoying to call the function when what you have is an rvalue. On the other hand, if the function doesn't accept its argument by ref, then you can pass an rvalue (and it will be moved, making that efficient), but then lvalues get copied when that's often not what you want. C++'s solution to this was rvalue references. That way, as long as the parameter is const, it can accept both rvalues and lvalues, and it won't copy unless it has to. The closest analogue that D has to this is auto ref, which requires templates, which may or may not be acceptable. In many cases, it works great, whereas in others, it doesn't work at all (e.g. virtual functions), and it can result in template bloat. The whole situation is complicated by the fact that sometimes it's actually faster to pass by value and copy the argument, even if it's an lvalue. Passing by const& can often result in unnecessary copies if anywhere in the chain passes the object by value, whereas if it's passed by value, the same object can often be moved multiple times, avoiding any copies. It's my understanding that prior to C++11, it was considered best practice to pass by const& as much as possible to avoid copies but that after C++11 (which added move constructors), it's often considered better to pass by value, because then in many cases, the compiler can move the object instead of copying it. So, C++ gives you control over which you do, and it's not necessarily straightforward as to which you should use (though plenty of older C++ programmers likely just use const& all over the place out of habit). D supports moving out of the box without move constructors, so it does a good job of handling the cases where a move is most appropriate, but it doesn't have rvalue references, so it doesn't have the same flexibility as C++ when you want to pass something by reference. So, anyone looking to pass stuff by reference all over the place (like Manu and his coworkers) is going to find the lack of rvalue references in D to be really annoying. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d wrote: > On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote: >> Well, to be honest I still can't understand why would you want to >> pass a RValue as reference. > > Well, it's frequently the case that you don't want to copy an object if you > don't have to - especially in the gaming world, where every ounce of > performance matters. If a function accepts an argument by ref, then no copy > is made, but then you have to pass an lvalue, making it really annoying to > call the function when what you have is an rvalue. On the other hand, if the > function doesn't accept its argument by ref, then you can pass an rvalue > (and it will be moved, making that efficient), but then lvalues get copied > when that's often not what you want. C++'s solution to this was rvalue > references. Ummm... rvalue-references are something completely different. rval-ref's are C++'s solution to move semantics. C++ just simply accepts rvalues passed to const& args. It makes a temp and passes the ref, as you expect. > That way, as long as the parameter is const, it can accept both > rvalues and lvalues, and it won't copy unless it has to. The closest > analogue that D has to this is auto ref, which requires templates, which may > or may not be acceptable. auto-ref == template function, which by definition is NOT an extern(C++) function. auto-ref may also resolve to NOT a ref, which means a move... data structures that are large (ie, a vector or matrix) still have to copy a bunch of memory, even if the copy is algorithmically 'cheap'. > In many cases, it works great, whereas in others, > it doesn't work at all (e.g. virtual functions), and it can result in > template bloat. And templates, that's another case where it fails. auto-ref is something else unrelated to this topic, and it's useful in a different set of cases for a different set of uses/reasons. It's got nothing to do with this. > The whole situation is complicated by the fact that sometimes it's actually > faster to pass by value and copy the argument, even if it's an lvalue. The api author wouldn't have made the arg a ref in that case. > Passing by const& can often result in unnecessary copies if anywhere in the > chain passes the object by value, whereas if it's passed by value, the same > object can often be moved multiple times, avoiding any copies**. **avoiding __calls to the copy constructor__. The memory is likely to still be moved around a bunch of times. The case you're thinking of is when values are *returned* by value, in that case, copy elision is possible, and the result can be constructed in place. That's a completely unrelated problem... you don't do return-by-ref ;) > It's my > understanding that prior to C++11, it was considered best practice to pass > by const& as much as possible to avoid copies but that after C++11 (which > added move constructors), it's often considered better to pass by value, > because then in many cases, the compiler can move the object instead of > copying it**. ** Assuming the object is tiny, but has an expensive copy constructor. In the case where an object is large (and has a primitive, or no copy constructor) it doesn't change the situation; you still wanna pass a big thing by ref in all cases; ie, vector/matrix. > So, C++ gives you control over which you do, and it's not > necessarily straightforward as to which you should use (though plenty of > older C++ programmers likely just use const& all over the place out of > habit). D gives you the same set of options; except that passing args by ref is a PITA in D, and ruins your code.
Re: rvalues -> ref (yup... again!)
On 23 March 2018 at 16:46, Jonathan M Davis via Digitalmars-d wrote: > On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote: >> It never made any sense to me that there could be any problem >> with the compiler automatically creating a temporary hidden >> lvalue so a ref could be taken. If there IS any problem, I can >> only imagine it would be symptomatic of a different, larger >> problem. > > It can be a serious API problem if you can't look at ref and know that the > intention is that the original argument's value will be used and then > mutated (whereas with out, it will be mutated, but the original value won't > be used). Okay, let's read 'const ref' every time I say 'ref'. I thought that would be fairly safe to assume. Sorry! > However, that could be solved by having a different attribute indicate that > the idea is that the compiler will accept rvalues and create temporaries for > them if they're passed instead of an lvalue. Then, the situation is clear. No, no attributes.. Just accept any argument to const-ref! The function's not gonna change it. > As for rvalue references in general, I can never remember what exactly the > problem is. IIRC, part of it relates to the fact that it makes it impossible > for the compiler to know whether it's dealing with an actual lvalue or not, > which has serious @safety implications, and in the case of C++, complicates > things considerably. Andrei has plenty of nasty things to say about how > rvalue references in C++ were a huge mistake. It's just that I can never > remember the arguments very well. > > The use of scope with DIP 1000 may reduce the problem such that scope ref > could be made to work @safely (I don't know) True, but if you follow that line of reasoning, then this case must equally be banned: T temp = f(); func(temp); The @safety fear is that the pointer to the stack arg may escape, and that's identical whether the argument is an explicit temp, or an implicit one. > but that by itself isn't > enough if you want it to be clear whether a function is supposed to be > mutating the argument Functions that receive const args make it pretty clear that they don't intend to mutate the arg. > I get the impression that this issue is one where it seems like a small one > on the surface but that when you get into the guts of what it actually means > to implement it, it gets nasty. It really doesn't... the compiler just make the same temp that I type with my hands. That's literally all that it needs to do. I'd bet money it's a one-paragraph change. > Clearly, it's a PR issue in Manu's world, but it may also be a performance > problem. I > don't know. Either way, it's clear that Manu and others like him think that > the fact that D doesn't have rvalue references is a serious deficiency. That's an understatement, but yes :P We want to call C++ code. The D code shouldn't be objectively worse than the C++ code we're trying to escape, otherwise we're undermining our gravity towards D. We can't have a situation that goes "I wrote this 3 line function in D, but now it's 6 lines because I had to break args to temps, it's objectively worse than the equivalent C++ function... why am I writing it in D again?".
Re: rvalues -> ref (yup... again!)
On Friday, 23 March 2018 at 23:58:05 UTC, Jonathan M Davis wrote: On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote: Well, to be honest I still can't understand why would you want to pass a RValue as reference. Well, it's frequently the case that you don't want to copy an object if you don't have to... - Jonathan M Davis Well the concept it's OK. (Differences between passing by value vs reference, copy etc.). Except for the const thing in C++, because I don't know this language, and by the way thanks for explaining that. Question: In C++ the signature of the function which will receive the references like in this case, need to be "const ref" parameters, right? - If yes, then since it's const ref parameter, will not change the value passed, even if it's lvalue, right? Matt.
Re: rvalues -> ref (yup... again!)
On 23 March 2018 at 17:48, MattCoder via Digitalmars-d wrote: > > Question: > > In C++ the signature of the function which will receive the references like > in this case, need to be "const ref" parameters, right? - If yes, then since > it's const ref parameter, will not change the value passed, even if it's > lvalue, right? Right. It's so obvious isn't it ;)
Re: rvalues -> ref (yup... again!)
On Friday, March 23, 2018 17:35:11 Manu via Digitalmars-d wrote: > > but that by itself isn't > > enough if you want it to be clear whether a function is supposed to be > > mutating the argument > > Functions that receive const args make it pretty clear that they don't > intend to mutate the arg. Yes, but with how restrictive const is in D, I have a very hard time believing that it's going to work well to start using const ref much even if it accepted rvalues. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On Friday, March 23, 2018 17:20:09 Manu via Digitalmars-d wrote: > On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d > > wrote: > > On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote: > >> Well, to be honest I still can't understand why would you want to > >> pass a RValue as reference. > > > > Well, it's frequently the case that you don't want to copy an object if > > you don't have to - especially in the gaming world, where every ounce > > of performance matters. If a function accepts an argument by ref, then > > no copy is made, but then you have to pass an lvalue, making it really > > annoying to call the function when what you have is an rvalue. On the > > other hand, if the function doesn't accept its argument by ref, then > > you can pass an rvalue (and it will be moved, making that efficient), > > but then lvalues get copied when that's often not what you want. C++'s > > solution to this was rvalue references. > > Ummm... rvalue-references are something completely different. > rval-ref's are C++'s solution to move semantics. > C++ just simply accepts rvalues passed to const& args. It makes a temp > and passes the ref, as you expect. It was my understanding that that _was_ an rvalue reference, and I remember that Andrei was against const ref accepting rvalues in D due to issues with rvalue references. It's quite possible that I misremember though, and I certainly am not an expert on all of the issues with rvalues and references in C++, and my C++ knowledge is getting increasingly rusty, since I don't use C++ much anymore. In any case, I have a terrible time remembering Andrei's exact arguments, but he feels very strongly about them, so anyone looking to convince him is going to have a hard time of it. My biggest concern in all of this is that I don't want to see ref start accepting rvalues as has been occasionally discussed. It needs to be clear when a function is accept an argument by ref because it's going to mutate the object and when it's accepting by ref because it wants to avoid a copy. The addition of const solves that problem for C++, but given how restrictive const is in D, I doubt that much of anyone would ultimately be very happy with using const ref in their code very often. > > In many cases, it works great, whereas in others, > > it doesn't work at all (e.g. virtual functions), and it can result in > > template bloat. > > And templates, that's another case where it fails. > auto-ref is something else unrelated to this topic, and it's useful in > a different set of cases for a different set of uses/reasons. It's got > nothing to do with this. auto ref has everything to do with a function accepting both rvalues and lvalues without making a copy. auto ref is the solution that was introduced into D to solve that very problem. It's just that it has limitations that make it inappropriate in a number of cases, so you don't consider it a solution to your problem. > > So, C++ gives you control over which you do, and it's not > > necessarily straightforward as to which you should use (though plenty of > > older C++ programmers likely just use const& all over the place out of > > habit). > > D gives you the same set of options; except that passing args by ref > is a PITA in D, and ruins your code. Which is why I said that D doesn't give you the same set of options. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On 23 March 2018 at 17:58, Jonathan M Davis via Digitalmars-d wrote: > On Friday, March 23, 2018 17:35:11 Manu via Digitalmars-d wrote: >> > but that by itself isn't >> > enough if you want it to be clear whether a function is supposed to be >> > mutating the argument >> >> Functions that receive const args make it pretty clear that they don't >> intend to mutate the arg. > > Yes, but with how restrictive const is in D, I have a very hard time > believing that it's going to work well to start using const ref much even if > it accepted rvalues. This is an interesting point, but I don't think it changes the balance in any way. Thinking of the majority of my anecdotal cases, I don't think it would be a problem. Something complex enough for const to be a problem likely doesn't conform to this pattern. Further, extern(C++) functions that receive const& args are 'const ref' regardless of D's const semantics. So for extern(C++), which I suspect is a high percentage of cases where this issue is significant (because D doesn't naturally follow C++'s pattern so much anyway), that point doesn't matter.
Re: rvalues -> ref (yup... again!)
On 23 March 2018 at 18:06, Jonathan M Davis via Digitalmars-d wrote: > On Friday, March 23, 2018 17:20:09 Manu via Digitalmars-d wrote: >> On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d >> >> wrote: >> > On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote: >> >> Well, to be honest I still can't understand why would you want to >> >> pass a RValue as reference. >> > >> > Well, it's frequently the case that you don't want to copy an object if >> > you don't have to - especially in the gaming world, where every ounce >> > of performance matters. If a function accepts an argument by ref, then >> > no copy is made, but then you have to pass an lvalue, making it really >> > annoying to call the function when what you have is an rvalue. On the >> > other hand, if the function doesn't accept its argument by ref, then >> > you can pass an rvalue (and it will be moved, making that efficient), >> > but then lvalues get copied when that's often not what you want. C++'s >> > solution to this was rvalue references. >> >> Ummm... rvalue-references are something completely different. >> rval-ref's are C++'s solution to move semantics. >> C++ just simply accepts rvalues passed to const& args. It makes a temp >> and passes the ref, as you expect. > > It was my understanding that that _was_ an rvalue reference, and I remember > that Andrei was against const ref accepting rvalues in D due to issues with > rvalue references. C++ const& to 'rvalues' are just lvalue refs to temporaries, exactly as I propose here. rval-ref's are something completely different (all about move semantics), and have nothing to do with this conversation. There is a potentially interesting parallel conversation which discusses how to interact with extern(C++) functions that receive rvalue ref's, but that actually is a complex conversation, and no simple answers exist. > In any case, I have a terrible time remembering > Andrei's exact arguments, but he feels very strongly about them, so anyone > looking to convince him is going to have a hard time of it. Fortunately, I'm not trying to make any sort of argument for rvalue-ref's in D. > My biggest concern in all of this is that I don't want to see ref start > accepting rvalues as has been occasionally discussed. It needs to be clear > when a function is accept an argument by ref because it's going to mutate > the object and when it's accepting by ref because it wants to avoid a copy. It's not going to mutate the argument, because it's const. > The addition of const solves that problem for C++, but given how restrictive > const is in D, I doubt that much of anyone would ultimately be very happy > with using const ref in their code very often. I expect I'll be 100% satisfied. And if not, it'll be something very close to 100%. This pattern is quite unlikely to proliferate within D code natively (because auto ref, D move semantics, classes-as-ref-types, and such), but it's essential for interacting with C++. > auto ref has everything to do with a function accepting both rvalues and > lvalues without making a copy. auto ref is the solution that was introduced > into D to solve that very problem. Yeah, somehow that emerged from this conversation years ago. I aggressively expressed at the time that I never accepted it as a solution, because it's not. It's got nothing to do with this issue, and I said at the time that I'll be very annoyed if it starts getting raised in this context ;) > It's just that it has limitations that > make it inappropriate in a number of cases, so you don't consider it a > solution to your problem. It's orthogonal to this conversation. It's the ability for templates to automate the ref-ness of args for calling efficiency. It's something like scott myers 'universal references'; ie, `template void func(T&& arg)`. It's really got nothing to do with this conversation. >> > So, C++ gives you control over which you do, and it's not >> > necessarily straightforward as to which you should use (though plenty of >> > older C++ programmers likely just use const& all over the place out of >> > habit). >> >> D gives you the same set of options; except that passing args by ref >> is a PITA in D, and ruins your code. > > Which is why I said that D doesn't give you the same set of options. It does though; D just forces you to write the temps that should be implicit by hand. There's no reason for this. It just makes code ugly, people angry, and there is no advantage.
Re: rvalues -> ref (yup... again!)
On 24.03.2018 01:35, Manu wrote: Okay, let's read 'const ref' every time I say 'ref'. I thought that would be fairly safe to assume. Sorry! Absolutely not. It makes absolutely no sense to restrict rvalue references to const objects. (Recall that const is transitive and actually prevents mutation. This is not C++.)
Re: rvalues -> ref (yup... again!)
On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d wrote: > On 24.03.2018 01:35, Manu wrote: >> >> Okay, let's read 'const ref' every time I say 'ref'. I thought that >> would be fairly safe to assume. Sorry! > > > Absolutely not. It makes absolutely no sense to restrict rvalue references > to const objects. (Recall that const is transitive and actually prevents > mutation. This is not C++.) We're not talking about rvalue-references... were talking about not-rvalue-references to temporaries.
Re: rvalues -> ref (yup... again!)
On 24.03.2018 02:16, Manu wrote: This is an interesting point, but I don't think it changes the balance in any way. Thinking of the majority of my anecdotal cases, I don't think it would be a problem. Something complex enough for const to be a problem likely doesn't conform to this pattern. Why aim for "it often works", when we want "it always works"? Forcing const upon people who want to pass rvalues by reference is just not good enough. It is bad language design. Also I think the point about documenting mutation intent is moot, as rvalues can be receivers for method calls, which will _already_ pass an rvalue by reference no matter whether it intends to mutate it or not. We can require some special annotation for this behavior, but I'd be perfectly fine without it.
Re: rvalues -> ref (yup... again!)
On 24.03.2018 04:14, Manu wrote: On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d wrote: On 24.03.2018 01:35, Manu wrote: Okay, let's read 'const ref' every time I say 'ref'. I thought that would be fairly safe to assume. Sorry! Absolutely not. It makes absolutely no sense to restrict rvalue references to const objects. (Recall that const is transitive and actually prevents mutation. This is not C++.) We're not talking about rvalue-references... were talking about not-rvalue-references to temporaries. Let me rephrase: Absolutely not. It makes absolutely no sense to force const for not-rvalue-references to temporaries. (Recall that const is transitive and actually prevents mutation. This is not C++.)
Re: rvalues -> ref (yup... again!)
On 23 March 2018 at 20:17, Timon Gehr via Digitalmars-d wrote: > On 24.03.2018 02:16, Manu wrote: >> >> This is an interesting point, but I don't think it changes the balance >> in any way. Thinking of the majority of my anecdotal cases, I don't >> think it would be a problem. >> Something complex enough for const to be a problem likely doesn't >> conform to this pattern. > > > Why aim for "it often works", when we want "it always works"? Forcing const > upon people who want to pass rvalues by reference is just not good enough. > It is bad language design. I think you need to re-read the whole thread, and understand what we're even talking about. Nobody wants to pass rvalues by mutable-ref... that's completely pointless, since it's an rvalue that will timeout immediately anyway. Passing by const-ref is perfectly acceptable. I suspect Jonathan's talking about classic D situations with const like, I might pass by const-ref, but then I can't call a getter that caches the result. That's a classic problem with D's const, and that's not on debate here. I don't think that has any impact on this proposal; people understand what const means in D, and that's no different here than anywhere else. > Also I think the point about documenting mutation intent is moot, as rvalues > can be receivers for method calls, which will _already_ pass an rvalue by > reference no matter whether it intends to mutate it or not. We can require > some special annotation for this behavior, but I'd be perfectly fine without > it. I have no idea what this paragraph means... can you elaborate further what you're talking about?
Re: rvalues -> ref (yup... again!)
On 23 March 2018 at 20:19, Timon Gehr via Digitalmars-d wrote: > On 24.03.2018 04:14, Manu wrote: >> >> On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d >> wrote: >>> >>> On 24.03.2018 01:35, Manu wrote: Okay, let's read 'const ref' every time I say 'ref'. I thought that would be fairly safe to assume. Sorry! >>> >>> >>> >>> Absolutely not. It makes absolutely no sense to restrict rvalue >>> references >>> to const objects. (Recall that const is transitive and actually prevents >>> mutation. This is not C++.) >> >> >> We're not talking about rvalue-references... were talking about >> not-rvalue-references to temporaries. >> > > Let me rephrase: > > Absolutely not. It makes absolutely no sense to force const for > not-rvalue-references to temporaries. (Recall that const is transitive and > actually prevents mutation. This is not C++.) Yes, I know... why would you mutate an argument who's life doesn't extend beyond the call? Any mutation is pointless.
Re: rvalues -> ref (yup... again!)
On 03/23/2018 07:46 PM, Jonathan M Davis wrote: On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote: It never made any sense to me that there could be any problem with the compiler automatically creating a temporary hidden lvalue so a ref could be taken. If there IS any problem, I can only imagine it would be symptomatic of a different, larger problem. It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used). ???. That's equally true when an lvalue is passed in. However, that could be solved by having a different attribute indicate that the idea is that the compiler will accept rvalues and create temporaries for them if they're passed instead of an lvalue. Then, the situation is clear. Why require the callee's author to add boilerplate? Just do it for all ref params that are given an rvalue. As for rvalue references in general, I can never remember what exactly the problem is. IIRC, part of it relates to the fact that it makes it impossible for the compiler to know whether it's dealing with an actual lvalue or not, As long as the compiler takes the advocated "automatic hidden temporary" approach, then it *is* an actual lvalue. We're talking nothing more than sugar here. Sugar for what we're already forced to do manually. There *cannot* be a technical problem here that isn't *already* a problem with the author manually creating a temp var.
Re: rvalues -> ref (yup... again!)
On Saturday, March 24, 2018 01:37:10 Nick Sabalausky via Digitalmars-d wrote: > On 03/23/2018 07:46 PM, Jonathan M Davis wrote: > > On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote: > >> It never made any sense to me that there could be any problem > >> with the compiler automatically creating a temporary hidden > >> lvalue so a ref could be taken. If there IS any problem, I can > >> only imagine it would be symptomatic of a different, larger > >> problem. > > > > It can be a serious API problem if you can't look at ref and know that > > the intention is that the original argument's value will be used and > > then mutated (whereas with out, it will be mutated, but the original > > value won't be used). > > ???. That's equally true when an lvalue is passed in. > > > However, that could be solved by having a different attribute indicate > > that the idea is that the compiler will accept rvalues and create > > temporaries for them if they're passed instead of an lvalue. Then, the > > situation is clear. > Why require the callee's author to add boilerplate? Just do it for all > ref params that are given an rvalue. Because if the point of the function accepting its argument by ref is because it's going to mutate the argument, then it makes no sense for it to accept rvalues, whereas if the point of the function accepting its argument by ref is to just avoid a copy, then it makes sense to accept both. It should be clear from the API which is intended. As it stands, because a function can't accept rvalues by ref, it's usually reasonable to assume that a function accepts its argument by ref because it's mutating that argument rather than simply because it's trying to avoid a copy. If ref suddenly starts accepting rvalues, then we lose that. With C++, you know the difference because, only const& accepts rvalues, and whether it's const or some other attribute, I'd very much like that any ref that accepts rvalues in D be marked with something to indicate that it does rather than making ref do double-duty and make it unclear what it's there for. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On 24 March 2018 at 00:04, Jonathan M Davis via Digitalmars-d wrote: > On Saturday, March 24, 2018 01:37:10 Nick Sabalausky via Digitalmars-d > wrote: >> On 03/23/2018 07:46 PM, Jonathan M Davis wrote: >> > On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d > wrote: >> >> It never made any sense to me that there could be any problem >> >> with the compiler automatically creating a temporary hidden >> >> lvalue so a ref could be taken. If there IS any problem, I can >> >> only imagine it would be symptomatic of a different, larger >> >> problem. >> > >> > It can be a serious API problem if you can't look at ref and know that >> > the intention is that the original argument's value will be used and >> > then mutated (whereas with out, it will be mutated, but the original >> > value won't be used). >> >> ???. That's equally true when an lvalue is passed in. >> >> > However, that could be solved by having a different attribute indicate >> > that the idea is that the compiler will accept rvalues and create >> > temporaries for them if they're passed instead of an lvalue. Then, the >> > situation is clear. >> Why require the callee's author to add boilerplate? Just do it for all >> ref params that are given an rvalue. > > Because if the point of the function accepting its argument by ref is > because it's going to mutate the argument, then it makes no sense for it to > accept rvalues, whereas if the point of the function accepting its argument > by ref is to just avoid a copy, then it makes sense to accept both. It > should be clear from the API which is intended. Write const; it's as clear as day!
Re: rvalues -> ref (yup... again!)
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: Forked from the x^^y thread... On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d wrote: On 3/23/2018 11:09 AM, Manu wrote: [...] Rvalue references are not trivial and can have major unintended consequences. They're a rather ugly feature in C++, with weirdities. I doubt D will ever have them. Can you please explain these 'weirdities'? What are said "major unintended consequences"? Explain how the situation if implemented would be any different than the workaround? This seems even simpler than the pow thing to me. Rewrite: func(f()); as: { auto __t0 = f(); func(__t0); } I understand what you want, but I'm struggling to understand why it's such a huge deal. The reason you want to pass by reference is for performance, to avoid copying the data at the call boundary. So there are 2 cases: an lvalue needs to be passed, or an rvalue needs to be passed. 1. The address of the lvalue is passed. 2. The rvalue is copied to a local, then the address of that local is passed. So in the rvalue case, you're not getting the performance benefit of passing by reference, because you have to copy to a local anyway. What I would do in D currently to get the same performance and API: void foo(float[32] v) { foo(v); } void foo(ref float[32] v) { ... } or void foo()(auto ref float[32] v) { ... } What is so totally unacceptable about those solutions? I personally like the second because it scales better to multiple parameters. I know you have said it's not relevant and annoying that people bring up auto ref, but I dont' get how or why. It's exactly D's solution to the problem. There's a little more work to be done when thinking about extern(C++) and/or virtual functions, but most code for most people isn't made of virtual extern(C++) functions that take large value types can't accept the cost of copying a few lvalues.
Re: rvalues -> ref (yup... again!)
On Saturday, 24 March 2018 at 11:57:25 UTC, John Colvin wrote: I understand what you want, but I'm struggling to understand why it's such a huge deal. ... What I would do in D currently to get the same performance and API: void foo(float[32] v) { foo(v); } void foo(ref float[32] v) { ... } or void foo()(auto ref float[32] v) { ... } What is so totally unacceptable about those solutions? I hope OP answers that, because that was what I tried to refer in my post. As your example, it works like in C and I'd prefer to use it in my code, instead of passing a rvalue as reference. Matt.
Re: rvalues -> ref (yup... again!)
On 24.03.2018 05:03, Manu wrote: On 23 March 2018 at 20:17, Timon Gehr via Digitalmars-d wrote: On 24.03.2018 02:16, Manu wrote: This is an interesting point, but I don't think it changes the balance in any way. Thinking of the majority of my anecdotal cases, I don't think it would be a problem. Something complex enough for const to be a problem likely doesn't conform to this pattern. Why aim for "it often works", when we want "it always works"? Forcing const upon people who want to pass rvalues by reference is just not good enough. It is bad language design. I think you need to re-read the whole thread, and understand what we're even talking about. ... That will not be necessary. I wouldn't even have had to read it the first time. Those discussions always go the same way: M: I wish we could pass rvalue arguments to ref parameters. J: That would be terrible, as people would then pass rvalues as ref by accident and not see the mutation that the author of the function intended them to see. M: Only do it for const ref parameters then. T: No, this has nothing to do with const. (M can be replaced by a variety of other letters; this is a somewhat common feature request.) Nobody wants to pass rvalues by mutable-ref... that's completely pointless, since it's an rvalue that will timeout immediately anyway. Passing by const-ref is perfectly acceptable. ... Your temporary might have mutable indirections. Maybe you don't want to be forced to annotate your methods as const, limiting your future options. I suspect Jonathan's talking about classic D situations with const like, I might pass by const-ref, but then I can't call a getter that caches the result. That's a classic problem with D's const, and that's not on debate here. I don't think that has any impact on this proposal; people understand what const means in D, and that's no different here than anywhere else. ... Your proposal _changes_ the meaning of const. I.e., "const does all it did previously and now it also allows rvalues to be passed to ref functions". This is bad, as one has little to do with the other, yet now you couple them. Programmers who want to pass rvalues as ref do not necessarily want to use D const on their objects. Also I think the point about documenting mutation intent is moot, as rvalues can be receivers for method calls, which will _already_ pass an rvalue by reference no matter whether it intends to mutate it or not. We can require some special annotation for this behavior, but I'd be perfectly fine without it. I have no idea what this paragraph means... can you elaborate further what you're talking about? This works: struct S{ int x; void inc(){ this.x++; // note: 'this' is passed by ref } } void main(){ S().inc(); } but this does not: struct S{ int x; } void inc(ref S self){ self.x++; // note: 'self' is passed by ref } void main(){ S().inc(); } I.e. there is a special case where your rewrite is already applied. Note how "inc" cannot even be made const. What I'm saying is that I don't really buy Jonathan's argument. Basically, you should just pass the correct arguments to functions, as you always need to do. If you cannot use the result of some mutation that you need to use, you will probably notice. There are only three sensible ways to fix the problem: 1. Just allow rvalue arguments to bind to ref parameters. (My preferred solution, though it will make the overloading rules slightly more complicated.) 2. Add some _new_ annotation for ref parameters that signifies that you want the same treatment for them that the implicit 'this' reference gets. (A close second.) 3. Continue to require code bloat (auto ref) or manual boilerplate (overloads). (I'm not very fond of this option, but it does not require a language change.)
Re: rvalues -> ref (yup... again!)
On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote: What I'm saying is that I don't really buy Jonathan's argument. Basically, you should just pass the correct arguments to functions, as you always need to do. If you cannot use the result of some mutation that you need to use, you will probably notice. I agree, but restricting it to const ref would be enough for almost all of my use cases. The MS C++ compiler just emits a warning when binding an rvalue to a mutable ref ('nonstandard extension used'), I'd find that absolutely viable for D too. There are only three sensible ways to fix the problem: 1. Just allow rvalue arguments to bind to ref parameters. (My preferred solution, though it will make the overloading rules slightly more complicated.) I always thought the main concern was potential escaping refs to the rvalue, which would be solvable by allowing rvalues to be bound to `scope ref` params only. That'd be my preferred choice as well. 2. Add some _new_ annotation for ref parameters that signifies that you want the same treatment for them that the implicit 'this' reference gets. (A close second.) *Shudder*. 3. Continue to require code bloat (auto ref) or manual boilerplate (overloads). (I'm not very fond of this option, but it does not require a language change.) While `auto ref` seems to have worked out surprisingly well for code written in D, it doesn't solve the problem when interfacing with (many) external C++ functions taking structs (represented in D by structs as well) by (mostly const) ref. You're forced to declare lvalues for all of these args, uglifying the code substantially.
Re: rvalues -> ref (yup... again!)
On 24.03.2018 15:56, kinke wrote: On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote: What I'm saying is that I don't really buy Jonathan's argument. Basically, you should just pass the correct arguments to functions, as you always need to do. If you cannot use the result of some mutation that you need to use, you will probably notice. I agree, but restricting it to const ref would be enough for almost all of my use cases. The MS C++ compiler just emits a warning when binding an rvalue to a mutable ref ('nonstandard extension used'), I'd find that absolutely viable for D too. ... A warning is not viable. (There's no good way to fix it.) There are only three sensible ways to fix the problem: 1. Just allow rvalue arguments to bind to ref parameters. (My preferred solution, though it will make the overloading rules slightly more complicated.) I always thought the main concern was potential escaping refs to the rvalue, which would be solvable by allowing rvalues to be bound to `scope ref` params only. That'd be my preferred choice as well. ... There is no difference between escaping refs to an rvalue and escaping refs to a short-lived lvalue, as the callee has no idea where the address is coming from anyway. According to Walter, ref parameters are not supposed to be escaped, and @safe will enforce it. Also, AFAIU, "scope" in "scope ref T" already applies to "T", not "ref". 2. Add some _new_ annotation for ref parameters that signifies that you want the same treatment for them that the implicit 'this' reference gets. (A close second.) *Shudder*. ... Well, it beats "const". 3. Continue to require code bloat (auto ref) or manual boilerplate (overloads). (I'm not very fond of this option, but it does not require a language change.) While `auto ref` seems to have worked out surprisingly well for code written in D, it doesn't solve the problem when interfacing with (many) external C++ functions taking structs (represented in D by structs as well) by (mostly const) ref. You're forced to declare lvalues for all of these args, uglifying the code substantially. You can add additional overloads on the D side. (This can even be automated using a string mixin.)
Re: rvalues -> ref (yup... again!)
On Saturday, 24 March 2018 at 15:36:14 UTC, Timon Gehr wrote: On 24.03.2018 15:56, kinke wrote: I agree, but restricting it to const ref would be enough for almost all of my use cases. The MS C++ compiler just emits a warning when binding an rvalue to a mutable ref ('nonstandard extension used'), I'd find that absolutely viable for D too. ... A warning is not viable. (There's no good way to fix it.) As long as specific warnings cannot be suppressed via pragmas, one would need to predeclare the lvalue to get rid of it; fine IMHO for the, as I expect, very rare use cases. There is no difference between escaping refs to an rvalue and escaping refs to a short-lived lvalue, as the callee has no idea where the address is coming from anyway. According to Walter, ref parameters are not supposed to be escaped, and @safe will enforce it. Alright, the less keywords overhead, the better. :) You can add additional overloads on the D side. (This can even be automated using a string mixin.) Right I can, but I don't want to add 7 overloads for a C++ function taking 3 params by const ref. Even if autogenerated by some tool or fancy mixins, the code's legibility would suffer a lot. D's syntax is IMO one of its strongest selling points, and that shouldn't degrade when it comes to C(++) interop.
Re: rvalues -> ref (yup... again!)
Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis: > As it stands, because a function can't accept rvalues by ref, it's > usually reasonable to assume that a function accepts its argument by ref > because it's mutating that argument rather than simply because it's > trying to avoid a copy. If ref suddenly starts accepting rvalues, then > we lose that. Any reason you can't simply use `ref` to imply 'modifies value' and `const ref` as 'passed by ref for performance reasons'? -- Johannes
Re: rvalues -> ref (yup... again!)
Am Sat, 24 Mar 2018 17:10:53 + schrieb Johannes Pfau: > Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis: > >> As it stands, because a function can't accept rvalues by ref, it's >> usually reasonable to assume that a function accepts its argument by >> ref because it's mutating that argument rather than simply because it's >> trying to avoid a copy. If ref suddenly starts accepting rvalues, then >> we lose that. > > Any reason you can't simply use `ref` to imply 'modifies value' and > `const ref` as 'passed by ref for performance reasons'? Sorry, I see Manu already asked the same question. -- Johannes
Re: rvalues -> ref (yup... again!)
On 24 March 2018 at 04:57, John Colvin via Digitalmars-d wrote: > On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: >> >> Forked from the x^^y thread... >> >> On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d >> wrote: >>> >>> On 3/23/2018 11:09 AM, Manu wrote: [...] >>> >>> >>> Rvalue references are not trivial and can have major unintended >>> consequences. They're a rather ugly feature in C++, with weirdities. I doubt >>> D will ever have them. >> >> >> Can you please explain these 'weirdities'? >> What are said "major unintended consequences"? >> Explain how the situation if implemented would be any different than >> the workaround? >> >> This seems even simpler than the pow thing to me. >> Rewrite: >> func(f()); >> as: >> { auto __t0 = f(); func(__t0); } >> > > I understand what you want, but I'm struggling to understand why it's such a > huge deal. Because it makes this kind of D code that interacts with C++ objectively worse than C++, and there's no reason for it. You can't say to someone who just frustrated-ly doubled their line count by manually introducing a bunch of temporaries in a tiny function that appears to do something so simple as 'call a function', that "oh yeah, isn't it cool that you can't just call your functions anymore! isn't D cool! we should switch to D right?" It's embarrassing. I've been put in the position where I have to try and 'explain' this feature quite some number of times... they usually just give me 'the look'â„¢; ya know, quietly wondering if I'm still sane, and all I end up with is someone who's about 95% less convinced that D is cool than they were 5 seconds beforehand. What pisses me off is that's such a pointless thing to happen, because this issue is so trivial! In my experience, people are evaluating how D will materially impact the exact same code they're already writing in C++. This is one of those ways that they will be materially impacted, and it's almost enough all on its own to cause people to dismiss the entire thing on the spot. Pretty much the best case at this phase is that the D code is exactly the same as C++. If we can trim off a few parens here and there (ufcs?), maybe remove some '::' operators (modules that don't suck), that's a huge win. > The reason you want to pass by reference is for performance, to avoid > copying the data at the call boundary. > > So there are 2 cases: an lvalue needs to be passed, or an rvalue needs to be > passed. > > 1. The address of the lvalue is passed. > > 2. The rvalue is copied to a local, then the address of that local is > passed. > > So in the rvalue case, you're not getting the performance benefit of passing > by reference, because you have to copy to a local anyway. > > What I would do in D currently to get the same performance and API: > > void foo(float[32] v) { foo(v); } > void foo(ref float[32] v) { ... } > > or > > void foo()(auto ref float[32] v) { ... } Can't be extern(C++), can't be virtual either (both is likely). I said before; you're talking about Scott Meyers 'universal references' as a language concept, and I'm just talking about calling a function. > but I dont' get how or why. It's exactly D's solution to the problem. It doesn't solve the problem... it doesn't even address the problem. You're talking about a totally different thing >_< > There's a little more work to be done when thinking about extern(C++) and/or > virtual functions, but most code for most people isn't made of virtual > extern(C++) functions that take large value types can't accept the cost of > copying a few lvalues. Correct, as I said before; people are getting bent out of shape over a thing that will likely never affect them! This will likely have very little impact on normal D code because D const, auto ref, D move semantics, etc. It will have large impact on interaction with C++ code, and the impression D is able to make on that set of users. They're an important target market.
Re: rvalues -> ref (yup... again!)
On 24 March 2018 at 06:49, Timon Gehr via Digitalmars-d wrote: > On 24.03.2018 05:03, Manu wrote: >> >> I have no idea what this paragraph means... can you elaborate further >> what you're talking about? > > This works: > > struct S{ > int x; > void inc(){ > this.x++; // note: 'this' is passed by ref > } > } > > void main(){ > S().inc(); > } > > but this does not: > > struct S{ > int x; > } > void inc(ref S self){ > self.x++; // note: 'self' is passed by ref > } > > void main(){ > S().inc(); > } > > I.e. there is a special case where your rewrite is already applied. Note how > "inc" cannot even be made const. > > What I'm saying is that I don't really buy Jonathan's argument. Basically, > you should just pass the correct arguments to functions, as you always need > to do. If you cannot use the result of some mutation that you need to use, > you will probably notice. Your example demonstrates the exact reason why rvalue->ref is const& in C++, and illegal in D though. You mutate a temporary that times out at the end of the statement... your statement is never assigned to anything, and has no effect. If your statement is assigned to something, then you already have an lvalue to pass to such a function that receives mutable ref.
Re: rvalues -> ref (yup... again!)
On 24 March 2018 at 07:56, kinke via Digitalmars-d wrote: > On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote: >> >> What I'm saying is that I don't really buy Jonathan's argument. Basically, >> you should just pass the correct arguments to functions, as you always need >> to do. If you cannot use the result of some mutation that you need to use, >> you will probably notice. > > > I agree, but restricting it to const ref would be enough for almost all of > my use cases. The MS C++ compiler just emits a warning when binding an > rvalue to a mutable ref ('nonstandard extension used'), I'd find that > absolutely viable for D too. > >> There are only three sensible ways to fix the problem: >> >> 1. Just allow rvalue arguments to bind to ref parameters. (My preferred >> solution, though it will make the overloading rules slightly more >> complicated.) > > > I always thought the main concern was potential escaping refs to the rvalue, > which would be solvable by allowing rvalues to be bound to `scope ref` > params only. That'd be my preferred choice as well. I touched on this. It sounds reasonable at first glance. But the reasoning goes: 1. for safety reasons, we won't allow implicit stack temporaries to pass to functions that may escape them. 2. support passing temporaries only to 'scope ref' 3. realise that the implicit temp created by an rvalue is identical to the manually authored temp (or any other stack locals in the caller), you find that the only reasonable option to satisfy the alleged safety concern, is that all stack variables may never be passed to any function that's not 'scope ref'. Ie, you can't do this for safety reasons, because the exact same reasoning would exclude all stack variables ever from being passed the same way. The proposition is self-defeating. I was attracted to this idea for a short while, until I realised it was ridiculous. ;) >> 3. Continue to require code bloat (auto ref) or manual boilerplate >> (overloads). (I'm not very fond of this option, but it does not require a >> language change.) > > > While `auto ref` seems to have worked out surprisingly well for code written > in D, it doesn't solve the problem when interfacing with (many) external C++ > functions taking structs (represented in D by structs as well) by (mostly > const) ref. You're forced to declare lvalues for all of these args, > uglifying the code substantially. It's also not great for libraries. auto ref functions are template functions... what if I want to export that function? I can't. There are practical API and ABI reasons that you don't want every function to be a template function. Library authors carefully control which functions are 'real' functions and which are templates, for a wide variety of reasons. Binary libs are a thing. DLL's are a thing.
Re: rvalues -> ref (yup... again!)
Here is what I've used if I had to: https://p0nce.github.io/d-idioms/#Rvalue-references:-Understanding-auto-ref-and-then-not-using-it
Re: rvalues -> ref (yup... again!)
On Saturday, 24 March 2018 at 11:57:25 UTC, John Colvin wrote: I understand what you want, but I'm struggling to understand why it's such a huge deal. The reason you want to pass by reference is for performance, to avoid copying the data at the call boundary. It's pretty simple: float foo() { ... } ref float bar() { ... } void someFunc(ref float); someFunc(bar()); // ok float temp = foo(); someFunc(temp); // Have to create a temporary anyways for the function to work someFunc(foo()); // Compile error, need to use the hideous code above So there really isn't any performance penalty cause if you want to call that function you are going to have to create a temporary variable anyways, but now you can't just call the function in one line. It requires multiple lines and a temporary variable name. This becomes especially horrible for math libraries: void someFunc(ref Vector3) { ... } someFunc(a + b); // Can't do this Vector3 temp = a + b; someFunc(temp); // Have to do this So there's no performance penalty for what he is requesting, but allows for a cleaner syntax to do the exact same thing. So there are 2 cases: an lvalue needs to be passed, or an rvalue needs to be passed. 1. The address of the lvalue is passed. 2. The rvalue is copied to a local, then the address of that local is passed. So in the rvalue case, you're not getting the performance benefit of passing by reference, because you have to copy to a local anyway. What I would do in D currently to get the same performance and API: void foo(float[32] v) { foo(v); } void foo(ref float[32] v) { ... } or void foo()(auto ref float[32] v) { ... } What is so totally unacceptable about those solutions? I personally like the second because it scales better to multiple parameters. I know you have said it's not relevant and annoying that people bring up auto ref, but I dont' get how or why. It's exactly D's solution to the problem. It doesn't scale better, that's part of the problem: void foo()(auto ref MyType1, auto ref MyType2, auto ref MyType3, auto ref MyType4) { ... } The above has the possibility of generating 16 different functions that basically all do the exact same thing. It creates excessive bloat, and now what if you want to take the address of the function? You can't cause you have to choose between one of the 16 variants. This is template bloat at it's finest, except it isn't even doing anything useful. I can only imagine telling someone they have to code gen 16 identical functions just to be able to call a function without having to create useless temporary variables in the scope a function is being called in.
Re: rvalues -> ref (yup... again!)
On Saturday, 24 March 2018 at 17:30:35 UTC, Manu wrote: On 24 March 2018 at 04:57, John Colvin via Digitalmars-d wrote: On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: Forked from the x^^y thread... On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d wrote: On 3/23/2018 11:09 AM, Manu wrote: [...] Rvalue references are not trivial and can have major unintended consequences. They're a rather ugly feature in C++, with weirdities. I doubt D will ever have them. Can you please explain these 'weirdities'? What are said "major unintended consequences"? Explain how the situation if implemented would be any different than the workaround? This seems even simpler than the pow thing to me. Rewrite: func(f()); as: { auto __t0 = f(); func(__t0); } I understand what you want, but I'm struggling to understand why it's such a huge deal. Because it makes this kind of D code that interacts with C++ objectively worse than C++, and there's no reason for it. You can't say to someone who just frustrated-ly doubled their line count by manually introducing a bunch of temporaries in a tiny function that appears to do something so simple as 'call a function', that "oh yeah, isn't it cool that you can't just call your functions anymore! isn't D cool! we should switch to D right?" It's embarrassing. I've been put in the position where I have to try and 'explain' this feature quite some number of times... they usually just give me 'the look'â„¢; ya know, quietly wondering if I'm still sane, and all I end up with is someone who's about 95% less convinced that D is cool than they were 5 seconds beforehand. What pisses me off is that's such a pointless thing to happen, because this issue is so trivial! In my experience, people are evaluating how D will materially impact the exact same code they're already writing in C++. This is one of those ways that they will be materially impacted, and it's almost enough all on its own to cause people to dismiss the entire thing on the spot. Pretty much the best case at this phase is that the D code is exactly the same as C++. If we can trim off a few parens here and there (ufcs?), maybe remove some '::' operators (modules that don't suck), that's a huge win. The reason you want to pass by reference is for performance, to avoid copying the data at the call boundary. So there are 2 cases: an lvalue needs to be passed, or an rvalue needs to be passed. 1. The address of the lvalue is passed. 2. The rvalue is copied to a local, then the address of that local is passed. So in the rvalue case, you're not getting the performance benefit of passing by reference, because you have to copy to a local anyway. What I would do in D currently to get the same performance and API: void foo(float[32] v) { foo(v); } void foo(ref float[32] v) { ... } or void foo()(auto ref float[32] v) { ... } Can't be extern(C++), can't be virtual either (both is likely). I said before; you're talking about Scott Meyers 'universal references' as a language concept, and I'm just talking about calling a function. but I dont' get how or why. It's exactly D's solution to the problem. It doesn't solve the problem... it doesn't even address the problem. You're talking about a totally different thing >_< Auto ref allows the unnecessary copy to be avoided for lvalues and creates a temporary (as part of passing the value) for rvalues. It has downsides (virtual functions and extern(C++), but it does directly address the problem you're talking about, unless I have totally misunderstood you. Here is a small proof of concept I made to demonstrate how easy it seems to be to use `auto ref` to call a C++ virtual const& function without incurring any more copies than would happen with the same calls from C++. I'm sure it could be improved a lot, but does the basic concept match what you would need? // D source file: /// mix this in to your extern(C++) class with a list of the functions where you /// want to be able to pass rvalues to ref parameters. auto rValueRefCalls(Funcs ...)() { string ret; foreach (foo; Funcs) ret ~= `extern(D) void ` ~ __traits(identifier, foo) ~ `(Args ...)(auto ref Args args) if (__traits(compiles, (&this.` ~ __traits(identifier, foo) ~ `)(args))) { (&this.foo)(args); }`; return ret; } extern(C++) { class A { void foo(const ref int v); mixin(rValueRefCalls!foo); } A makeA(); } void main() { int x = 3; auto a = makeA(); a.foo(x); a.foo(3); } // C++ source file: #include class A { public: virtual void foo(const int& v) { printf("%d\n", v); } }; A *makeA() { return new A; }
Re: rvalues -> ref (yup... again!)
On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote: Auto ref allows the unnecessary copy to be avoided for lvalues and creates a temporary (as part of passing the value) for rvalues. It has downsides (virtual functions and extern(C++), but it does directly address the problem you're talking about, unless I have totally misunderstood you. You are trading syntax bloat for binary bloat. Having 4 parameters with auto ref can generate 16 different variants for the exact same function. If you are doing any sort of mathematical calculations you could cause a potential cache miss for calling the same exact function because one of your parameters didn't end up being the same type of value. You are causing all this bloat and potential slowdowns all to avoid having to do this: float value0 = a + b; float value1 = c + d; float value2 = e + f; someFunc(value0, value1, value2); That's an inherent flaw in design. You obviously agree that there is a problem, but how can you justify "auto ref" being a "good" solution to the problem? It's a horrible one, it causes excessive bloat and potential slowdowns just to get a cleaner readable syntax. That shouldn't be a solution that is deemed acceptable.
Re: rvalues -> ref (yup... again!)
On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote: Here is a small proof of concept I made to demonstrate how easy it seems to be to use `auto ref` to call a C++ virtual const& function without incurring any more copies than would happen with the same calls from C++. I'm sure it could be improved a lot, but does the basic concept match what you would need? // D source file: /// mix this in to your extern(C++) class with a list of the functions where you /// want to be able to pass rvalues to ref parameters. auto rValueRefCalls(Funcs ...)() { string ret; foreach (foo; Funcs) ret ~= `extern(D) void ` ~ __traits(identifier, foo) ~ `(Args ...)(auto ref Args args) if (__traits(compiles, (&this.` ~ __traits(identifier, foo) ~ `)(args))) { (&this.foo)(args); }`; return ret; } extern(C++) { class A { void foo(const ref int v); mixin(rValueRefCalls!foo); } A makeA(); } void main() { int x = 3; auto a = makeA(); a.foo(x); a.foo(3); } // C++ source file: #include class A { public: virtual void foo(const int& v) { printf("%d\n", v); } }; A *makeA() { return new A; } That isn't going to scale to the number of potential functions that are going to need this. It's going to cause slow compile speeds and create a bloated binary.
Re: rvalues -> ref (yup... again!)
On Sunday, March 25, 2018 00:34:38 Rubn via Digitalmars-d wrote: > On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote: > > Auto ref allows the unnecessary copy to be avoided for lvalues > > and creates a temporary (as part of passing the value) for > > rvalues. It has downsides (virtual functions and extern(C++), > > but it does directly address the problem you're talking about, > > unless I have totally misunderstood you. > > You are trading syntax bloat for binary bloat. Having 4 > parameters with auto ref can generate 16 different variants for > the exact same function. If you are doing any sort of > mathematical calculations you could cause a potential cache miss > for calling the same exact function because one of your > parameters didn't end up being the same type of value. You are > causing all this bloat and potential slowdowns all to avoid > having to do this: > > float value0 = a + b; > float value1 = c + d; > float value2 = e + f; > someFunc(value0, value1, value2); > > That's an inherent flaw in design. You obviously agree that there > is a problem, but how can you justify "auto ref" being a "good" > solution to the problem? It's a horrible one, it causes excessive > bloat and potential slowdowns just to get a cleaner readable > syntax. That shouldn't be a solution that is deemed acceptable. How good or bad it is depends on what you're doing and how many auto ref parameters there are in your code. If you're using it occasionally, it's not a problem at all, whereas if you're using it all over the place, you do get a lot of template bloat. Regardless, John's point was that auto ref solves the problem of being able to call a function with both lvalues and rvalues without copying lvalues, and he didn't understand why anyone was trying to argue that it doesn't do that. And I agree with him on that point. It does not help with virtual functions or extern(C++), and it creates a lot of template bloat if it's used heavily, so there are downsides to it, but it _does_ solve the basic problem of being able to call a function with both lvalues and rvalues without copying the lvalues. It just doesn't solve it in a way that everyone considers acceptable. auto ref also helps with forwarding refness, so it's useful for more than just avoiding copying lvalues, but the entire reason that auto ref was originally added to the language was to solve this exact problem. And maybe we need a different solution for some use cases (like virtual functions or cases where the template bloat is deemed unacceptable), but auto ref is in the language to solve this problem. So, much as at may make sense to argue that it's not a good solution to this problem, it really doesn't make sense to argue that it has nothing to do with it. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On 03/24/2018 03:03 AM, Jonathan M Davis wrote: On Saturday, March 24, 2018 01:37:10 Nick Sabalausky via Digitalmars-d wrote: Why require the callee's author to add boilerplate? Just do it for all ref params that are given an rvalue. Because if the point of the function accepting its argument by ref is because it's going to mutate the argument, then it makes no sense for it to accept rvalues, 1. That *isn't* always the core point of a function which takes a non-const ref argument. 2. It's the caller who decides whether or not the ref-result is needed, not the callee. 3. The frequent recurring complaints about no rvalue references are a testament that this is too common a use-case for the current "manually insert temporaries" workaround to be satisfactory. 4. If the whole point of disallowing rvalue references is to prevent accidents due to callers not realizing a param is being passed by ref (as it sounds like you're suggesting), then it's nothing but a broken half-solution, because it fails to provide that safety (and thus fails its own charter) when that same caller, once again not realizing a param is ref, passes an *lvalue* without expecting it to change. *If* the problem we want to solve here really is making sure a caller knows when a param is ref, we've failed, and the only way to actually accomplish it is the C# approach: Require callers to mark their ref args with "ref" and raise an error when they don't.
Re: rvalues -> ref (yup... again!)
On Sunday, 25 March 2018 at 01:43:43 UTC, Jonathan M Davis wrote: On Sunday, March 25, 2018 00:34:38 Rubn via Digitalmars-d wrote: On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote: > Auto ref allows the unnecessary copy to be avoided for > lvalues and creates a temporary (as part of passing the > value) for rvalues. It has downsides (virtual functions and > extern(C++), but it does directly address the problem you're > talking about, unless I have totally misunderstood you. You are trading syntax bloat for binary bloat. Having 4 parameters with auto ref can generate 16 different variants for the exact same function. If you are doing any sort of mathematical calculations you could cause a potential cache miss for calling the same exact function because one of your parameters didn't end up being the same type of value. You are causing all this bloat and potential slowdowns all to avoid having to do this: float value0 = a + b; float value1 = c + d; float value2 = e + f; someFunc(value0, value1, value2); That's an inherent flaw in design. You obviously agree that there is a problem, but how can you justify "auto ref" being a "good" solution to the problem? It's a horrible one, it causes excessive bloat and potential slowdowns just to get a cleaner readable syntax. That shouldn't be a solution that is deemed acceptable. How good or bad it is depends on what you're doing and how many auto ref parameters there are in your code. If you're using it occasionally, it's not a problem at all, whereas if you're using it all over the place, you do get a lot of template bloat. Regardless, John's point was that auto ref solves the problem of being able to call a function with both lvalues and rvalues without copying lvalues, and he didn't understand why anyone was trying to argue that it doesn't do that. And I agree with him on that point. It does not help with virtual functions or extern(C++), and it creates a lot of template bloat if it's used heavily, so there are downsides to it, but it _does_ solve the basic problem of being able to call a function with both lvalues and rvalues without copying the lvalues. It just doesn't solve it in a way that everyone considers acceptable. That's a horrible way to look at it. You can still technically drive a car with square wheels, it may not be a very comfortable or efficient ride, but it still __does__ solve the basic problem of moving the car. Because it works we shouldn't look at any other solution. auto ref also helps with forwarding refness, so it's useful for more than just avoiding copying lvalues, but the entire reason that auto ref was originally added to the language was to solve this exact problem. And maybe we need a different solution for some use cases (like virtual functions or cases where the template bloat is deemed unacceptable), but auto ref is in the language to solve this problem. So, much as at may make sense to argue that it's not a good solution to this problem, it really doesn't make sense to argue that it has nothing to do with it. I never said it has nothing to do with it. It's just a really horrible solution. If you are going to add an exception just for two minor cases, at that point you might as well start looking at adding rvalue references, cause that's what it is going to lead to. It'd provided better integration with C++ and provide an __acceptable__ solution to the problem (not one that solves the problem in a roundabout square-wheel-like way) that doesn't create runtime bloat and slowdown just to have clean readable syntax.
Re: rvalues -> ref (yup... again!)
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: Forked from the x^^y thread... There are too many replies on this thread, addressing all the comments would take forever and pollute the thread itself. So forgive me if I say something that was covered already by someone else. AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when calling C++ functions that take `const T&` (especially since that is common). I have not yet heard any other use for them. I'd be in favour of allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or have overloads for pass-by-value and pass-by-ref. I too, once a recent immigrant from the lands of C++, used to keep writing `ref const(T)`. I just pass by value now. C++ T&& (forwarding reference) -> D auto ref T C++ T&& (Rvalue reference) -> D T C++ const T& -> D T C++ T& -> D ref T If replacing const T& with T chafes, I understand. I used to feel that way too. It's _possible_ that would incur a penalty in copying/moving, but IME the cost is either 0, negligible, or negative (!). As mentioned above, if calling C++ code there's no choice about using T instead of const T&, so for pragmatic reasons that should be allowed. But only there, because... Can you please explain these 'weirdities'? What are said "major unintended consequences"? Rvalue references. They exist because of being able to bind temporaries to const T& in C++, which means there's no way of knowing if your const T& was originally a temporary or not. To disambiguate C++11 introduced the type system horror that are rvalue references (which also do double-duty in enabling perfect forwarding!). D doesn't have or need rvalue references _because_ of not allowing temporaries to bind to ref const(T). You get move semantics in D without the pain. That's a win in my book. Atila
Re: rvalues -> ref (yup... again!)
On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote: On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: Forked from the x^^y thread... There are too many replies on this thread, addressing all the comments would take forever and pollute the thread itself. So forgive me if I say something that was covered already by someone else. AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when calling C++ functions that take `const T&` (especially since that is common). I have not yet heard any other use for them. I'd be in favour of allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or have overloads for pass-by-value and pass-by-ref. I too, once a recent immigrant from the lands of C++, used to keep writing `ref const(T)`. I just pass by value now. C++ T&& (forwarding reference) -> D auto ref T C++ T&& (Rvalue reference) -> D T C++ const T& -> D T C++ T& -> D ref T If replacing const T& with T chafes, I understand. I used to feel that way too. It's _possible_ that would incur a penalty in copying/moving, but IME the cost is either 0, negligible, or negative (!). I'm tearing my remaining stubs of hair out trying to understand why memory copies (not talking about copy constructors) are needed when passing an rvalue to a non-ref function: https://stackoverflow.com/questions/49474685/passing-rvalue-to-non-ref-parameter-why-cant-the-compiler-elide-the-copy
Re: rvalues -> ref (yup... again!)
On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d wrote: > On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: >> >> Forked from the x^^y thread... >> > > > There are too many replies on this thread, addressing all the comments would > take forever and pollute the thread itself. So forgive me if I say something > that was covered already by someone else. > > AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when > calling C++ functions that take `const T&` (especially since that is > common). I have not yet heard any other use for them. I'd be in favour of > allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or > have overloads for pass-by-value and pass-by-ref. > I too, once a recent immigrant from the lands of C++, used to keep writing > `ref const(T)`. I just pass by value now. > > C++ T&& (forwarding reference) -> D auto ref T > C++ T&& (Rvalue reference) -> D T > C++ const T& -> D T Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user. > C++ T& -> D ref T I agree the other 3 cases are correct.
Re: rvalues -> ref (yup... again!)
On 26 March 2018 at 11:13, John Colvin via Digitalmars-d wrote: > On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote: >> >> On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: >>> >>> Forked from the x^^y thread... >>> >> >> >> There are too many replies on this thread, addressing all the comments >> would take forever and pollute the thread itself. So forgive me if I say >> something that was covered already by someone else. >> >> AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when >> calling C++ functions that take `const T&` (especially since that is >> common). I have not yet heard any other use for them. I'd be in favour of >> allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or >> have overloads for pass-by-value and pass-by-ref. >> I too, once a recent immigrant from the lands of C++, used to keep writing >> `ref const(T)`. I just pass by value now. >> >> C++ T&& (forwarding reference) -> D auto ref T >> C++ T&& (Rvalue reference) -> D T >> C++ const T& -> D T >> C++ T& -> D ref T >> >> If replacing const T& with T chafes, I understand. I used to feel that way >> too. It's _possible_ that would incur a penalty in copying/moving, but IME >> the cost is either 0, negligible, or negative (!). > > > I'm tearing my remaining stubs of hair out trying to understand why memory > copies (not talking about copy constructors) are needed when passing an > rvalue to a non-ref function: > https://stackoverflow.com/questions/49474685/passing-rvalue-to-non-ref-parameter-why-cant-the-compiler-elide-the-copy Passing rvalues to non-ref functions may elide a memory copy. Moves can be very efficient. But we're talking about ref functions right? Not not-ref functions...?
Re: rvalues -> ref (yup... again!)
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: Can you please explain these 'weirdities'? What are said "major unintended consequences"? Explain how the situation if implemented would be any different than the workaround? This seems even simpler than the pow thing to me. Rewrite: func(f()); as: { auto __t0 = f(); func(__t0); } How is that worse than the code you have to write: T temp = f(); T zero = 0; func(temp, zero); I feel like this example wasn't really concrete enough for me. I wrote a version below that I think made it a little clearer for myself. - import std.stdio : writeln; struct Foo { int data; } int foo(Foo x) { writeln("here"); return x.data; } int foo(ref Foo x) { writeln("there"); return x.data; } void main() { auto x = Foo(5); auto y = foo(x); writeln(y); auto z = foo(Foo(5)); writeln(z); }
Re: rvalues -> ref (yup... again!)
On 3/26/2018 12:24 PM, Manu wrote: On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d C++ const T& -> D T Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user. Copying does suck, I agree. Consider the following: void foo(T t) { foo(t); } <= add this overload void foo(ref T t) { ... } T aaa(); foo(aaa()); With inlining, I suspect we can get the compiler to not make any extra copies. It's not that different from NRVO. And as a marvy bonus, no weird semantic problems (as Atila mentioned).
Re: rvalues -> ref (yup... again!)
On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote: On 3/26/2018 12:24 PM, Manu wrote: On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d C++ const T& -> D T Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user. Copying does suck, I agree. Consider the following: void foo(T t) { foo(t); } <= add this overload void foo(ref T t) { ... } T aaa(); foo(aaa()); With inlining, I suspect we can get the compiler to not make any extra copies. It's not that different from NRVO. And as a marvy bonus, no weird semantic problems (as Atila mentioned). How do you add this overload for the following? void foo(ref T t) { ... } void function(ref int) func = &foo; int aaa(); func(aaa()); // err
Re: rvalues -> ref (yup... again!)
On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote: C++ T&& (Rvalue reference) -> D T Not really, in C++ it is an actual reference and you get to choose which function actually does the move. In D it just does the copy when passed to the function. So you can't do this in D. void bar(T&& t) { // actually move contents of T } void foo(T&& t) { bar(std::forward(t)); // Can't do this in D without making another actual copy cause it isn't a reference } If replacing const T& with T chafes, I understand. I used to feel that way too. It's _possible_ that would incur a penalty in copying/moving, but IME the cost is either 0, negligible, or negative (!). As mentioned above, if calling C++ code there's no choice about using T instead of const T&, so for pragmatic reasons that should be allowed. But only there, because... Can you please explain these 'weirdities'? What are said "major unintended consequences"? Rvalue references. They exist because of being able to bind temporaries to const T& in C++, which means there's no way of knowing if your const T& was originally a temporary or not. To disambiguate C++11 introduced the type system horror that are rvalue references (which also do double-duty in enabling perfect forwarding!). What's a concrete example that you would be required to know whether a const& is a temporary or not. I don't really see it otherwise. The solution everyone is saying to use as an alternative would be the literal case of how it would be implemented in the language. D doesn't have or need rvalue references _because_ of not allowing temporaries to bind to ref const(T). You get move semantics in D without the pain. That's a win in my book. Atila I've come across a few pains of such. It make be easier to use but it comes at a performance hit. In part binaries become huge because of how "init" is implemented. struct StaticArray(T, size_t capacity) { size_t length; T[capacity] values; } Copying the above structure copies unnecessary data for any move/copy operation. Eg when length = 0, it'll still copy everything. This includes initialization.
Re: rvalues -> ref (yup... again!)
On 26 March 2018 at 15:48, Walter Bright via Digitalmars-d wrote: > On 3/26/2018 12:24 PM, Manu wrote: >> >> On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d >>> >>> C++ const T& -> D T >> >> >> Yeah, no... T may be big. Copying a large thing sucks. Memory copying >> is the slowest thing computers can do. >> As an API author, exactly as in C++, you will make a judgement on a >> case-by-case basis on this matter. It may be by-value, it may be by >> const-ref. It depends on a bunch of things, and they are points for >> consideration by the API author, not the user. > > > Copying does suck, I agree. Consider the following: > > void foo(T t) { foo(t); } <= add this overload > void foo(ref T t) { ... } > T aaa(); > > foo(aaa()); > > With inlining, I suspect we can get the compiler to not make any extra > copies. It's not that different from NRVO. And as a marvy bonus, no weird > semantic problems (as Atila mentioned). It's a terrible experience to add 2^n overloads, just to abuse the parameter list (as in your example) to create the temporary in an implicit manner. The compiler can easily create the exact same temporary without requiring a bunch of overloads to be declared. What's the advantage of requiring that effort from the user, rather than just doing it at the call site? There's side effects to that hack too. Now there are overloads; so taking function pointers becomes awkward (might be important in some cases). If the interface has a binary boundary (static lib/dll) then what does that look like with respect to inlining? Where do the overloads go if the function is virtual? I already know the answers to these questions, but the point is, there's a whole lot more baggage introduced into the scene that just doesn't need to be there. So, while I agree that's an existing workaround, it kinda misses the point. This thread isn't about inlining, it's about NOT inlining. You don't use ref args unless you have a reason to, and that reason is likely to have friction with that particular work-around.
Re: rvalues -> ref (yup... again!)
On 26 March 2018 at 16:21, Rubn via Digitalmars-d wrote: > On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote: >> >> On 3/26/2018 12:24 PM, Manu wrote: >>> >>> On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d C++ const T& -> D T >>> >>> >>> Yeah, no... T may be big. Copying a large thing sucks. Memory copying >>> is the slowest thing computers can do. >>> As an API author, exactly as in C++, you will make a judgement on a >>> case-by-case basis on this matter. It may be by-value, it may be by >>> const-ref. It depends on a bunch of things, and they are points for >>> consideration by the API author, not the user. >> >> >> Copying does suck, I agree. Consider the following: >> >> void foo(T t) { foo(t); } <= add this overload >> void foo(ref T t) { ... } >> T aaa(); >> >> foo(aaa()); >> >> With inlining, I suspect we can get the compiler to not make any extra >> copies. It's not that different from NRVO. And as a marvy bonus, no weird >> semantic problems (as Atila mentioned). > > > How do you add this overload for the following? > > > void foo(ref T t) { ... } > > void function(ref int) func = &foo; > int aaa(); > > func(aaa()); // err Exactly.
Re: rvalues -> ref (yup... again!)
On 26 March 2018 at 19:25, Manu wrote: > On 26 March 2018 at 16:21, Rubn via Digitalmars-d > wrote: >> On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote: >>> >>> On 3/26/2018 12:24 PM, Manu wrote: On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d > > C++ const T& -> D T Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user. >>> >>> >>> Copying does suck, I agree. Consider the following: >>> >>> void foo(T t) { foo(t); } <= add this overload >>> void foo(ref T t) { ... } >>> T aaa(); >>> >>> foo(aaa()); >>> >>> With inlining, I suspect we can get the compiler to not make any extra >>> copies. It's not that different from NRVO. And as a marvy bonus, no weird >>> semantic problems (as Atila mentioned). >> >> >> How do you add this overload for the following? >> >> >> void foo(ref T t) { ... } >> >> void function(ref int) func = &foo; >> int aaa(); >> >> func(aaa()); // err > > Exactly. We're just kicking the can. And the only reason to do so is ideological, as far as I can tell. I want to hear an argument against... or any issue that's introduced by allowing the implicit temporary?
Re: rvalues -> ref (yup... again!)
On 26 March 2018 at 17:30, Rubn via Digitalmars-d wrote: > On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote: >> >> C++ T&& (Rvalue reference) -> D T > > > Not really, in C++ it is an actual reference and you get to choose which > function actually does the move. In D it just does the copy when passed to > the function. So you can't do this in D. > > void bar(T&& t) > { > // actually move contents of T > } > > void foo(T&& t) > { > bar(std::forward(t)); // Can't do this in D without making another > actual copy cause it isn't a reference > } You can still get there with D, if you assume the forwarding function is simple enough to inline (and the code is public). I was cautiously concerned about this for a long time, but I've never made a noise about it. I've digested and resolved that's okay though :) There are very few cases where you will fail to achieve equivalent efficiency, and I think D's amazingly simplified move semantics more than compensate for any conceivable loss. >> If replacing const T& with T chafes, I understand. I used to feel that way >> too. It's _possible_ that would incur a penalty in copying/moving, but IME >> the cost is either 0, negligible, or negative (!). >> >> As mentioned above, if calling C++ code there's no choice about using T >> instead of const T&, so for pragmatic reasons that should be allowed. But >> only there, because... >> >>> Can you please explain these 'weirdities'? >>> What are said "major unintended consequences"? >> >> >> Rvalue references. They exist because of being able to bind temporaries to >> const T& in C++, which means there's no way of knowing if your const T& was >> originally a temporary or not. To disambiguate C++11 introduced the type >> system horror that are rvalue references (which also do double-duty in >> enabling perfect forwarding!). > > > What's a concrete example that you would be required to know whether a > const& is a temporary or not. I don't really see it otherwise. The solution > everyone is saying to use as an alternative would be the literal case of how > it would be implemented in the language. He's trying to say that C++ introduced rvalue references because normal references weren't able to allow for move semantics to exist. It's a red-herring. D already has move semantics, they work well, and they're not on trial here. In C++'s case, it's not that references were deficient at being references that C++ needed rval-references, it's that references were deficient at being move-able. That is not a problem that exists in D. It's fine for references to just be references in D. We're not struggling to make references move-able in D, that's not a thing, we already have move semantics. Any extension of this conversation about references into C++ rvalue-references (T&&) and or move-semantics are red-herrings. There's no such problem in D that needs to be resolved, and the existing solution is excellent.
Re: rvalues -> ref (yup... again!)
On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote: On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d wrote: On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: Forked from the x^^y thread... C++ T&& (forwarding reference) -> D auto ref T C++ T&& (Rvalue reference) -> D T C++ const T& -> D T Yeah, no... T may be big. Copying a large thing sucks. Memory copying is the slowest thing computers can do. That's _if_ T is big and _if_ it even gets copied, the combination of which I think happens very rarely. When that happens I think that the temporary isn't a big deal. This code: struct Foo { int[1024] data; } int byValue(Foo f) { return f.data[42]; } Generates this assembly (ldc2 -O3, clang does the same for C++): <_D3foo7byValueFSQo3FooZi>: 0: 8b 84 24 b0 00 00 00moveax,DWORD PTR [rsp+0xb0] 7: c3 ret And I wrote a type for which a move and a copy are the same on purpose: in "real life" it's more likely that the memory will be dynamically allocated, probably held in a slice, and moved instead. Are there cases in which there will be an expensive copy? Yes, then pass by ref/pointer. But measure first, and prefer to pass by value by default. It's different in C++ - stick a std::vector or std::string in your struct and passing by value (usually) copies the dynamically allocated memory. In D it moves. As an API author, exactly as in C++, you will make a judgement on a case-by-case basis on this matter. It may be by-value, it may be by const-ref. It depends on a bunch of things, and they are points for consideration by the API author, not the user. You can still do that in D. There well may be a reason to pass by const ref. I'm just saying that there aren't that many, and in that in those rare cases a temporary is fine. Especially if the alternative are rvalue references. Atila
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 00:30:24 UTC, Rubn wrote: On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote: C++ T&& (Rvalue reference) -> D T Not really, in C++ it is an actual reference and you get to choose which function actually does the move. In D it just does the copy when passed to the function. It doesn't copy. So you can't do this in D. void bar(T&& t) { // actually move contents of T } void foo(T&& t) { bar(std::forward(t)); // Can't do this in D without making another actual copy cause it isn't a reference } You can most definitely do this in D: void bar(T)(auto ref T t) { // T is a ref for lvalues, by value for rvalues } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } More to the point: import std.stdio; struct Foo { ubyte[] data; this(int n) { writeln("ctor n = ", n); data.length = n; } this(this) { writeln("postBlit n = ", data.length); data = data.dup; } } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } void bar(T)(auto ref T t) { writeln("bar: ", t.data[0]); } void main() { bar(Foo(10)); auto f = Foo(5); f.data[0] = 42; bar(f); } The output is: ctor n = 10 bar: 0 ctor n = 5 bar: 42 Notice the lack of "postBlit" in the output. No copies were made. In D, by value *does not* mean copy. And given that, contrary to C++, the compiler doesn't write the postBlit constructor for you, you'd only ever get copies if you implemented it yourself! What's a concrete example that you would be required to know whether a const& is a temporary or not. To know whether or not you can move instead of copy. If it's a temporary, you can move. If it's not, you have to copy. Since temporaries bind to const T& in C++, you might have a temporary, or you might have an lvalue. Since you don't know, you have to copy. To support move semantics, C++ got T&&, which lvalues can't bind to. So if you have a T&&, you know it's about to go away and a move is possible. In D, if it's ref then it can't be a temporary. If it's a value then it can, and it gets moved. I've come across a few pains of such. It make be easier to use but it comes at a performance hit. In part binaries become huge because of how "init" is implemented. struct StaticArray(T, size_t capacity) { size_t length; T[capacity] values; } Copying the above structure copies unnecessary data for any move/copy operation. Eg when length = 0, it'll still copy everything. This includes initialization. This is that rare type for which moving is the same as copying. In that case (assuming it gets copied, see my reply to Manu), pass by ref. You won't be able to pass in temporaries, but I think that's a small price to pay for not having rvalue references. In this case specifically, I don't know why you wouldn't just slice it when passing to functions. Atila
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 02:41:12 UTC, Manu wrote: He's trying to say that C++ introduced rvalue references because normal references weren't able to allow for move semantics to exist. It's a red-herring. D already has move semantics, they work well, and they're not on trial here. In C++'s case, it's not that references were deficient at being references that C++ needed rval-references, it's that references were deficient at being move-able. There were deficient at being moveable because temporaries can bind to const T&. That is not a problem that exists in D. Because temporaries can't bind to ref const(T). It's fine for references to just be references in D. We're not struggling to make references move-able in D, that's not a thing, we already have move semantics. Any extension of this conversation about references into C++ rvalue-references (T&&) and or move-semantics are red-herrings. There's no such problem in D that needs to be resolved, and the existing solution is excellent. If I'm reading you correctly (which I might not), you seem to be saying that there's a way forward in which: 1) D's move semantics aren't affected 2) No rvalue references are introduced 3) Temporaries can bind to ref const(T) I'd love to know what that would look like. Atila
Re: rvalues -> ref (yup... again!)
On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d wrote: > On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote: >> >> On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d >> wrote: >>> >>> On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: Forked from the x^^y thread... >>> >>> >>> >>> C++ T&& (forwarding reference) -> D auto ref T >>> C++ T&& (Rvalue reference) -> D T >>> C++ const T& -> D T >> >> >> Yeah, no... T may be big. Copying a large thing sucks. Memory copying >> is the slowest thing computers can do. > > > That's _if_ T is big and _if_ it even gets copied, You've just described the exact anatomy of a ref function! You wouldn't write a function to receive T by ref UNLESS T was both big, and the function probably won't inline (therefore definitely copy), and that condition will be triggered by any of the list of reasons I've said a bunch of times (extern, dll, lib, virtual, etc). People don't just love writing ref (well, some people might), but they use it deliberately, typically in user-facing boundary API's for these exact reasons. > the combination of which > I think happens very rarely. When that happens I think that the temporary > isn't a big deal. This code: > > struct Foo { int[1024] data; } > int byValue(Foo f) { return f.data[42]; } > > Generates this assembly (ldc2 -O3, clang does the same for C++): > > <_D3foo7byValueFSQo3FooZi>: >0: 8b 84 24 b0 00 00 00moveax,DWORD PTR [rsp+0xb0] >7: c3 ret > > And I wrote a type for which a move and a copy are the same on purpose: in > "real life" it's more likely that the memory will be dynamically allocated, > probably held in a slice, and moved instead. > > Are there cases in which there will be an expensive copy? Yes, then pass by > ref/pointer. But measure first, and prefer to pass by value by default. Right. I'm talking about deliberate use of ref... Or *existing* (deliberate) use of ref, as is the case in almost all my my cases. The code already exists. As you assess, use of ref in D is fairly rare. I've been looking for cases where other people are inconvenienced by this... hard to find. I suspect there are reasons for this. One of them is that this inconvenience suppresses it; ie, you will choose not to use a ref even when you might prefer to. Others include the fact that extern(C++) isn't super popular, DLL's aren't popular, closed-source distributed code is non popular, OOP is not popular, etc. > It's different in C++ - stick a std::vector or std::string in your struct > and passing by value (usually) copies the dynamically allocated memory. In D > it moves. Only if you ARE moving, and not copying. D must deep copy too if you actually copy. Your example assumes C++ doesn't have a move constructor. D has implicit move semantics, so you can only make an equivalent comparison where C++ also defines the move constructor so the move case doesn't pollute the ref comparison. Also, irrespective of whether move semantics are performed (eliding potential deep copying, as in your example), the binary memcpy still has to be performed when handling values by-val, unless RVO (we're not talking about return values), or inlining is able to eliminate it. Also, we're not talking about move semantics! >> As an API author, exactly as in C++, you will make a judgement on a >> case-by-case basis on this matter. It may be by-value, it may be by >> const-ref. It depends on a bunch of things, and they are points for >> consideration by the API author, not the user. > > > You can still do that in D. There well may be a reason to pass by const ref. > I'm just saying that there aren't that many, and in that in those rare cases > a temporary is fine. Especially if the alternative are rvalue references. We're not talking about rvalue references, we're talking about normal references >_< >> He's trying to say that C++ introduced rvalue references because normal >> references weren't able to allow for move semantics to exist. It's a >> red-herring. D already has move semantics, they work well, and they're not >> on trial here. >> >> In C++'s case, it's not that references were deficient at being >> references that C++ needed rval-references, it's that references were >> deficient at being move-able. > > > There were deficient at being moveable because temporaries can bind to const > T&. ... what? That's just not true at all. If temporaries couldn't bind to C++ ref, then you *definitely* wouldn't be able to move it, because you can guarantee that someone else owns the reference. You can't move references, under any circumstances, in either language... and that's actually the whole point of references. rvalue references were introduced in C++ to capture the calls with rvalues into a separate function call, exactly the same way as the by-value overload will catch the rvalues in D (and perform an implicit move). It was impossible for C++ to implement D's implicit mov
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 07:33:12 UTC, Atila Neves wrote: On Tuesday, 27 March 2018 at 00:30:24 UTC, Rubn wrote: On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote: C++ T&& (Rvalue reference) -> D T Not really, in C++ it is an actual reference and you get to choose which function actually does the move. In D it just does the copy when passed to the function. It doesn't copy. It copies the memory, so it does 2 memcpy's in the sense where as C++ only calls its move constructor once. So you can't do this in D. void bar(T&& t) { // actually move contents of T } void foo(T&& t) { bar(std::forward(t)); // Can't do this in D without making another actual copy cause it isn't a reference } You can most definitely do this in D: void bar(T)(auto ref T t) { // T is a ref for lvalues, by value for rvalues } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } More to the point: import std.stdio; struct Foo { ubyte[] data; this(int n) { writeln("ctor n = ", n); data.length = n; } this(this) { writeln("postBlit n = ", data.length); data = data.dup; } } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } void bar(T)(auto ref T t) { writeln("bar: ", t.data[0]); } void main() { bar(Foo(10)); auto f = Foo(5); f.data[0] = 42; bar(f); } The output is: ctor n = 10 bar: 0 ctor n = 5 bar: 42 Notice the lack of "postBlit" in the output. No copies were made. In D, by value *does not* mean copy. And given that, contrary to C++, the compiler doesn't write the postBlit constructor for you, you'd only ever get copies if you implemented it yourself! Well for starters your code is wrong. You are calling bar() instead of foo(). D has hidden implementation details, from your perspective it looks like it isn't doing any copying from the high-level viewpoint, but from the low-level viewpoint your object has been copied (moved memory) multiple times. struct Foo { ubyte[1024] data; } void foo(T)(auto ref T t) { import std.functional: forward; bar(forward!t); } Foo gfoo; void bar(T)(auto ref T t) { import std.algorithm.mutation : move; move(t, gfoo); } void main() { foo(Foo(10)); } _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv: push rbp mov rbp, rsp sub rsp, 3104 lea rax, [rbp + 16] lea rdi, [rbp - 2048] lea rcx, [rbp - 1024] mov edx, 1024 mov rsi, rcx mov qword ptr [rbp - 2056], rdi mov rdi, rsi mov rsi, rax mov qword ptr [rbp - 2064], rcx call memcpy@PLT<- hidden copy Then the other copy is in move() to gfoo. That hidden copy will happen for every additional function call you try to pass Foo through. What's a concrete example that you would be required to know whether a const& is a temporary or not. To know whether or not you can move instead of copy. If it's a temporary, you can move. If it's not, you have to copy. Since temporaries bind to const T& in C++, you might have a temporary, or you might have an lvalue. Since you don't know, you have to copy. To support move semantics, C++ got T&&, which lvalues can't bind to. So if you have a T&&, you know it's about to go away and a move is possible. In D, if it's ref then it can't be a temporary. If it's a value then it can, and it gets moved. D already has move semantics, an easy solution to this is to just use another keyword. It doesn't have to bind to const ref to get what is desired: // what was suggested in the original DIP, since scope is being used for something else now void foo(@temp ref value) { } Now you don't have this problem. You only get this behavior when you basically say you don't care whether it is a temporary or not. So what's your problem with it now ? I've come across a few pains of such. It make be easier to use but it comes at a performance hit. In part binaries become huge because of how "init" is implemented. struct StaticArray(T, size_t capacity) { size_t length; T[capacity] values; } Copying the above structure copies unnecessary data for any move/copy operation. Eg when length = 0, it'll still copy everything. This includes initialization. This is that rare type for which moving is the same as copying. In that case (assuming it gets copied, see my reply to Manu), pass by ref. You won't be able to pass in temporaries, but I think that's a small price to pay for not having rvalue references. In this case specifically, I don't know why you wouldn't just slice it when passing to functions. Atila This wasn't an example for rvalue references. This was an example illustrating the negative results of the current "pain-free" system. I have a 100 mb binary, 90 mb of that come from a single structure. I mean sure you don't really have to worry about implementing move constructors and such but it is far from being pain free, that'
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote: That's exactly what I've been saying. For like, 9 years.. It looks like this: https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md (contribution appreciated) I've followed this thread since it was made as this has been one of the very few disappointments of the language for me. I only tend to write game code and use D for hobbyist projects whilst using C++ full-time as a junior at a small games company. Even if I take the attitude that I should use D as it is intended, instead of trying to write C++ by using D, it always felt unnecessarily obstructive to require me to make a temporary variable to avoid copying something simple like a vector or a matrix. It feels very restrictive when trying to express mathematical calculations in a concise manner. Thanks for writing that DIP, you have covered everything I would love to see in great detail with good examples! I honestly couldn't think of anything more that could be added.
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 15:50:37 UTC, Atila Neves wrote: It's fine for references to just be references in D. We're not struggling to make references move-able in D, that's not a thing, we already have move semantics. Any extension of this conversation about references into C++ rvalue-references (T&&) and or move-semantics are red-herrings. There's no such problem in D that needs to be resolved, and the existing solution is excellent. If I'm reading you correctly (which I might not), you seem to be saying that there's a way forward in which: 1) D's move semantics aren't affected 2) No rvalue references are introduced 3) Temporaries can bind to ref const(T) I'd love to know what that would look like. Atila Well currently if you only have this implemented: void foo(const ref Type); Type temp = Type(10); foo(temp); Where the hell are you going to do your move semantics? You can't do it anyways currently, it's completely meaningless cause you can't. void foo(Type); void foo(const ref Type); foo(Type(10)); Now we have move semantics with an additional definition. With the proposed change, nothing about that would change. A temporary is only passed to a const ref as a "last resort". If it can do a move instead, it will do the move. The only change that is desired is to make the first sample code above have nicer syntax. That's it, like in the first example you don't care about it being a temporary or not.
Re: rvalues -> ref (yup... again!)
On Tue, Mar 27, 2018 at 08:25:36PM +, Rubn via Digitalmars-d wrote: [...] > _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv: > push rbp > mov rbp, rsp > sub rsp, 3104 > lea rax, [rbp + 16] > lea rdi, [rbp - 2048] > lea rcx, [rbp - 1024] > mov edx, 1024 > mov rsi, rcx > mov qword ptr [rbp - 2056], rdi > mov rdi, rsi > mov rsi, rax > mov qword ptr [rbp - 2064], rcx > call memcpy@PLT<- hidden copy [...] Is this generated by dmd, or gdc/ldc? Generally, when it comes to performance issues, I don't even bother looking at dmd-generated code anymore. If the extra copying is still happening with gdc -O2 / ldc -O, then you have a point. Otherwise, it doesn't really say very much. T -- People tell me that I'm skeptical, but I don't believe them.
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 20:38:35 UTC, H. S. Teoh wrote: On Tue, Mar 27, 2018 at 08:25:36PM +, Rubn via Digitalmars-d wrote: [...] _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv: push rbp mov rbp, rsp sub rsp, 3104 lea rax, [rbp + 16] lea rdi, [rbp - 2048] lea rcx, [rbp - 1024] mov edx, 1024 mov rsi, rcx mov qword ptr [rbp - 2056], rdi mov rdi, rsi mov rsi, rax mov qword ptr [rbp - 2064], rcx call memcpy@PLT<- hidden copy [...] Is this generated by dmd, or gdc/ldc? Generally, when it comes to performance issues, I don't even bother looking at dmd-generated code anymore. If the extra copying is still happening with gdc -O2 / ldc -O, then you have a point. Otherwise, it doesn't really say very much. T It happens with LDC too, not sure how it would be able to know to do any kind of optimization like that unless it was able to inline every single function called into one function and be able to do optimize it from there. I don't imagine that'll be likely though.
Re: rvalues -> ref (yup... again!)
On Tue, Mar 27, 2018 at 09:52:25PM +, Rubn via Digitalmars-d wrote: > On Tuesday, 27 March 2018 at 20:38:35 UTC, H. S. Teoh wrote: > > On Tue, Mar 27, 2018 at 08:25:36PM +, Rubn via Digitalmars-d wrote: > > [...] > > > _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv: > > > push rbp > > > mov rbp, rsp > > > sub rsp, 3104 > > > lea rax, [rbp + 16] > > > lea rdi, [rbp - 2048] > > > lea rcx, [rbp - 1024] > > > mov edx, 1024 > > > mov rsi, rcx > > > mov qword ptr [rbp - 2056], rdi > > > mov rdi, rsi > > > mov rsi, rax > > > mov qword ptr [rbp - 2064], rcx > > > call memcpy@PLT<- hidden copy > > [...] > > > > Is this generated by dmd, or gdc/ldc? > > > > Generally, when it comes to performance issues, I don't even bother > > looking at dmd-generated code anymore. If the extra copying is > > still happening with gdc -O2 / ldc -O, then you have a point. > > Otherwise, it doesn't really say very much. > > > > > > T > > It happens with LDC too, not sure how it would be able to know to do > any kind of optimization like that unless it was able to inline every > single function called into one function and be able to do optimize it > from there. I don't imagine that'll be likely though. You'll be surprised. Don't underestimate the power of modern optimizers. I've seen LDC do inlining that's so aggressive, that it essentially evaluated an entire series of function calls at compile-time (likely on the IR) and generated a single instruction to load the answer into the return register at runtime. :-D Of course, it still generated the individual functions, but those are never actually called at runtime. (On one occasion, this produced odd-looking "benchmark" results where the ldc executable computed the answer in exactly 0ms, whereas everyone else took a lot longer than that. :-D (Well, it was probably a few nanosecs while the CPU decoded and ran the instruction, but I don't think any benchmark could measure that!)) For your code example, you might want to look at the code generated for callers of the function, since when compiling individual functions in isolation, LDC is obligated to follow the ABI, which could include redundant copying. But if inlining was possible, it could generate very different code. T -- Dogs have owners ... cats have staff. -- Krista Casada
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote: It happens with LDC too, not sure how it would be able to know to do any kind of optimization like that unless it was able to inline every single function called into one function and be able to do optimize it from there. I don't imagine that'll be likely though. It does it in your code sample with `-O`, there's no call to bar and the foo() by-value arg is memcpy'd to the global. If you compile everything with LTO, your code and all 3rd-party libs as well as druntime/Phobos, LLVM is able to optimize the whole program as if it were inside a single gigantic 'object' file in LLVM bitcode IR, and is thus indeed theoretically able to inline *all* functions.
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 23:35:44 UTC, kinke wrote: On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote: It happens with LDC too, not sure how it would be able to know to do any kind of optimization like that unless it was able to inline every single function called into one function and be able to do optimize it from there. I don't imagine that'll be likely though. It does it in your code sample with `-O`, there's no call to bar and the foo() by-value arg is memcpy'd to the global. If you compile everything with LTO, your code and all 3rd-party libs as well as druntime/Phobos, LLVM is able to optimize the whole program as if it were inside a single gigantic 'object' file in LLVM bitcode IR, and is thus indeed theoretically able to inline *all* functions. A bit off topic now but anyways: Well that example I posted didn't do anything, so it would optimize it out quite easily. The entire function was excluded essentially. Just adding a few writeln it isn't able to remove the function entirely anymore and can't optimize it out. Idk if you want to try some different options but flto didn't do anything for it. https://godbolt.org/g/bLdpnm import std.stdio : writeln; struct Foo { ubyte[1024] data; this(int a) { data[0] = cast(ubyte)a; } } void foo(T)(auto ref T t) { import std.functional: forward; writeln(gfoo.data[0]); bar(forward!t); writeln(gfoo.data[0]); } __gshared Foo gfoo; void bar(T)(auto ref T t) { import std.algorithm.mutation : move; writeln(gfoo.data[0]); move(t, gfoo); } void main() { foo(Foo(10)); }
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 23:35:44 UTC, kinke wrote: On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote: It happens with LDC too, not sure how it would be able to know to do any kind of optimization like that unless it was able to inline every single function called into one function and be able to do optimize it from there. I don't imagine that'll be likely though. It does it in your code sample with `-O`, there's no call to bar and the foo() by-value arg is memcpy'd to the global. For reference: https://run.dlang.io/is/2vDEXP Note that main() boils down to a `memset(&gfoo, 10, 1024); return 0;`: _Dmain: .cfi_startproc pushq %rax .Lcfi0: .cfi_def_cfa_offset 16 data16 leaqonlineapp.Foo onlineapp.gfoo@TLSGD(%rip), %rdi data16 data16 rex64 callq __tls_get_addr@PLT movl$10, %esi movl$1024, %edx movq%rax, %rdi callq memset@PLT xorl%eax, %eax popq%rcx retq
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 23:59:09 UTC, Rubn wrote: Just adding a few writeln it isn't able to remove the function entirely anymore and can't optimize it out. Well writeln() here involves number -> string formatting, GC, I/O, template bloat... There are indeed superfluous memcpy's in your foo() there (although the forward and bar calls are still inlined), which after a quick glance seem to be LLVM optimizer shortcomings, the IR emitted by LDC looks fine. For an abitrary external function, it's all fine as it should be, boiling down to a single memcpy in foo() and a direct memset in main(): https://run.dlang.io/is/O1aeLK
Re: rvalues -> ref (yup... again!)
On Wednesday, 28 March 2018 at 00:56:29 UTC, kinke wrote: On Tuesday, 27 March 2018 at 23:59:09 UTC, Rubn wrote: Just adding a few writeln it isn't able to remove the function entirely anymore and can't optimize it out. Well writeln() here involves number -> string formatting, GC, I/O, template bloat... There are indeed superfluous memcpy's in your foo() there (although the forward and bar calls are still inlined), which after a quick glance seem to be LLVM optimizer shortcomings, the IR emitted by LDC looks fine. For an abitrary external function, it's all fine as it should be, boiling down to a single memcpy in foo() and a direct memset in main(): https://run.dlang.io/is/O1aeLK Well somethings wrong if writeln causes optimization to not occur, if that is the case then it'd be best to just use printf() instead. Anyways using small examples to show optimization is usually not what's going to happen in actual code. Functions are rarely that simple, and if adding a single writeln() to a call is enough to eliminate that optimization, I can only imagine what other little things do as well.
Re: rvalues -> ref (yup... again!)
On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: By contrast, people will NOT forgive the fact that they have to change: func(f(x), f(y), f(z)); to: T temp = f(x); T temp2 = f(y); T temp3 = f(z); func(temp, temp2, temp3); That's just hideous and in-defensible. A better story would be: func(f(x), f(y), f(z)); => func(x.f, y.f, z.f); Another workaround: auto r(T)(T a) { struct R { T val; } return R(a); } void f(in ref int p); int main() { f(1.r.val); return 0; }
Re: rvalues -> ref (yup... again!)
On 27.03.2018 20:14, Manu wrote: That's exactly what I've been saying. For like, 9 years.. It looks like this: https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md (contribution appreciated) As far as I can tell, it's completely benign, it just eliminates the annoying edge cases when interacting with functions that take arguments by ref. There's no spill-over affect anywhere that I'm aware of, and if you can find a single wart, I definitely want to know about it. ??? I've asked so many times for a technical destruction, nobody will present any opposition that is anything other than a rejection *in principle*. This is a holy war, not a technical one. That's extremely unfair. It is just a bad idea to overload D const for this purpose. Remove the "const" requirement and I'm on board.
Re: rvalues -> ref (yup... again!)
On 28.03.2018 13:34, Timon Gehr wrote: On 27.03.2018 20:14, Manu wrote: That's exactly what I've been saying. For like, 9 years.. It looks like this: https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md  (contribution appreciated) As far as I can tell, it's completely benign, it just eliminates the annoying edge cases when interacting with functions that take arguments by ref. There's no spill-over affect anywhere that I'm aware of, and if you can find a single wart, I definitely want to know about it. ??? >> I've asked so many times for a technical destruction, nobody will present any opposition that is anything other than a rejection *in principle*. This is a holy war, not a technical one. That's extremely unfair. It is just a bad idea to overload D const for this purpose. Remove the "const" requirement and I'm on board. "The proposal could be amended to accept mutable ref's depending on the value-judgement balancing these 2 use cases. Sticking with const requires no such value judgement to be made at this time, and it's much easier to relax the spec in the future with emergence of evidence to do so." Just get it right the first time. "const" is a serious API restriction, and it shouldn't be forced on anyone, even intermittently until they figure out that it is too restrictive (as well as viral).
Re: rvalues -> ref (yup... again!)
On 27.03.2018 20:14, Manu wrote: That's exactly what I've been saying. For like, 9 years.. It looks like this: https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md (contribution appreciated) "Temporary destruction Destruction of any temporaries occurs naturally at the end of the scope, as usual." This is actually unusual. --- import std.stdio; struct S{ ~this(){ writeln("destroyed!"); } } void foo(S s){} void main(){ foo(S()); writeln("end the scope"); } --- prints: destroyed! end the scope "Overload resolution ... This follows existing language rules. No change is proposed here." Yes, you _are_ proposing a change. Right now rvalues "prefer" the by-value overload because the ref overload _does not match at all_. Now you make it match both, so you are adding additional disambiguation rules. You need to be more explicit about those. Note that lvalues prefer the ref overload because the ref overload is more specialized. The new rule is the only instance where a less specialized overload is preferred. You also need to specify the interactions with matching levels (https://dlang.org/spec/function.html#function-overloading): E.g., your DIP is compatible with the following behavior: --- import std.stdio; struct S{} void fun(S){ writeln("A"); } void fun(ref const(S)){ writeln("B"); } void main(){ fun(S()); // calls A S s; fun(s); // calls A const(S) t; fun(t); // calls B fun(const(S)()); // calls B } --- The first example will cause friction when people try to add an explicit rvalue overload alongside a previous catch-it-all overload, the second example shows a breaking language change. You cannot "fix" the first example without introducing breaking language changes. The above code compiles and runs in current D. This just smells bad. Remove the "const" requirement.
Re: rvalues -> ref (yup... again!)
On 27.03.2018 22:25, Rubn wrote: D already has move semantics, an easy solution to this is to just use another keyword. It doesn't have to bind to const ref to get what is desired: // what was suggested in the original DIP, since scope is being used for something else now void foo(@temp ref value) { } Now you don't have this problem. You only get this behavior when you basically say you don't care whether it is a temporary or not. Another benefit of this solution is that the overload resolution rules are obvious. foo(@temp ref T value) is less specialized than both foo(T value) and foo(ref T value). @Manu: Consider this.
Re: rvalues -> ref (yup... again!)
On 28 Mar. 2018 4:35 am, "Timon Gehr via Digitalmars-d" < digitalmars-d@puremagic.com> wrote: On 27.03.2018 20:14, Manu wrote: > That's exactly what I've been saying. For like, 9 years.. > It looks like this: > https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx > -rval_to_ref.md > (contribution appreciated) > > As far as I can tell, it's completely benign, it just eliminates the > annoying edge cases when interacting with functions that take > arguments by ref. There's no spill-over affect anywhere that I'm aware > of, and if you can find a single wart, I definitely want to know about > it. > ??? I've asked so many times for a technical destruction, nobody will > present any opposition that is anything other than a rejection *in > principle*. This is a holy war, not a technical one. > That's extremely unfair. It is just a bad idea to overload D const for this purpose. Remove the "const" requirement and I'm on board. I discussed that in that document. I'm happy to remove const, but it requires a value judgement on the meaning of non-const in this case. It becomes controversial without const, but I'm personally happy to remove it if you can make The argument in favour. Can you give me some ideas where it would be useful?
Re: rvalues -> ref (yup... again!)
On 28 March 2018 at 05:22, Timon Gehr via Digitalmars-d wrote: > On 27.03.2018 20:14, Manu wrote: >> >> That's exactly what I've been saying. For like, 9 years.. >> It looks like this: >> >> https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md >> (contribution appreciated) > > > "Temporary destruction > Destruction of any temporaries occurs naturally at the end of the scope, as > usual." > > This is actually unusual. > > --- > import std.stdio; > struct S{ > ~this(){ > writeln("destroyed!"); > } > } > > void foo(S s){} > > void main(){ > foo(S()); > writeln("end the scope"); > } > --- > > prints: > > destroyed! > end the scope Right, exactly... So, what's wrong? > "Overload resolution > ... > This follows existing language rules. No change is proposed here." > > Yes, you _are_ proposing a change. Right now rvalues "prefer" the by-value > overload because the ref overload _does not match at all_. Now you make it > match both, so you are adding additional disambiguation rules. You need to > be more explicit about those. Oh right... yeah okay, I'll tweak the language. > Note that lvalues prefer the ref overload because the ref overload is more > specialized. The new rule is the only instance where a less specialized > overload is preferred. I've never heard any discussion involving the term 'specialised', or seen any definition where overloading prefers a "more specialised' version... is that a thing? Given: void f(int); void f(const(int)); f(10); That calls the 'int' one, but it could call either one... that's definitely not choosing a 'more specialised' match. > You also need to specify the interactions with matching levels > (https://dlang.org/spec/function.html#function-overloading): > > E.g., your DIP is compatible with the following behavior: > > --- > import std.stdio; > > struct S{} > void fun(S){ writeln("A"); } > void fun(ref const(S)){ writeln("B"); } > > void main(){ > fun(S()); // calls A > S s; > fun(s); // calls A > > const(S) t; > fun(t); // calls B > fun(const(S)()); // calls B > } > --- > > The first example will cause friction when people try to add an explicit > rvalue overload alongside a previous catch-it-all overload, the second > example shows a breaking language change. > > You cannot "fix" the first example without introducing breaking language > changes. The above code compiles and runs in current D. > > This just smells bad. Remove the "const" requirement. This is very compelling reason to remove the const.
Re: rvalues -> ref (yup... again!)
On 28 March 2018 at 05:38, Timon Gehr via Digitalmars-d wrote: > On 27.03.2018 22:25, Rubn wrote: >> >> >> D already has move semantics, an easy solution to this is to just use >> another keyword. It doesn't have to bind to const ref to get what is >> desired: >> >> // what was suggested in the original DIP, since scope is being used for >> something else now >> void foo(@temp ref value) >> { >> } >> >> Now you don't have this problem. You only get this behavior when you >> basically say you don't care whether it is a temporary or not. > > Another benefit of this solution is that the overload resolution rules are > obvious. foo(@temp ref T value) is less specialized than both foo(T value) > and foo(ref T value). > > @Manu: Consider this. This defeats the entire point to me. I want symmetrical calling code in all cases... the current edges are a massive pain in the arse. In the event of yet-another-attribute, then we just shift the set of edge cases onto that attribute instead, and it makes no practical difference in the end.
Re: rvalues -> ref (yup... again!)
On 28.03.2018 20:20, Manu wrote: On 28 March 2018 at 05:22, Timon Gehr via Digitalmars-d wrote: On 27.03.2018 20:14, Manu wrote: That's exactly what I've been saying. For like, 9 years.. It looks like this: https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md (contribution appreciated) "Temporary destruction Destruction of any temporaries occurs naturally at the end of the scope, as usual." This is actually unusual. ... ... So, what's wrong? ... In my example, the temporary is destroyed after/at the end of the function call, not at the end of the scope. Re-reading the DIP, I think you meant the right thing, but the wording is a bit confusing. Maybe just clarify that "the scope" is the one your rewrite introduces implicitly, or explicitly state that the lifetime ends at the end of the function call. "Overload resolution ... Note that lvalues prefer the ref overload because the ref overload is more specialized. The new rule is the only instance where a less specialized overload is preferred. I've never heard any discussion involving the term 'specialised', or seen any definition where overloading prefers a "more specialised' version... is that a thing? Given: void f(int); void f(const(int)); f(10); That calls the 'int' one, but it could call either one... The overload resolution rules in D have four different matching levels: - exact match - match with type qualifier conversion - match with general implicit conversion - no match The matching level for one overload is the minimal matching level for any argument. In your example f(int) matches exactly, but f(const(int)) matches with type qualifier conversion, therefore f(int) is chosen as it is the unique function that matches best. Only if two overloads match with the same best level is specialization used. An overload A is more specialized than another overload B if we can call B with all arguments with which we can call A. As it is possible to call a by-value function with an lvalue or an rvalue, but ref cannot be called with an rvalue, ref is more specialized. that's definitely not choosing a 'more specialised' match. ... Implicit conversions are ignored when checking for specialization so, yes, here both functions are equally specialized. However, f(int*) is more specialized than f(const(int)*): --- import std.stdio; void f(int* a){ writeln("A"); } void f(const(int)* b){ writeln("B"); } void main(){ f(new immutable(int)); // guess what this prints. :) } ---
Re: rvalues -> ref (yup... again!)
On Saturday, 24 March 2018 at 17:34:09 UTC, Manu wrote: You mutate a temporary that times out at the end of the statement... your statement is never assigned to anything, and has no effect. That is solved by having the ref function return its argument (so it can be chained): struct S; ref S modify(return ref S s); S(data).modify.writeln; This rvalue pattern would be disallowed if argument `s` was const. Why not make `modify` just return a copy then, maybe the optimizer could remove the copy? So that you can also use it with lvalues: S s; modify(s);
Re: rvalues -> ref (yup... again!)
Just to be sure it does not got los: You know that you can avoid the temp/copy if you add one method to your struct, yes? import std.stdio; struct Big { string name; float[1000] values; this(string name) { this.name = name; } @disable this(this); ref auto byRef() inout { return this; } } void foo(ref const Big b) { writeln(b.name); } void main() { Big b = Big("#1"); foo(b); foo(Big("#2").byRef); } That works like a charm and avoids any need for rvalue references. Just add it as mixin template in Phobos or something like that.
Re: rvalues -> ref (yup... again!)
On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote: Just to be sure it does not got los: You know that you can avoid the temp/copy if you add one method to your struct, yes? import std.stdio; struct Big { string name; float[1000] values; this(string name) { this.name = name; } @disable this(this); ref auto byRef() inout { return this; } } void foo(ref const Big b) { writeln(b.name); } void main() { Big b = Big("#1"); foo(b); foo(Big("#2").byRef); } That works like a charm and avoids any need for rvalue references. Just add it as mixin template in Phobos or something like that. Doesn't work with built-in types like float. Just adds bloat for operators like opBinary if you want that to be ref. foo((a.byRef + b.byRef * c.byRef).byRef) // vs foo(a + b * c); It's kind of funny all this talk about allowing temporaries to bind to refs being messy, yet you can already bind a temporary to a ref in a messy way using that.
Re: rvalues -> ref (yup... again!)
On Thursday, 29 March 2018 at 20:05:48 UTC, Rubn wrote: On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote: Just to be sure it does not got los: You know that you can avoid the temp/copy if you add one method to your struct, yes? import std.stdio; struct Big { string name; float[1000] values; this(string name) { this.name = name; } @disable this(this); ref auto byRef() inout { return this; } } void foo(ref const Big b) { writeln(b.name); } void main() { Big b = Big("#1"); foo(b); foo(Big("#2").byRef); } That works like a charm and avoids any need for rvalue references. Just add it as mixin template in Phobos or something like that. Doesn't work with built-in types like float. Why would you want to use a float as a rvalue reference? Just adds bloat for operators like opBinary if you want that to be ref. foo((a.byRef + b.byRef * c.byRef).byRef) // vs foo(a + b * c); It's kind of funny all this talk about allowing temporaries to bind to refs being messy, yet you can already bind a temporary to a ref in a messy way using that. Yeah, it is a bit messy. It is not perfect, but is does avoid any temp var! Let's look at a Vector2f example with the following opBinary: auto opBinary(string op)(ref const Vector2f v) { return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" ~ op ~ "v.y")); } void foo(ref const Vector2f v) { writeln(v.x, ':', v.y); } foo((Vector2f(2, 4).byRef + Vector2f(4, 6).byRef).byRef); Since opBinary needs to be a template, you can combine my solution with auto ref: auto opBinary(string op)(auto ref const Vector2f v) { return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" ~ op ~ "v.y")); } void foo(ref const Vector2f v) { writeln(v.x, ':', v.y); } foo((Vector2f(2, 4) + Vector2f(4, 6)).byRef); That is cleaner. :)
Re: rvalues -> ref (yup... again!)
On Thursday, 29 March 2018 at 20:22:47 UTC, Dgame wrote: On Thursday, 29 March 2018 at 20:05:48 UTC, Rubn wrote: On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote: Just to be sure it does not got los: You know that you can avoid the temp/copy if you add one method to your struct, yes? import std.stdio; struct Big { string name; float[1000] values; this(string name) { this.name = name; } @disable this(this); ref auto byRef() inout { return this; } } void foo(ref const Big b) { writeln(b.name); } void main() { Big b = Big("#1"); foo(b); foo(Big("#2").byRef); } That works like a charm and avoids any need for rvalue references. Just add it as mixin template in Phobos or something like that. Doesn't work with built-in types like float. Why would you want to use a float as a rvalue reference? In templates to avoid template bloat with auto ref. Just adds bloat for operators like opBinary if you want that to be ref. foo((a.byRef + b.byRef * c.byRef).byRef) // vs foo(a + b * c); It's kind of funny all this talk about allowing temporaries to bind to refs being messy, yet you can already bind a temporary to a ref in a messy way using that. Yeah, it is a bit messy. It is not perfect, but is does avoid any temp var! Let's look at a Vector2f example with the following opBinary: auto opBinary(string op)(ref const Vector2f v) { return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" ~ op ~ "v.y")); } void foo(ref const Vector2f v) { writeln(v.x, ':', v.y); } foo((Vector2f(2, 4).byRef + Vector2f(4, 6).byRef).byRef); Since opBinary needs to be a template, you can combine my solution with auto ref: auto opBinary(string op)(auto ref const Vector2f v) { return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" ~ op ~ "v.y")); } void foo(ref const Vector2f v) { writeln(v.x, ':', v.y); } foo((Vector2f(2, 4) + Vector2f(4, 6)).byRef); That is cleaner. :) Just adding more template bloat and it is still messy, it's not like we are trying to find a work around to a problem. A solution already exists, the cleaner syntax you'd get with rvalue references (along with better compatibility with C++ and other benefits) can't be beat.
Re: rvalues -> ref (yup... again!)
On 03/23/2018 09:06 PM, Jonathan M Davis wrote: My biggest concern in all of this is that I don't want to see ref start accepting rvalues as has been occasionally discussed. It needs to be clear when a function is accept an argument by ref because it's going to mutate the object and when it's accepting by ref because it wants to avoid a copy. That ship sailed ages ago: It's already unclear. If we want to fix that, we can fix it, but blocking rvalue ref does nothing for that cause.
Re: rvalues -> ref (yup... again!)
On 03/24/2018 12:03 AM, Manu wrote: Why aim for "it often works", when we want "it always works"? Forcing const upon people who want to pass rvalues by reference is just not good enough. It is bad language design. I think you need to re-read the whole thread, and understand what we're even talking about. Nobody wants to pass rvalues by mutable-ref... I do. The ban serves no useful purpose (at least not any purpose that D doesn't *already* fail at - like knowing at the callsite whether a param is intended to be mutated). And it would permit the "avoid a copy" benefits in a completely orthogonal way - *without* relying on the param fitting D's transitive const requirements.
Re: rvalues -> ref (yup... again!)
On Thursday, March 29, 2018 23:28:54 Nick Sabalausky via Digitalmars-d wrote: > On 03/23/2018 09:06 PM, Jonathan M Davis wrote: > > My biggest concern in all of this is that I don't want to see ref start > > accepting rvalues as has been occasionally discussed. It needs to be > > clear when a function is accept an argument by ref because it's going > > to mutate the object and when it's accepting by ref because it wants to > > avoid a copy. > That ship sailed ages ago: It's already unclear. If we want to fix that, > we can fix it, but blocking rvalue ref does nothing for that cause. Really? And how often does ref get used just to avoid copying? You can almost always look at a function, see that it accepts ref, and know that it's supposed to mutate the argument. Functions that want to avoid copying lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a number of folks would start using it all over the place to avoid copying. Right now, folks rarely use ref that way, because it becomes too annoying to call the function with an rvalue. So, while it might not be the case 100% of the time right now that ref is used with the purpose of mutating the argument, it almost always is. As such, you can pretty reliably look at a function signature and expect that if one of its parameters is ref, it's going to be mutating that argument. The function that accepts an argument by ref with no intention of mutating it is very much the exception, and I really don't want to see that change. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote: On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d wrote: On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote: On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d wrote: On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: That's _if_ T is big and _if_ it even gets copied, You've just described the exact anatomy of a ref function! You wouldn't write a function to receive T by ref UNLESS T was both big, and the function probably won't inline (therefore definitely copy), and that condition will be triggered by any of the list of reasons I've said a bunch of times (extern, dll, lib, virtual, etc). People don't just love writing ref (well, some people might), but they use it deliberately, typically in user-facing boundary API's for these exact reasons. I know. I was arguing that those cases are uncommon and the API pain is therefore not too big of an issue. I'm pretty sure that you feel it more because of what you write in D. Right. I'm talking about deliberate use of ref... Or *existing* (deliberate) use of ref, as is the case in almost all my my cases. The code already exists. Right, and I was assuming (perhaps incorrectly) that this existing code was C++, hence me being on board with binding rvalues to const ref there. Only if you ARE moving, and not copying. D must deep copy too if you actually copy. Your example assumes C++ doesn't have a move constructor. D has implicit move semantics, so you can only make an equivalent comparison where C++ also defines the move constructor so the move case doesn't pollute the ref comparison. I wasn't assuming the lack of a move constructor. What I was saying is that passing by value in C++ will usually mean a copy, whereas in D it usually means a move. Also, irrespective of whether move semantics are performed (eliding potential deep copying, as in your example), the binary memcpy still has to be performed when handling values by-val, unless RVO (we're not talking about return values), or inlining is able to eliminate it. Good point about memcpy. In C++'s case, it's not that references were deficient at being references that C++ needed rval-references, it's that references were deficient at being move-able. There were deficient at being moveable because temporaries can bind to const T&. ... what? That's just not true at all. If temporaries couldn't bind to C++ ref, then you *definitely* wouldn't be able to move it, because you can guarantee that someone else owns the reference. Precisely. That's how D works. rvalue references were introduced in C++ to capture the calls with rvalues into a separate function call, exactly the same way as the by-value overload will catch the rvalues in D (and perform an implicit move). Yes. It was impossible for C++ to implement D's implicit move semantics when receiving by-value for reasons that have nothing to do with references; C++ allows interior pointers which breaks implicit moving, C++ can overload default constructor which also breaks implicit moving (no implicit way to reset the state of the prior owner). I hadn't thought about the implications of interior pointers. Very good point. References have no interaction with move semantics. Even given interior pointers, I disagree. But again, we're not talking about move semantics here, we're just talking about references ;) See comment above ;) I'd love to know what that would look like. That's exactly what I've been saying. For like, 9 years.. It looks like this: https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md (contribution appreciated) I was unaware of this (or I forgot). After reading it I'm not sure of what corner cases might arise, but if I'm getting it right I think it could work. And as far as I can tell, it basically only affects me, because I do so much work against established C++ code! >_< That's entirely possible. I can use my fingers to count the number of times I've written `extern(C++)`. Atila
Re: rvalues -> ref (yup... again!)
On Wednesday, 28 March 2018 at 16:15:53 UTC, Manu wrote: I discussed that in that document. I'm happy to remove const, but it requires a value judgement on the meaning of non-const in this case. It becomes controversial without const, but I'm personally happy to remove it if you can make The argument in favour. Can you give me some ideas where it would be useful? doesn't make sense because the output would be immediately discarded; such a function call given an rvalue as argument likely represents an accidental mistake on the users part, and we can catch that invalid code. Most obvious: void Close(ref HANDLE h) { CloseHandle(h); h=INVALID_HANDLE; } If you want to take input argument, then of course mark it as input, but if not then not. Why the callee would need to care if the argument is rvalue or not? The only criticism against rvalue references I saw was when the reference outlives the temporary.
Re: rvalues -> ref (yup... again!)
On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d wrote: > On Thursday, March 29, 2018 23:28:54 Nick Sabalausky via Digitalmars-d > wrote: >> On 03/23/2018 09:06 PM, Jonathan M Davis wrote: >> > My biggest concern in all of this is that I don't want to see ref start >> > accepting rvalues as has been occasionally discussed. It needs to be >> > clear when a function is accept an argument by ref because it's going >> > to mutate the object and when it's accepting by ref because it wants to >> > avoid a copy. >> That ship sailed ages ago: It's already unclear. If we want to fix that, >> we can fix it, but blocking rvalue ref does nothing for that cause. > > Really? And how often does ref get used just to avoid copying? You can > almost always look at a function, see that it accepts ref, and know that > it's supposed to mutate the argument. Functions that want to avoid copying > lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a > number of folks would start using it all over the place to avoid copying. > Right now, folks rarely use ref that way, because it becomes too annoying to > call the function with an rvalue. So, while it might not be the case 100% of > the time right now that ref is used with the purpose of mutating the > argument, it almost always is. As such, you can pretty reliably look at a > function signature and expect that if one of its parameters is ref, it's > going to be mutating that argument. The function that accepts an argument by > ref with no intention of mutating it is very much the exception, and I > really don't want to see that change. Interesting. Just understand that you're trading that feeling for a suite of edge cases though. Accepting asymmetric calling rules is a pretty big cost to pay for that 'nice thought'. That idea also dismisses the existence of the set of cases where ref is genuinely useful/correct. Your sentiment effectively puts those use cases into the position of 2nd-class citizens. I understand your sentiment, but I think the cost is not balanced, or even particularly fair with respect to users in those niche groups :/
Re: rvalues -> ref (yup... again!)
On 30 March 2018 at 02:06, Atila Neves via Digitalmars-d wrote: > On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote: >> >> On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d >> wrote: >>> >>> On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote: On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d wrote: > > > On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote: >> >> > >>> That's _if_ T is big and _if_ it even gets copied, >> >> >> You've just described the exact anatomy of a ref function! >> You wouldn't write a function to receive T by ref UNLESS T was both >> big, and the function probably won't inline (therefore definitely >> copy), and that condition will be triggered by any of the list of >> reasons I've said a bunch of times (extern, dll, lib, virtual, etc). >> People don't just love writing ref (well, some people might), but they >> use it deliberately, typically in user-facing boundary API's for these >> exact reasons. > > > I know. I was arguing that those cases are uncommon and the API pain is > therefore not too big of an issue. I'm pretty sure that you feel it more > because of what you write in D. Totally. But I think it's uncommon at least in-part because of this inconvenience. It's at least a little bit circular. Even you're saying above "yeah, just do by-val, it's *probably* insignificant"... you only say that because you've allowed this inconvenience to change your behaviour. And I think that's to be expected. Other reasons that it's not common: extern(C++) is not common. OOP (virtuals) in D are fairly uncommon. We don't OOP much in D. Closed-source (binary lib) distribution is uncommon... possibly unheard of? (yet!) DLL's are still problematic onvarious fronts, and I don't think they enjoy anywhere near as much use as they deserve. Most of these aren't desirable... we'd like to see more commercial (closed source?) users, DLL's should be more popular than they are... and more extern(C++) means more people using a key development investment in D. So yeah, I agree it's relatively uncommon if you don't interact with one of the niches where it tends to be common! :P >> Right. I'm talking about deliberate use of ref... Or *existing* >> (deliberate) use of ref, as is the case in almost all my my cases. The >> code already exists. > > > Right, and I was assuming (perhaps incorrectly) that this existing code was > C++, hence me being on board with binding rvalues to const ref there. Right. But I'm not a fan is just rearranging the edge cases by limiting it to a different set of cases and not applying it generally. This also interacts with meta constructions, and while there are ANY cases where calling doesn't just work as usual and it needs special case handling, meta will still need static-if's to handle those cases. We're just changing the criteria for the 'if'. By allowing fully symmetric function calling rules, only then does the noisy case-handling logic disappear from meta. >> Only if you ARE moving, and not copying. D must deep copy too if you >> actually copy. >> Your example assumes C++ doesn't have a move constructor. D has >> implicit move semantics, so you can only make an equivalent comparison >> where C++ also defines the move constructor so the move case doesn't >> pollute the ref comparison. > > I wasn't assuming the lack of a move constructor. What I was saying is that > passing by value in C++ will usually mean a copy, whereas in D it usually > means a move. Right. But we're not talking about move's, we're talking about NOT-move's (ie, preventing copies by passing by ref) ;) Despite the appearance that we might be talking about rvalues, we're actually talking about passing lvalues by ref... that's WHY you write a function to accept its args by ref; to prevent deep copies when passing lvalues. The issue is, it's super-common to call functions with rvalues, and the edge cases created by using ref are annoying and asymmetric, hence such functions should receive rvalues too. In C++'s case, it's not that references were deficient at being references that C++ needed rval-references, it's that references were deficient at being move-able. >>> >>> There were deficient at being moveable because temporaries can bind to >>> const T&. >> >> ... what? That's just not true at all. >> If temporaries couldn't bind to C++ ref, then you *definitely* >> wouldn't be able to move it, because you can guarantee that someone >> else owns the reference. > > Precisely. That's how D works. Sorry, I'm lost now. I don't understand your initial point. You created a relationship between ref's accepting rvalues, and the reason that C++ introduced rvalue-references. No such relationship exists... and I was trying to show that your reasoning was inverted. >> rvalue references were introduced in C++ to capture the calls with >> rvalues into a separate function call, exactly the same way as the >> by-value overload will catch th
Re: rvalues -> ref (yup... again!)
On 30 March 2018 at 02:47, Kagamin via Digitalmars-d wrote: > > The only criticism against rvalue references I saw was when the > reference outlives the temporary. That's the only *technical* criticism I've ever heard too. Fortunately, D now has mechanisms for preventing that; D's @safety story with respect to reference lifetime seems to be solid now. 'return ref', and enforcing ref pointers don't escape the callee cleared to the road to allow accepting temporaries safety.
Re: rvalues -> ref (yup... again!)
On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote: > On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d > > wrote: > > On Thursday, March 29, 2018 23:28:54 Nick Sabalausky via Digitalmars-d > > > > wrote: > >> On 03/23/2018 09:06 PM, Jonathan M Davis wrote: > >> > My biggest concern in all of this is that I don't want to see ref > >> > start > >> > accepting rvalues as has been occasionally discussed. It needs to be > >> > clear when a function is accept an argument by ref because it's going > >> > to mutate the object and when it's accepting by ref because it wants > >> > to > >> > avoid a copy. > >> > >> That ship sailed ages ago: It's already unclear. If we want to fix > >> that, > >> we can fix it, but blocking rvalue ref does nothing for that cause. > > > > Really? And how often does ref get used just to avoid copying? You can > > almost always look at a function, see that it accepts ref, and know that > > it's supposed to mutate the argument. Functions that want to avoid > > copying lvalues usually use auto ref, not ref, whereas if ref accepted > > rvalues, a number of folks would start using it all over the place to > > avoid copying. Right now, folks rarely use ref that way, because it > > becomes too annoying to call the function with an rvalue. So, while it > > might not be the case 100% of the time right now that ref is used with > > the purpose of mutating the argument, it almost always is. As such, you > > can pretty reliably look at a function signature and expect that if one > > of its parameters is ref, it's going to be mutating that argument. The > > function that accepts an argument by ref with no intention of mutating > > it is very much the exception, and I really don't want to see that > > change. > > Interesting. Just understand that you're trading that feeling for a > suite of edge cases though. Accepting asymmetric calling rules is a > pretty big cost to pay for that 'nice thought'. > That idea also dismisses the existence of the set of cases where ref > is genuinely useful/correct. Your sentiment effectively puts those use > cases into the position of 2nd-class citizens. > I understand your sentiment, but I think the cost is not balanced, or > even particularly fair with respect to users in those niche groups :/ I'm not arguing against having a way to indicate that a parameter accepts both rvalues and lvalues and that the rvalues get copied to an invisible variable so that they can be passed as an lvalue. I'm arguing against simply making ref have that behavior. Assuming that the details of how that worked internally didn't involve problematic stuff like the rvalue reference stuff that Andrei and Walter are so against, having an attribute such as @rvalue to attach to a ref parameter to allow it to accept rvalues would be fine with me. I just don't want ref by itself to lose its current semantics, because that would dilute its meaning and increase its amiguity. I want to be able to look at a function signature, see ref without other qualifiers, and be reasonably certain that the function is supposed to be mutating that argument, whereas if ref by itself accepted rvalues, then we lose that. If an attribute were used to make it allow rvalues, then we wouldn't. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d wrote: > On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote: >> On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d >> >> wrote: >> > On Thursday, March 29, 2018 23:28:54 Nick Sabalausky via Digitalmars-d >> > >> > wrote: >> >> On 03/23/2018 09:06 PM, Jonathan M Davis wrote: >> >> > My biggest concern in all of this is that I don't want to see ref >> >> > start >> >> > accepting rvalues as has been occasionally discussed. It needs to be >> >> > clear when a function is accept an argument by ref because it's going >> >> > to mutate the object and when it's accepting by ref because it wants >> >> > to >> >> > avoid a copy. >> >> >> >> That ship sailed ages ago: It's already unclear. If we want to fix >> >> that, >> >> we can fix it, but blocking rvalue ref does nothing for that cause. >> > >> > Really? And how often does ref get used just to avoid copying? You can >> > almost always look at a function, see that it accepts ref, and know that >> > it's supposed to mutate the argument. Functions that want to avoid >> > copying lvalues usually use auto ref, not ref, whereas if ref accepted >> > rvalues, a number of folks would start using it all over the place to >> > avoid copying. Right now, folks rarely use ref that way, because it >> > becomes too annoying to call the function with an rvalue. So, while it >> > might not be the case 100% of the time right now that ref is used with >> > the purpose of mutating the argument, it almost always is. As such, you >> > can pretty reliably look at a function signature and expect that if one >> > of its parameters is ref, it's going to be mutating that argument. The >> > function that accepts an argument by ref with no intention of mutating >> > it is very much the exception, and I really don't want to see that >> > change. >> >> Interesting. Just understand that you're trading that feeling for a >> suite of edge cases though. Accepting asymmetric calling rules is a >> pretty big cost to pay for that 'nice thought'. >> That idea also dismisses the existence of the set of cases where ref >> is genuinely useful/correct. Your sentiment effectively puts those use >> cases into the position of 2nd-class citizens. >> I understand your sentiment, but I think the cost is not balanced, or >> even particularly fair with respect to users in those niche groups :/ > > I'm not arguing against having a way to indicate that a parameter accepts > both rvalues and lvalues and that the rvalues get copied to an invisible > variable so that they can be passed as an lvalue. I'm arguing against simply > making ref have that behavior. Assuming that the details of how that worked > internally didn't involve problematic stuff like the rvalue reference stuff > that Andrei and Walter are so against, having an attribute such as @rvalue > to attach to a ref parameter to allow it to accept rvalues would be fine > with me. I just don't want ref by itself to lose its current semantics, > because that would dilute its meaning and increase its amiguity. https://imgflip.com/i/27gjv9 > I want to be able to look at a function signature, see ref without other > qualifiers, and be reasonably certain that the function is supposed to be > mutating that argument, whereas if ref by itself accepted rvalues, then we > lose that. If an attribute were used to make it allow rvalues, then we > wouldn't. I don't know any reason why someone wouldn't attribute the argument 'const' if it doesn't intend to write to it. Likewise 'return ref' if it's going to be modified and returned. I'm not sure your concern is actually a thing...?
Re: rvalues -> ref (yup... again!)
On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote: > On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d > > I want to be able to look at a function signature, see ref without other > > qualifiers, and be reasonably certain that the function is supposed to > > be > > mutating that argument, whereas if ref by itself accepted rvalues, then > > we lose that. If an attribute were used to make it allow rvalues, then > > we wouldn't. > > I don't know any reason why someone wouldn't attribute the argument > 'const' if it doesn't intend to write to it. Likewise 'return ref' if > it's going to be modified and returned. > I'm not sure your concern is actually a thing...? const would do it, but given how restrictive const is in D, I don't see how it would be very reasonable to restrict passing rvalues to const. And honestly, I don't really want to encourage the use of const. It's fine if folks use it where it really works, and it's potentially even quite valuable there, but it seems like too often folks try to add const in places where it's just going to cause problems. It's particularly bad when folks try to add const to generic code - e.g. we recently had to revert a commit to std.random which worked with dynamic arrays but not other ranges because of const. And there have been other cases where folks have wanted to try to make stuff in Phobos "const-correct", which would cause problems. So, I'm not a big fan of the idea of doing anything that would make folks want to use const more. Now, if you can convince Walter and Andrei to allow const ref to accept rvalues, then fine. I think that that's definitely worse than an attribute specifically for that, given how limiting const is, but it wouldn't screw up normal ref in the process, which is what I'm most worried about here. So, I don't think that going with const would be the best solution to the problem, but it's far better than making ref in general accept rvalues. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d wrote: > On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote: >> On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d >> > I want to be able to look at a function signature, see ref without other >> > qualifiers, and be reasonably certain that the function is supposed to >> > be >> > mutating that argument, whereas if ref by itself accepted rvalues, then >> > we lose that. If an attribute were used to make it allow rvalues, then >> > we wouldn't. >> >> I don't know any reason why someone wouldn't attribute the argument >> 'const' if it doesn't intend to write to it. Likewise 'return ref' if >> it's going to be modified and returned. >> I'm not sure your concern is actually a thing...? > > const would do it, but given how restrictive const is in D, I don't see how > it would be very reasonable to restrict passing rvalues to const. I'm actually coming around to the idea of not restricting it to const... But what I'm trying to say is, the number of cases where your principle will feel violated should be as infrequent as the cases where const is insufficient for 'reasons' ;) As far as I know, in usages that I've ever encountered, I can't imagine a case where you would feel violated :) > And honestly, I don't really want to encourage the use of const. It's fine if > folks use it where it really works, and it's potentially even quite valuable > there, but it seems like too often folks try to add const in places where > it's just going to cause problems. It's particularly bad when folks try to > add const to generic code - e.g. we recently had to revert a commit to > std.random which worked with dynamic arrays but not other ranges because of > const. And there have been other cases where folks have wanted to try to > make stuff in Phobos "const-correct", which would cause problems. So, I'm > not a big fan of the idea of doing anything that would make folks want to > use const more. It's interesting... I recognise the general backlash against const. I personally just make myself a `struct Mutable(T) { ... }` which I use when I want to violate const ;) But if it turns out that const is useless, then we really need to reconsider the basic design of const >_< Like, what's the point? You're advocating active discouragement of const... why is there a feature which it's accepted is a loaded-gun? Handing it to people creates a high probability they'll shoot themselves in the feet. > Now, if you can convince Walter and Andrei to allow const ref to accept > rvalues, then fine. I think that that's definitely worse than an attribute > specifically for that, given how limiting const is, but it wouldn't screw up > normal ref in the process, which is what I'm most worried about here. So, I > don't think that going with const would be the best solution to the problem, > but it's far better than making ref in general accept rvalues. Useful examples using 'return ref' have been presented, which I find quite compelling too. That's an interesting case of non-const ref. Safe to say I'm not convincing anyone of anything in any way other than DIP form. But I'd like to understand your concern better. You say it's about scanning an API and understanding some details from it based on seeing 'ref' written there... how does a function accepting an rvalue interact with your visibility of the API? Like, your criticism is with respect to understanding the API at a glance... I don't understand how this proposal interferes with that in any way? Sadly, I don't think I'll be able to make it to DConf this year... which is probably a reason for rejoice of literally everybody attending! :P It would be nice to workshop it in person though.
Re: rvalues -> ref (yup... again!)
On 3/30/2018 6:03 PM, Manu wrote: Sadly, I don't think I'll be able to make it to DConf this year... :-( which is probably a reason for rejoice of literally everybody attending! :P You'll be missed. It would be nice to workshop it in person though.
Re: rvalues -> ref (yup... again!)
On Friday, March 30, 2018 18:03:49 Manu via Digitalmars-d wrote: > On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d > > And honestly, I don't really want to encourage the use of const. It's > > fine if folks use it where it really works, and it's potentially even > > quite valuable there, but it seems like too often folks try to add > > const in places where it's just going to cause problems. It's > > particularly bad when folks try to add const to generic code - e.g. we > > recently had to revert a commit to std.random which worked with dynamic > > arrays but not other ranges because of const. And there have been other > > cases where folks have wanted to try to make stuff in Phobos > > "const-correct", which would cause problems. So, I'm not a big fan of > > the idea of doing anything that would make folks want to use const > > more. > > It's interesting... I recognise the general backlash against const. I > personally just make myself a `struct Mutable(T) { ... }` which I use > when I want to violate const ;) Just don't do anything where you cast away const and then mutate. That violates the type system and results in undefined behavior. And depending on what the compiler chooses to do based on const, you could get some rather subtle and nasty bugs. > But if it turns out that const is useless, then we really need to > reconsider the basic design of const >_< > Like, what's the point? You're advocating active discouragement of > const... why is there a feature which it's accepted is a loaded-gun? > Handing it to people creates a high probability they'll shoot > themselves in the feet. const makes some sense when you want to write code that accepts both mutable and immutable arguments, and there are times where it works perfectly fine to use const. I see no problem with using const in such situations. The problem is that there are a _lot_ cases where const simply can't be used in D, because you need to mutate _something_ involved, even if it's not something that's part of the "logical" state of the object - e.g. in D, you can't put a mutex in a type and then have it protect anything in a const member function, because locking the mutex would require mutating it, and D provides no backdoors like C++'s mutable. What's const is actually const. Also, const tends to interact very badly with generic code. Whatever API you're duck typing has to include const as part of it, or you can't use const - e.g. ranges can't be const, because they have to be mutated to be iterated, and because the range API does not require that properties like front or empty be const, no generic code can assume that even those work with const (and for many ranges, they _can_'t be const - especially when one range wraps another). In general, it can be quite difficult to even do something like have a tail-const range over a container. Sometimes, const can be made to work in more complex situations, but often it can't, and when it can't, it's often a royal pain. For non-generic code that can clearly treat an object as fully const without needing any backdoors, const is just fine, and it may even help prevent bugs. But for a _lot_ of code - especially idiomatic D code that does a lot with templates and ranges - const simply doesn't work. So, const can be used on some level, but anyone who tries to be "const-correct" in D like many of us usually try to do in C++ is in for a world of frustration. If you haven't, I'd suggest that you read this article I recently wrote on the topic: http://jmdavisprog.com/articles/why-const-sucks.html And here's the newsgroup thread discussing it: https://forum.dlang.org/thread/mailman.581.1520247475.3374.digitalmars-d-annou...@puremagic.com > > Now, if you can convince Walter and Andrei to allow const ref to accept > > rvalues, then fine. I think that that's definitely worse than an > > attribute specifically for that, given how limiting const is, but it > > wouldn't screw up normal ref in the process, which is what I'm most > > worried about here. So, I don't think that going with const would be > > the best solution to the problem, but it's far better than making ref > > in general accept rvalues. > > Useful examples using 'return ref' have been presented, which I find > quite compelling too. That's an interesting case of non-const ref. > Safe to say I'm not convincing anyone of anything in any way other > than DIP form. Yes, at this point, convincing Walter or Andrei will require writing a DIP. A well-written proposal that clearly doesn't have the downsides that they're so against may have a chance with them, but without something that formally and clearly provides the arguments, I don't think that they're even going to pay much attention at this point. The topic has been debated to death previously, and without a DIP, it really doesn't matter what Walter and Andrei think, since at this point, they wouldn't introduce a language change like that without a DIP. So, they'll just do like
Re: rvalues -> ref (yup... again!)
On Friday, March 30, 2018 18:52:38 Walter Bright via Digitalmars-d wrote: > On 3/30/2018 6:03 PM, Manu wrote: > > Sadly, I don't think I'll be able to make it to DConf this year... > : > :-( > : > > which is probably a reason for rejoice of literally everybody > > attending! :P > > You'll be missed. My usual response to that is that it's better to be missed than to be hit. ;) But yes, it will be a shame if it's unable to come. - Jonathan M Davis
Re: rvalues -> ref (yup... again!)
On 30 March 2018 at 18:52, Walter Bright via Digitalmars-d wrote: > On 3/30/2018 6:03 PM, Manu wrote: >> >> Sadly, I don't think I'll be able to make it to DConf this year... > > > :-( > >> which is probably a reason for rejoice of literally everybody >> attending! :P > > > You'll be missed. Thank your stars... I wouldn't be able to resist heckling everyone on this the whole time! :P
Re: rvalues -> ref (yup... again!)
On 30.03.2018 11:06, Atila Neves wrote: Right, and I was assuming (perhaps incorrectly) that this existing code was C++, hence me being on board with binding rvalues to const ref there. That must not happen. D const and C++ const don't even mean the same thing, and now suddenly you will see people use extern(C++) just to get the binding of rvalues to ref. Add to this the confusing overload behavior if const signifies anything other than read-only.
Re: rvalues -> ref (yup... again!)
On 3/31/18 9:28 AM, Timon Gehr wrote: On 30.03.2018 11:06, Atila Neves wrote: Right, and I was assuming (perhaps incorrectly) that this existing code was C++, hence me being on board with binding rvalues to const ref there. That must not happen. D const and C++ const don't even mean the same thing, and now suddenly you will see people use extern(C++) just to get the binding of rvalues to ref. Add to this the confusing overload behavior if const signifies anything other than read-only. This is a very important aspect that must not be overlooked. Allow me to reiterate: the one way to get this going is convert the forum dialog into a team and a DIP. Andrei