Re: Const ref and rvalues again...
On 13.11.2012 15:07, martin wrote:
On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:
Your proposal isn't really related to this thread's topic
Um, Const ref and rvalues again, I suggest it to be the default
behavior, how is this not related ?
The topic here is binding rvalues to (const) ref parameters. You, on the
other hand, are suggesting to flip the constness of by-value parameters
(int = val/mutable int, const int = int), which affects both rvalues
and lvalues (no difference between them) and only by-value parameters.
Yes, you understood correctly:
void f (const ref int x, int y, ref int z); =
void f (int x, val int y, ref int z);
The point here is to make We need a way for a function to declare
that it doesn't want it's argument to be copied, but it also doesn't
care whether the argument is an rvalue or an lvalue. a default
behavior.
So now tell me why argument x wouldn't be copied. It's passed by value,
so of course it is copied (lvalues)/moved (rvalues) just as it is now.
The only difference is that the parameter won't be modified by f().
I guess what you furthermore implicate is that you'd expect the compiler
to automatically pass appropriate arguments to such parameters by
reference to avoid copying (for large structs or structs with
non-trivial copy constructors). Such a (handy!) optimization is sadly
not possible due to aliasing issues, e.g.:
int foo(ref int dst, const int src)
{
dst = 2*src;
return src;
}
// optimized foo():
int bar(ref int dst, const ref int src)
{
dst = 2*src;
return src;
}
int i = 1;
assert(foo(i, i) == 1 i == 2); // okay
i = 1;
assert(bar(i, i) == 2 i == 2); // wtf?!
// the const src parameter is actually modified since the
// original argument i is also used as mutable dst parameter!
Would it ? How many functions actually change their non ref/out
arguments ? Can you point out any existing public code that would be
broken ?
I don't want to look for examples in Phobos etc. as it should be trivial
to imagine cases such as:
void bla(float x)
{
// restrict x to safe range [0,1]
x = max(0, min(1, x));
}
I see, you are correct, if it is not copied then it can be changed
before function finished through some other references hence it must be
copied.
Re: Const ref and rvalues again...
On 13.11.2012 2:16, martin wrote:
On Monday, 12 November 2012 at 23:38:43 UTC, luka8088 wrote:
What about making this a default behavior and introducing a new
keyword if the function wants to modify the argument but it is not ref
(pass by value) ? The reason I think that this should be a default
behavior because not many functions actually modify their arguments
and so it leaves a lot of space for optimization.
For example:
void f (int x, val int y, ref int z) {
x = 1; // x is not copied
// compiler throws an error, x is not passed by value
// and therefor could not / should not be changed
y = 2; // ok, y is copied
z = 3; // ok, z is a reference
}
Your proposal isn't really related to this thread's topic, but I
Um, Const ref and rvalues again, I suggest it to be the default
behavior, how is this not related ?
understand what you mean (although your code comments distract me a bit):
void f(const int x, int y, ref int z); =
void f(int x, val/mutable int y, ref int z);
Yes, you understood correctly:
void f (const ref int x, int y, ref int z); =
void f (int x, val int y, ref int z);
The point here is to make We need a way for a function to declare that
it doesn't want it's argument to be copied, but it also doesn't care
whether the argument is an rvalue or an lvalue. a default behavior.
I use const/in ;) parameters a lot in my code too to prevent accidental
modifications, so my function signatures may be more compact by treating
normal pass-by-value parameters as const if not denoted with a special
keyword. I guess it wouldn't be very important for optimization though
because I'd expect the optimizer to detect unchanged parameters. Anyway,
your proposal would completely break existing code.
Would it ? How many functions actually change their non ref/out
arguments ? Can you point out any existing public code that would be
broken ?
Re: Const ref and rvalues again...
On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:
Would it ? How many functions actually change their non ref/out
arguments ? Can you point out any existing public code that
would be broken ?
It would be possible that if the language became
const-preference that a simple regex tool could be made that
would do the conversions, thereby any code broken in this way
could be un-broken just as easily; But that assumes you aren't
using mixins or magic as part of your signatures.
Somehow this reminds me a little of when I worked at a company
where we were trying out asp as a web server; The whole VB script
was by default 'by ref' so you littered all your functions with
'byVal' in order for your behavior to act as you expected.
Anyways, my take on this is consistency would be a lot more
difficult and annoying unless you had different rules for the
signature vs all other references... I doubt you would say 'this
is mutable here but immutable here' type of thing. So
assuming 'mutable' is used, then the following would be
comparable...
//D as of now
int func(int x, int y, const ref int z) {
int something; //mutable far more likely
int something2;
const int lessOften;
}
//then would become...
//if x y aren't ever changed then mutable may be unneeded.
mutable int func(mutable int x, mutable int y, ref int z) {
mutable int something;
mutable int something2;
int lessOften; //const (once set)
}
//or for inconsistancy..
//mutable or const as a return? (Or either?)
//and which would/should you use to reverse it?
int func(mutable int x, mutable int y, ref int z) {
int something; //mutable
int something2;
const int lessOften; //const
}
Seems in a function body you are far more likely to have mutable
items, while in the signature you're more likely to have const
items; But mixing them or changing how you do it would likely
break code very easily if it isn't signature only, but it doesn't
seem like a good idea...
Now in the above the function may not specify 'x' is const it
doesn't guarantees it ever changes it (but it's a local copy so
does it matter?), but specifically specifying it may be more
clutter than actually useful.
All in all it seems like it would have far more confusion (and
break code) than help; although having it prefer const versions
of functions/methods to non-const ones should probably have a
higher priority (Although you then couldn't have a non-const one
unless it was as part of the struct/class constness and not the
variables, (and ref preferred over non-ref)).
Re: Const ref and rvalues again...
On 13.11.2012 11:00, Era Scarecrow wrote:
On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:
Would it ? How many functions actually change their non ref/out
arguments ? Can you point out any existing public code that would be
broken ?
It would be possible that if the language became const-preference that a
simple regex tool could be made that would do the conversions, thereby
any code broken in this way could be un-broken just as easily; But that
assumes you aren't using mixins or magic as part of your signatures.
Somehow this reminds me a little of when I worked at a company where we
were trying out asp as a web server; The whole VB script was by default
'by ref' so you littered all your functions with 'byVal' in order for
your behavior to act as you expected.
Anyways, my take on this is consistency would be a lot more difficult
and annoying unless you had different rules for the signature vs all
other references... I doubt you would say 'this is mutable here but
immutable here' type of thing. So assuming 'mutable' is used, then
the following would be comparable...
//D as of now
int func(int x, int y, const ref int z) {
int something; //mutable far more likely
int something2;
const int lessOften;
}
//then would become...
//if x y aren't ever changed then mutable may be unneeded.
mutable int func(mutable int x, mutable int y, ref int z) {
mutable int something;
mutable int something2;
int lessOften; //const (once set)
}
//or for inconsistancy..
//mutable or const as a return? (Or either?)
//and which would/should you use to reverse it?
int func(mutable int x, mutable int y, ref int z) {
int something; //mutable
int something2;
const int lessOften; //const
}
Seems in a function body you are far more likely to have mutable items,
while in the signature you're more likely to have const items; But
mixing them or changing how you do it would likely break code very
easily if it isn't signature only, but it doesn't seem like a good idea...
Now in the above the function may not specify 'x' is const it doesn't
guarantees it ever changes it (but it's a local copy so does it
matter?), but specifically specifying it may be more clutter than
actually useful.
All in all it seems like it would have far more confusion (and break
code) than help; although having it prefer const versions of
functions/methods to non-const ones should probably have a higher
priority (Although you then couldn't have a non-const one unless it was
as part of the struct/class constness and not the variables, (and ref
preferred over non-ref)).
Can you point out any existing public code that would be broken ?
Re: Const ref and rvalues again...
On Tuesday, 13 November 2012 at 10:09:27 UTC, luka8088 wrote:
Can you point out any existing public code that would be broken?
Off hand, no.. I'm not that familiar with a lot of the code or
the in depth details of phoboes; However suddenly reversing what
is mutable and what is const is bound to break a lot of things
even unintentionally.
Hmmm.. Does remind me of a bit of my code sometimes... Something
went like...
class S {
void func(S s) {
if (!s)
s = new S(); //create one if not passed in.
//process using S and s (be it new or passed in)
}
}
That would certainly cause a problem.. In my own code example it
may empty the pointer/reference if certain requirements were met,
and then use if the class reference was set or null for later
logic (even if the object referenced to wasn't changed...).
Course being passed something like a string... (seems the most
likely place you'd find it), in which case using .dup on the
input string right back into the variable would be completely
understandable. ie:
string someStringTransformationWithCOW(string x) {
//if COW'd then
x = x.dup;
//...
return x;
}
Course having the input as char[] rather than string makes more
sense for on the fly changes before returning it...
Re: Const ref and rvalues again...
On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:
Your proposal isn't really related to this thread's topic
Um, Const ref and rvalues again, I suggest it to be the
default behavior, how is this not related ?
The topic here is binding rvalues to (const) ref parameters. You,
on the other hand, are suggesting to flip the constness of
by-value parameters (int = val/mutable int, const int = int),
which affects both rvalues and lvalues (no difference between
them) and only by-value parameters.
Yes, you understood correctly:
void f (const ref int x, int y, ref int z); =
void f (int x, val int y, ref int z);
The point here is to make We need a way for a function to
declare that it doesn't want it's argument to be copied, but it
also doesn't care whether the argument is an rvalue or an
lvalue. a default behavior.
So now tell me why argument x wouldn't be copied. It's passed by
value, so of course it is copied (lvalues)/moved (rvalues) just
as it is now. The only difference is that the parameter won't be
modified by f().
I guess what you furthermore implicate is that you'd expect the
compiler to automatically pass appropriate arguments to such
parameters by reference to avoid copying (for large structs or
structs with non-trivial copy constructors). Such a (handy!)
optimization is sadly not possible due to aliasing issues, e.g.:
int foo(ref int dst, const int src)
{
dst = 2*src;
return src;
}
// optimized foo():
int bar(ref int dst, const ref int src)
{
dst = 2*src;
return src;
}
int i = 1;
assert(foo(i, i) == 1 i == 2); // okay
i = 1;
assert(bar(i, i) == 2 i == 2); // wtf?!
// the const src parameter is actually modified since the
// original argument i is also used as mutable dst parameter!
Would it ? How many functions actually change their non ref/out
arguments ? Can you point out any existing public code that
would be broken ?
I don't want to look for examples in Phobos etc. as it should be
trivial to imagine cases such as:
void bla(float x)
{
// restrict x to safe range [0,1]
x = max(0, min(1, x));
}
Re: Const ref and rvalues again...
What about making this a default behavior and introducing a new keyword
if the function wants to modify the argument but it is not ref (pass by
value) ? The reason I think that this should be a default behavior
because not many functions actually modify their arguments and so it
leaves a lot of space for optimization.
For example:
void f (int x, val int y, ref int z) {
x = 1; // x is not copied
// compiler throws an error, x is not passed by value
// and therefor could not / should not be changed
y = 2; // ok, y is copied
z = 3; // ok, z is a reference
}
On 18.10.2012 5:07, Malte Skarupke wrote:
Hello,
I realize that this has been discussed before, but so far there is no
solution and this really needs to be a high priority:
We need a way for a function to declare that it doesn't want it's
argument to be copied, but it also doesn't care whether the argument is
an rvalue or an lvalue.
The C++ way of doing this would be to declare the argument as a const .
Apparently it is not desired that we do the same thing for const ref.
Currently, if you want that behavior, you have to write 2^n permutations
of your function, with n being the number of arguments that the function
takes.
Here's my attempt at passing a struct to a function that takes three
arguments without the struct being copied:
int copyCounter = 0;
struct CopyCounter
{
this(this) { ++copyCounter; }
}
void takeThree(ref in CopyCounter a, ref in CopyCounter b, ref in
CopyCounter c)
{
writeln(took three);
}
void takeThree(in CopyCounter a, ref in CopyCounter b, ref in
CopyCounter c)
{
takeThree(a, b, c);
}
void takeThree(ref in CopyCounter a, in CopyCounter b, ref in
CopyCounter c)
{
takeThree(a, b, c);
}
void takeThree(ref in CopyCounter a, ref in CopyCounter b, in
CopyCounter c)
{
takeThree(a, b, c);
}
void takeThree(in CopyCounter a, in CopyCounter b, ref in CopyCounter c)
{
takeThree(a, b, c);
}
void takeThree(in CopyCounter a, ref in CopyCounter b, in CopyCounter c)
{
takeThree(a, b, c);
}
void takeThree(ref in CopyCounter a, in CopyCounter b, in CopyCounter c)
{
takeThree(a, b, c);
}
void takeThree(in CopyCounter a, in CopyCounter b, in CopyCounter c)
{
takeThree(a, b, c);
}
static CopyCounter createCopyCounter()
{
return CopyCounter();
}
void main()
{
CopyCounter first;
CopyCounter second;
CopyCounter third;
takeThree(first, second, third);
takeThree(createCopyCounter(), second, createCopCounter());
assert(copyCounter == 0); // yay, works
}
My propsed solution is this:
- Make functions that take ref in arguments also accept rvalues.
- The user can still provide an overload that accepts an rvalue, using
the in keyword, and that one will be preferred over the ref in version.
What do you think?
Malte
Re: Const ref and rvalues again...
On Monday, 12 November 2012 at 23:38:43 UTC, luka8088 wrote:
What about making this a default behavior and introducing a new
keyword if the function wants to modify the argument but it is
not ref (pass by value) ? The reason I think that this should
be a default behavior because not many functions actually
modify their arguments and so it leaves a lot of space for
optimization.
For example:
void f (int x, val int y, ref int z) {
x = 1; // x is not copied
// compiler throws an error, x is not passed by value
// and therefor could not / should not be changed
y = 2; // ok, y is copied
z = 3; // ok, z is a reference
}
Your proposal isn't really related to this thread's topic, but I
understand what you mean (although your code comments distract me
a bit):
void f(const int x, int y, ref int z); =
void f(int x, val/mutable int y, ref int z);
I use const/in ;) parameters a lot in my code too to prevent
accidental modifications, so my function signatures may be more
compact by treating normal pass-by-value parameters as const if
not denoted with a special keyword. I guess it wouldn't be very
important for optimization though because I'd expect the
optimizer to detect unchanged parameters. Anyway, your proposal
would completely break existing code.
Re: Const ref and rvalues again...
11/10/2012 11:07 PM, Jonathan M Davis пишет:
And actually, to make matters worse, I'm not sure that scope on delegates is
working correctly. I thought that it was, but this code compiles:
import std.stdio;
void delegate() global;
void foo(scope void delegate() del)
{
global = del;
}
void main()
{
{
char[5] bar = hello;
foo((){writeln(bar);});
}
char[7] baz = goodbye;
global();
}
It also prints out hello, and if a closure had not been allocated,
I would
have at least half-expected it to print out goodb, because I'd have thought
that baz would have been taking up the same memory that bar had been.
Nope. It's just that the stack is intact and contains: hello and goodbye
one after another. Without optimizations { } scope doesn't mean reuse
stack space.
Now if play with stack a bit, for me the next one prints:
-²↑
import std.stdio;
void delegate() global;
void foo(scope void delegate() del)
{
global = del;
}
void f()
{
{
char[5] bar = hello;
foo((){writeln(bar);});
}
}
void main()
{
char[7] baz = goodbye;
f();
global();
}
--
Dmitry Olshansky
Re: Const ref and rvalues again...
On Sunday, November 11, 2012 13:36:05 Dmitry Olshansky wrote:
Nope. It's just that the stack is intact and contains: hello and goodbye
one after another. Without optimizations { } scope doesn't mean reuse
stack space.
Now if play with stack a bit, for me the next one prints:
-²↑
import std.stdio;
void delegate() global;
void foo(scope void delegate() del)
{
global = del;
}
void f()
{
{
char[5] bar = hello;
foo((){writeln(bar);});
}
}
void main()
{
char[7] baz = goodbye;
f();
global();
}
It still prints hello, even with full optimations turned on. So, it must be
allocating a closure in spite of scope. So, it looks to me like scope is just
completely ignored and does absolutely nothing at this point, unless I'm just
completely missing something here.
- Jonathan M Davis
Re: Const ref and rvalues again...
On Sunday, 11 November 2012 at 10:09:17 UTC, Jonathan M Davis
wrote:
It still prints hello, even with full optimations turned on.
So, it must be
allocating a closure in spite of scope. So, it looks to me like
scope is just
completely ignored and does absolutely nothing at this point,
unless I'm just
completely missing something here.
Try this:
---
import std.stdio;
void delegate() global;
void foo(scope void delegate() del)
{
global = del;
}
void f()
{
{
char[5] bar = hello;
foo((){writeln(bar);});
}
}
void smashStack() {
uint[1000] dummy = 0xbadcab1e;
asm { nop; }
}
void main()
{
char[7] baz = goodbye;
f();
smashStack();
global();
}
---
David
Re: Const ref and rvalues again...
11/11/2012 2:08 PM, Jonathan M Davis пишет:
On Sunday, November 11, 2012 13:36:05 Dmitry Olshansky wrote:
Nope. It's just that the stack is intact and contains: hello and goodbye
one after another. Without optimizations { } scope doesn't mean reuse
stack space.
Now if play with stack a bit, for me the next one prints:
-²↑
import std.stdio;
void delegate() global;
void foo(scope void delegate() del)
{
global = del;
}
void f()
{
{
char[5] bar = hello;
foo((){writeln(bar);});
}
}
void main()
{
char[7] baz = goodbye;
f();
global();
}
It still prints hello, even with full optimations turned on.
I tried with and without optimizations. I get garbage as expected.
So, it must be
allocating a closure in spite of scope. So, it looks to me like scope is just
completely ignored and does absolutely nothing at this point, unless I'm just
completely missing something here.
Something must be screwed up. I dunno what, I use near-latest DMD from
github and Win32 binaries.
For good measure try making stack variables larger if you are on 64bit.
Drop in a couple of calls to writeln before and after calling 'f' it
should scramble the stack.
--
Dmitry Olshansky
Re: Const ref and rvalues again...
On 10 November 2012 21:07, Jonathan M Davis jmdavisp...@gmx.com wrote:
On Saturday, November 10, 2012 13:21:42 Manu wrote:
I'm still not buying this. Here's a common struct I will pass by ref
(perhaps the most common struct in my industry):
struct Vector { float, x,y,z,w; }
struct Matrix { Vector xRow, yRow, zRow, wRow; }
Vector mul( scope const ref Matrix m, scope const Vector v)
{
Vector v;
// perform a matrix multiply against the vector...
// this work uses every single field of the inputs given, but the
result
it produces has to references to the sources.
// everything is operated on and copied to the output struct, which is
returned.
return result;
}
Why should this be a problem?
The majority of my work-horse structs apply to this pattern. This is
what I
imagine 'scope' to be for...
The main advantage I expect is that I can have confidence that passing
rvalues (temporaries) is safe, and that external code won't take
references
to memory that I may not own/control. Is that not the point?
Surely the problem that scope should be protecting against is a pointer
to
any part of the argument escaping. *Copies* of values contained in the
argument/s are fine.
H. scope on value types is pointless, because there are no references
to
escape, but if you pass by ref, then it does become possible for a pointer
to
the argument to escape,
Precisely, to me, this seems like the ENTIRE POINT of 'in'? (I originally
presumed 'in' implied ref, but I was wrong, 'in ref' is supported however)
but I don't know that that's actually actually covered
by scope. The description for scope in docs is that references in the
pa
rameter cannot be escaped (e.g. assigned to a global variable). And
taking the address of a local variable (which is the only way that any
sort of
reference to the data could escape) is never @safe anyway.
When did '@safe'ty enter into it? Are you saying that a ref variable is
somehow a local variable? It's a local pointer to a foreign variable... and
a function can usually operate on that data however it likes.
Scope would promise that nothing other than the function I give it to will
get its grubby little hands on it.
Let's say that function wanted to call through to some other function and
pass the variable along (by ref). Obviously, the second function would also
have to have it's inputs marked scope, to promise that it never escapes
from there.
I imagine scope similarly to const, once it goes scope, the whole callstack
must maintain the scope property, otherwise the outermost function can't
trust it anymore.
This makes perfect sense for any function that is likely to receive
immediate or local variables by reference (which is extremely common).
It also seems absolutely relevant to the rvalues - ref thing.
If you passed in a
pointer, and scope were fully working, then you'd be protected against the
pointer escaping, but passing by ref isn't really the same thing. I'd have
thought that taking the address of a variable passed by ref would fall into
pretty much the same camp as taking the address of any other local
variable,
which is completely unsafe to escape to the point that I'm not sure that
there's any point in protecting against it. It's just completely stupid to
do
anyway and is definitely @system. Outside of taking the address of a ref
parameter, taking the address of a local variable and escpaing it is
_always_
going to result in garbage, and ref parameters aren't really references in
the
normal sense, so I don't know.
What do you mean 'aren't really references in the normal sense'?
You bring up a good point, but I don't know if it's applicable. Certainly,
without the ref there (like is the case with the Vector that you're passing
in), scope would never do anything, because it doesn't even theoretically
have
anything to do. It's purely a value type that's not even being passed by
ref.
Correct, scope on a purely value type passed by-value means absolutely
nothing.
scope on a pointer parameter means something; I would expect the pointer
its self, nor a pointer INTO anything under the pointer could escape.
scope on a by-value parameter that contains pointers (like
delegates/slices) has meaning, I presume scope would be transitive like
const, apply the pointer rule above.
ref is sugar for a pointer, and the above applies verbatim. If you take the
address of the ref argument, you have the pointer, and it mustn't escape,
likewise, no pointers or pointer to anything beneath it.
If my imagination of this concept completely wrong?
This sounds useful to me, I can't imagine another scenario where the
keyword a) makes sense, and b) is useful...
In general though, putting scope on struct parameters would cause a lot of
problems, because of arrays that they might hold and whatnot. Slices
wouldn't
be able to escape (and so copies of the struct wouldn't be able escape
without
deep
Re: Const ref and rvalues again...
On Sunday, November 11, 2012 12:10:59 David Nadlinger wrote:
On Sunday, 11 November 2012 at 10:09:17 UTC, Jonathan M Davis
wrote:
It still prints hello, even with full optimations turned on.
So, it must be
allocating a closure in spite of scope. So, it looks to me like
scope is just
completely ignored and does absolutely nothing at this point,
unless I'm just
completely missing something here.
Try this:
---
import std.stdio;
void delegate() global;
void foo(scope void delegate() del)
{
global = del;
}
void f()
{
{
char[5] bar = hello;
foo((){writeln(bar);});
}
}
void smashStack() {
uint[1000] dummy = 0xbadcab1e;
asm { nop; }
}
void main()
{
char[7] baz = goodbye;
f();
smashStack();
global();
}
---
That did it. And if scope is removed, it works again. I clearly don't get the
low level stuff though, since it's just plain bizarre to me that the previous
example didn't do it.
- Jonathan M Davis
Re: Const ref and rvalues again...
On Sunday, November 11, 2012 15:17:03 Dmitry Olshansky wrote: Something must be screwed up. I dunno what, I use near-latest DMD from github and Win32 binaries. I'm on 64-bit Linux, so that may change things a bit. - Jonathan M Davis
Re: Const ref and rvalues again...
On Sunday, November 11, 2012 13:30:12 Manu wrote:
What do you mean 'aren't really references in the normal sense'?
ref is not really part of a variable's type. It's just a storage class and is
really only applicable in very specific circumstances. A function parameter is
a local variable, but ref makes it so that it just so happens to affect a
variable outside the function as well. The semantics of the function itself or
how the parameter is used aren't any different either way. So, if you were to
remove ref from the parameter, the function itself would be unaffected
semantically-speaking (it would affect the code generation some though). It's
the caller whose semantics change.
This is in direct contrast with a pointer or reference type where the fact
that it refers to something else outside the function is fully part of the
type.
Taking the address of a local variable is already something that is incredibly
unsafe and doesn't work. So, anyone doing it is being an idiot anyway. And if
it's _always_ stupid, then protecting it with scope isn't really necessary.
pure mostly stops anything like that from happening in the general case, but
doesn't prevent people from doing stupid stuff like
int* foo() pure
{
int i;
return bar(i);
}
int* bar(int* p) pure
{
return p;
}
Regardless, my point was that the refness of the parameter isn't really part
of the type of the parameter, whereas all other issues with reference escaping
that scope would affect _are_ part of the type. So, you're talking about scope
protecting against something quite different from what it would protect from in
all other circumstances. Protecting against escaping a ref parameter and a
reference type escaping are two different (albeit not completely unrelated)
things.
Honestly, before you brought up the possibility of scope protecting against
pointers to ref parameters escaping, it had never occurred to me. I've never
seen anyone bring it up before. And as you're talking about protecting against
escaping a pointer to a local variable, which is an incredibly stupid thing to
do anyway, I'm not sure that the protection is really needed.
But I don't know what Walter's or Andrei's intentions were with regards to
whether scope would extend to the fact that the parameter is ref. I don't
think that it's necessarily the case that it wouldn't, but you're then
protecting against escaping a pointer to a local variable (albeit one which
affects a variable outside of the function) rather than against escaping a
reference type. And from everything I've seen, it's protecting against
escaping reference types which was its purpose.
Unfortunately, TDPL doesn't seem to discuss scope parameters at all (I thought
that it did, but I can't find it anywhere now if it does), meaning that the
online docs are the only official documentation on scope, and all other
information on it comes from newsgroup discussions on the matter. It's quite
possible that many of us have misunderstood aspects of what scope parameters
were intended to do.
So, assuming that if/when scope actually starts affecting more than delegates
as quite a few us think that it's supposed to do, then what you're looking for
may very well be on the table, much as I wouldn't have expected it to be. I
don't know.
Feel free to create an enhancement request for it. Even it wasn't part of the
original intention of scope parameters, Walter may think that it's worth
making it so that it is.
That would be the point though. If you don't want that, then you don't want
scope.
The problem is that in most cases, I really don't think that that is what the
average programmer wants, and there _are_ people who use in as a matter
course. It would be one thing if you had to explicitly use scope, then I
suspect that it wouldn't be used as a matter of course by much of anyone. For
the most part, it would then just be used when the programmer knew that that's
what they wanted. But with in, so many people seem to really like the concept
of it being the opposite of out that they use it as a matter of course without
understanding the consequences.
- Jonathan M Davis
Re: Const ref and rvalues again...
On 9 November 2012 21:39, Jonathan M Davis jmdavisp...@gmx.com wrote:
On Friday, November 09, 2012 15:55:12 Manu wrote:
Does that actually make sense? Surely a function that receives a scope
argument can return that argument, since it's only passing it back to the
same function that already owns it... it knows it can trust that
function,
since it was received from that function.
It can't. That would mean that the reference escaped. That would be
particularly deadly for delegates. Think about what happens if the scoped
delegate is put into a struct which is returned.
struct Result
{
delegate del;
...
}
Result foo(scope delegate... bar)
{
..
return Result(bar);
}
auto baz()
{
Result r;
{
int n = 5;
r = foo((){writeln(n);});
}
r.del();
}
baz has no idea where the delegate in r came from. It has no idea that it
wasn't allocated as a closure. So, it's not going to allocate one, which
means
that the delegate refers to a part of the stack which won't exist anymore
when
the delegate actually gets called. If scope wasn't used, that wouldn't have
been a problem, because a closure would have been allocated as soon as the
delegate had been passed to foo, but because scope was used, it knows that
the
delegate won't escape, so it doesn't allocate the closure (since it's not
necessary). But that only works because scope prevents escaping -
including by
the return value. So, the above code _must_ be invalid.
Okay, makes sense.
Any struct holding any reference types would be
in the same boat, as would any class or AA.
I don't follow the problem with reference args. surely they can be
evaluated just fine? Just that nothing can escape the function...
It's the fact that they can't escape the function which is the problem. If
scope is working correctly, than any and all reference types which are
passed
to the function via a scope parameter (be it directly or within another
object) cannot escape the function in any way shape or form. So, you can't
assign any of them to any static or module-level variables (not generally a
big deal) and you can't use any of them in the return value (often a big
deal). Sometimes, that's exactly what you want, but in the general case,
you
don't want to prevent anything you pass into a function from being returned
from it, so scope quickly becames overly restrictive.
I'm still not buying this. Here's a common struct I will pass by ref
(perhaps the most common struct in my industry):
struct Vector { float, x,y,z,w; }
struct Matrix { Vector xRow, yRow, zRow, wRow; }
Vector mul( scope const ref Matrix m, scope const Vector v)
{
Vector v;
// perform a matrix multiply against the vector...
// this work uses every single field of the inputs given, but the result
it produces has to references to the sources.
// everything is operated on and copied to the output struct, which is
returned.
return result;
}
Why should this be a problem?
The majority of my work-horse structs apply to this pattern. This is what I
imagine 'scope' to be for...
The main advantage I expect is that I can have confidence that passing
rvalues (temporaries) is safe, and that external code won't take references
to memory that I may not own/control. Is that not the point?
Surely the problem that scope should be protecting against is a pointer to
any part of the argument escaping. *Copies* of values contained in the
argument/s are fine.
Re: Const ref and rvalues again...
On Saturday, November 10, 2012 13:21:42 Manu wrote:
I'm still not buying this. Here's a common struct I will pass by ref
(perhaps the most common struct in my industry):
struct Vector { float, x,y,z,w; }
struct Matrix { Vector xRow, yRow, zRow, wRow; }
Vector mul( scope const ref Matrix m, scope const Vector v)
{
Vector v;
// perform a matrix multiply against the vector...
// this work uses every single field of the inputs given, but the result
it produces has to references to the sources.
// everything is operated on and copied to the output struct, which is
returned.
return result;
}
Why should this be a problem?
The majority of my work-horse structs apply to this pattern. This is what I
imagine 'scope' to be for...
The main advantage I expect is that I can have confidence that passing
rvalues (temporaries) is safe, and that external code won't take references
to memory that I may not own/control. Is that not the point?
Surely the problem that scope should be protecting against is a pointer to
any part of the argument escaping. *Copies* of values contained in the
argument/s are fine.
H. scope on value types is pointless, because there are no references to
escape, but if you pass by ref, then it does become possible for a pointer to
the argument to escape, but I don't know that that's actually actually covered
by scope. The description for scope in docs is that references in the pa
rameter cannot be escaped (e.g. assigned to a global variable). And
taking the address of a local variable (which is the only way that any sort of
reference to the data could escape) is never @safe anyway. If you passed in a
pointer, and scope were fully working, then you'd be protected against the
pointer escaping, but passing by ref isn't really the same thing. I'd have
thought that taking the address of a variable passed by ref would fall into
pretty much the same camp as taking the address of any other local variable,
which is completely unsafe to escape to the point that I'm not sure that
there's any point in protecting against it. It's just completely stupid to do
anyway and is definitely @system. Outside of taking the address of a ref
parameter, taking the address of a local variable and escpaing it is _always_
going to result in garbage, and ref parameters aren't really references in the
normal sense, so I don't know.
You bring up a good point, but I don't know if it's applicable. Certainly,
without the ref there (like is the case with the Vector that you're passing
in), scope would never do anything, because it doesn't even theoretically have
anything to do. It's purely a value type that's not even being passed by ref.
In general though, putting scope on struct parameters would cause a lot of
problems, because of arrays that they might hold and whatnot. Slices wouldn't
be able to escape (and so copies of the struct wouldn't be able escape without
deep copying, let alone the array itself). So, while scope may be very useful
in some such cases (assuming that it worked), it's not necessarily something
that you'd want as a matter of course. Part of it probably depends on your
programming style though. If you have a lot of functions that take arguments
and don't return anything that was in them ever, then scope is less of a big
deal, but that's the sort of thing that happens a _lot_ in my experience, so
scope would very quickly become extremely annoying.
And actually, to make matters worse, I'm not sure that scope on delegates is
working correctly. I thought that it was, but this code compiles:
import std.stdio;
void delegate() global;
void foo(scope void delegate() del)
{
global = del;
}
void main()
{
{
char[5] bar = hello;
foo((){writeln(bar);});
}
char[7] baz = goodbye;
global();
}
It also prints out hello, and if a closure had not been allocated, I would
have at least half-expected it to print out goodb, because I'd have thought
that baz would have been taking up the same memory that bar had been. So, it
looks like scope may be completely and utterly broken at this point. I don't
know.
- Jonathan M Davis
Re: Const ref and rvalues again...
On 9 November 2012 00:44, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, November 08, 2012 21:49:58 Manu wrote: That's cute, but it really feels like a hack. All of a sudden the debugger doesn't work properly anymore, you need to step-in twice to enter the function, and it's particularly inefficient in debug builds (a point of great concern for my industry!). Please just with the compiler creating a temporary in the caller space. Restrict is to const ref, or better, in ref (scope seems particularly important here). I honestly wish that in didn't exist in the language. The fact that it it's an alias two different attributes is confusing, and people keep using it without realizing what they're getting into. I understand it's an alias for 'const scope', but what's this about scope not working correctly? What's wrong with it? It seems like precisely what you want in this case... you don't want a temporary escaping the function you pass it to. What's the problem? If scope worked correctly, you'd only want it in specific circumstances, not in general. And since it doesn't work correctly aside from delegates, once it _does_ work correctly, it'll break code all over the place, because people keep using in, because they like how it corresponds with out or whatever. I like how it promises that the thing I pass can't escape the callee, not that it's the natural english compliment to 'out'.
Re: Const ref and rvalues again...
On Friday, November 09, 2012 10:33:45 Manu wrote: On 9 November 2012 00:44, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, November 08, 2012 21:49:58 Manu wrote: That's cute, but it really feels like a hack. All of a sudden the debugger doesn't work properly anymore, you need to step-in twice to enter the function, and it's particularly inefficient in debug builds (a point of great concern for my industry!). Please just with the compiler creating a temporary in the caller space. Restrict is to const ref, or better, in ref (scope seems particularly important here). I honestly wish that in didn't exist in the language. The fact that it it's an alias two different attributes is confusing, and people keep using it without realizing what they're getting into. I understand it's an alias for 'const scope', but what's this about scope not working correctly? What's wrong with it? It seems like precisely what you want in this case... you don't want a temporary escaping the function you pass it to. What's the problem? It only works with delegates. In all other cases, it's ignored. If scope worked correctly, you'd only want it in specific circumstances, not in general. And since it doesn't work correctly aside from delegates, once it _does_ work correctly, it'll break code all over the place, because people keep using in, because they like how it corresponds with out or whatever. I like how it promises that the thing I pass can't escape the callee, not that it's the natural english compliment to 'out'. There are cases where it's useful, but there are a ton of cases where it would be really annoying if it worked properly. For instance, in a _lot_ of cases, it would cause problems for a function taking an array, because it couldn't return a slice of the array. Any struct holding any reference types would be in the same boat, as would any class or AA. In most cases, it really doesn't matter whether references escape, because the stuff is on the GC heap, and it'll collect it when it's appropriate. It would be a much bigger deal if you had to worry about ownership. For delegates, it's great because it allows you to avoid allocating a closure, so if you don't need the delegate to escape, it makes perfect sense to make it scope. For everything else, whether it makes sense depends on what you're doing, and sometimes it would be useful, but most of the time, it really wouldn't be. But since in currently only works for delegates, it's absolutely pointless to use it instead of const for anything else, and if/when scope is ever fixed to work with reference types in general, then all kinds of code that uses in will break, because it'll be escaping references, and the programmer didn't notice it. So, you might as well just not use it at this point except for delegates. Regardless, you don't need in to use scope. You can just use scope directly. So, having in buys you _nothing_, and it unfortunately encourages people to use it (which IMHO is bad due to the issues with scope), because they like how it's the opposite of out. So, if we didn't have in, we'd be able to do exactly what we can do now, but people wouldn't be using scope all over the place via in just because they liked the idea of in being the opposite of out. But originally, in comes from D1 (where it had a meaning similar to const IIRC, but scope had nothing to do with it), so it's been around for a long time, and there's pretty much no way that it would be removed from the language. I just think that it was mistake to have it given its current semantics. It's also a bit pointless IMHO to have an attribute which is an alias for other attributes. It's a needless duplication of functionality and just increases confusion. - Jonathan M Davis
Re: Const ref and rvalues again...
On 9 November 2012 10:46, Jonathan M Davis jmdavisp...@gmx.com wrote: On Friday, November 09, 2012 10:33:45 Manu wrote: On 9 November 2012 00:44, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, November 08, 2012 21:49:58 Manu wrote: That's cute, but it really feels like a hack. All of a sudden the debugger doesn't work properly anymore, you need to step-in twice to enter the function, and it's particularly inefficient in debug builds (a point of great concern for my industry!). Please just with the compiler creating a temporary in the caller space. Restrict is to const ref, or better, in ref (scope seems particularly important here). I honestly wish that in didn't exist in the language. The fact that it it's an alias two different attributes is confusing, and people keep using it without realizing what they're getting into. I understand it's an alias for 'const scope', but what's this about scope not working correctly? What's wrong with it? It seems like precisely what you want in this case... you don't want a temporary escaping the function you pass it to. What's the problem? It only works with delegates. In all other cases, it's ignored. If scope worked correctly, you'd only want it in specific circumstances, not in general. And since it doesn't work correctly aside from delegates, once it _does_ work correctly, it'll break code all over the place, because people keep using in, because they like how it corresponds with out or whatever. I like how it promises that the thing I pass can't escape the callee, not that it's the natural english compliment to 'out'. There are cases where it's useful, but there are a ton of cases where it would be really annoying if it worked properly. For instance, in a _lot_ of cases, it would cause problems for a function taking an array, because it couldn't return a slice of the array. Does that actually make sense? Surely a function that receives a scope argument can return that argument, since it's only passing it back to the same function that already owns it... it knows it can trust that function, since it was received from that function. Any struct holding any reference types would be in the same boat, as would any class or AA. I don't follow the problem with reference args. surely they can be evaluated just fine? Just that nothing can escape the function... In most cases, it really doesn't matter whether references escape, because the stuff is on the GC heap, and it'll collect it when it's appropriate. It would be a much bigger deal if you had to worry about ownership. It's true, and stack locals are perhaps the most important thing that scope could possibly protect... which makes it particularly applicable here. For delegates, it's great because it allows you to avoid allocating a closure, so if you don't need the delegate to escape, it makes perfect sense to make it scope. For everything else, whether it makes sense depends on what you're doing, and sometimes it would be useful, but most of the time, it really wouldn't be. Of course, and whether you apply the scope attribute to a function argument equally depends on what you're doing :) I still can't see any inherent problems from what you're saying. It's still just an access restriction to enhance safety. But since in currently only works for delegates, it's absolutely pointless to use it instead of const for anything else, and if/when scope is ever fixed to work with reference types in general, then all kinds of code that uses in will break, because it'll be escaping references, and the programmer didn't notice it. So, you might as well just not use it at this point except for delegates. Very good point. I had no idea it didn't work. It'd be nice to see this fixed. Regardless, you don't need in to use scope. You can just use scope directly. So, having in buys you _nothing_, and it unfortunately encourages people to use it (which IMHO is bad due to the issues with scope), because they like how it's the opposite of out. The issue here is that 'scope' is an incomplete feature. I think it's theoretically sound. Every conversation/thing I've read on the topic certainly gave me the impression it worked... So, if we didn't have in, we'd be able to do exactly what we can do now, but people wouldn't be using scope all over the place via in just because they liked the idea of in being the opposite of out. But originally, in comes from D1 (where it had a meaning similar to const IIRC, but scope had nothing to do with it), so it's been around for a long time, and there's pretty much no way that it would be removed from the language. Oh really? That's not at all what I had imagined. I presumed 'in' was invented when it was realised that 'const scope' was a super-common pattern. I still don't see any less value in it. Just that it should work properly. I
Re: Const ref and rvalues again...
Let me summarize my (final, I guess) proposal. I think it makes sense to compare it to C++ in order to anticipate and hopefully invalidate (mainly Andrei's) objections. parameter type | lvalue|rvalue | C++ D | C++ D |-| T | copy copy | copy move T / ref T | refref | n/an/a out T (D only) |ref |n/a T (C++ only) | n/a | move auto ref T (D only) (*) |ref |ref |-| const T | copy copy | copy move const T / const ref T | refref | refref (*) const T (C++ only)| n/a | move (*): proposed additions For lvalues in both C++ and D, there are 2 options: either copy the argument (pass-by-value) or pass it by ref. There's no real difference between both languages except for D's additional 'out' keyword and, with the proposed 'auto ref' syntax, an (imo negligible) ambiguity between 'ref T' and 'auto ref T' in D. Rvalues are a different topic though. There are 3 possibilites in general: copy, move and pass by ref. Copying rvalue arguments does not make sense - the argument won't be used by the caller after the invokation, so a copy is redundant and hurts performance. D corrects this design flaw of C++ (which had to introduce rvalue refs to add move semantics on top of the default copy semantics) and therefore only supports moving instead. C++ additionally supports pass-by-ref of rvalues to const refs, but not to mutable refs. I propose to allow pass-by-ref to both const (identical syntax as C++, it's perfectly safe and logical) and mutable refs (new syntax with 'auto ref' to emphasize that the parameter may be an rvalue reference, with related consequences such as potentially missing side effects). Regarding the required overloading priorities for the proposed additions to work properly, I propose: 1) lvalues: prefer pass-by-ref so: ref/out T - auto ref T (*) - const ref T - (const) T - const lvalues: const ref T - (const) T - mutable lvalues: ref/out T - auto ref T (*) - const ref T - (const) T 2) rvalues: prefer pass-by-value (moving: argument allocated directly on callee's stack (parameter) vs. pointer/reference indirection implied by pass-by-ref) so: (const) T - auto ref T (*) - const ref T (*) Finally, regarding templates, I'm in favor of dropping the current 'auto ref' semantics and propose to simply adopt the proposed semantics for consistency and simplicity and to avoid excessive code bloating. That shouldn't break existing code I hope (unless parameters have been denoted with 'const auto ref T', which would need to be changed to 'const ref T'). Now please go ahead and shoot. :)
Re: Const ref and rvalues again...
On Fri, 09 Nov 2012 16:26:21 -, martin ki...@libero.it wrote: Let me summarize my (final, I guess) proposal. I think it makes sense to compare it to C++ in order to anticipate and hopefully invalidate (mainly Andrei's) objections. parameter type | lvalue|rvalue | C++ D | C++ D |-| T | copy copy | copy move T / ref T | refref | n/an/a out T (D only) |ref |n/a T (C++ only) | n/a | move auto ref T (D only) (*) |ref |ref |-| const T | copy copy | copy move const T / const ref T | refref | refref (*) const T (C++ only)| n/a | move (*): proposed additions For lvalues in both C++ and D, there are 2 options: either copy the argument (pass-by-value) or pass it by ref. There's no real difference between both languages except for D's additional 'out' keyword and, with the proposed 'auto ref' syntax, an (imo negligible) ambiguity between 'ref T' and 'auto ref T' in D. Rvalues are a different topic though. There are 3 possibilites in general: copy, move and pass by ref. Copying rvalue arguments does not make sense - the argument won't be used by the caller after the invokation, so a copy is redundant and hurts performance. D corrects this design flaw of C++ (which had to introduce rvalue refs to add move semantics on top of the default copy semantics) and therefore only supports moving instead. C++ additionally supports pass-by-ref of rvalues to const refs, but not to mutable refs. I propose to allow pass-by-ref to both const (identical syntax as C++, it's perfectly safe and logical) and mutable refs (new syntax with 'auto ref' to emphasize that the parameter may be an rvalue reference, with related consequences such as potentially missing side effects). Regarding the required overloading priorities for the proposed additions to work properly, I propose: 1) lvalues: prefer pass-by-ref so: ref/out T - auto ref T (*) - const ref T - (const) T - const lvalues: const ref T - (const) T - mutable lvalues: ref/out T - auto ref T (*) - const ref T - (const) T 2) rvalues: prefer pass-by-value (moving: argument allocated directly on callee's stack (parameter) vs. pointer/reference indirection implied by pass-by-ref) so: (const) T - auto ref T (*) - const ref T (*) Finally, regarding templates, I'm in favor of dropping the current 'auto ref' semantics and propose to simply adopt the proposed semantics for consistency and simplicity and to avoid excessive code bloating. That shouldn't break existing code I hope (unless parameters have been denoted with 'const auto ref T', which would need to be changed to 'const ref T'). Now please go ahead and shoot. :) Nice detailed proposal. I would suggest starting a new thread with it, to catch anyone who dozed off in this thread :p R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Const ref and rvalues again...
On Friday, 9 November 2012 at 16:53:01 UTC, Regan Heath wrote: Nice detailed proposal. I would suggest starting a new thread with it, to catch anyone who dozed off in this thread :p Thanks, and you're probably right. Here's the link: http://forum.dlang.org/thread/zteryxwxyngvyqvuk...@forum.dlang.org I've elaborated on it a little bit to avoid redundant posts.
Re: Const ref and rvalues again...
On Friday, November 09, 2012 15:55:12 Manu wrote:
Does that actually make sense? Surely a function that receives a scope
argument can return that argument, since it's only passing it back to the
same function that already owns it... it knows it can trust that function,
since it was received from that function.
It can't. That would mean that the reference escaped. That would be
particularly deadly for delegates. Think about what happens if the scoped
delegate is put into a struct which is returned.
struct Result
{
delegate del;
...
}
Result foo(scope delegate... bar)
{
..
return Result(bar);
}
auto baz()
{
Result r;
{
int n = 5;
r = foo((){writeln(n);});
}
r.del();
}
baz has no idea where the delegate in r came from. It has no idea that it
wasn't allocated as a closure. So, it's not going to allocate one, which means
that the delegate refers to a part of the stack which won't exist anymore when
the delegate actually gets called. If scope wasn't used, that wouldn't have
been a problem, because a closure would have been allocated as soon as the
delegate had been passed to foo, but because scope was used, it knows that the
delegate won't escape, so it doesn't allocate the closure (since it's not
necessary). But that only works because scope prevents escaping - including by
the return value. So, the above code _must_ be invalid.
Any struct holding any reference types would be
in the same boat, as would any class or AA.
I don't follow the problem with reference args. surely they can be
evaluated just fine? Just that nothing can escape the function...
It's the fact that they can't escape the function which is the problem. If
scope is working correctly, than any and all reference types which are passed
to the function via a scope parameter (be it directly or within another
object) cannot escape the function in any way shape or form. So, you can't
assign any of them to any static or module-level variables (not generally a
big deal) and you can't use any of them in the return value (often a big
deal). Sometimes, that's exactly what you want, but in the general case, you
don't want to prevent anything you pass into a function from being returned
from it, so scope quickly becames overly restrictive.
- Jonathan M Davis
Re: Const ref and rvalues again...
On Thursday, 8 November 2012 at 03:07:00 UTC, Jonathan M Davis wrote: Okay. Here are more links to Andrei discussing the problem: http://forum.dlang.org/post/4f83dbe5.20...@erdani.com http://www.mail-archive.com/digitalmars-d@puremagic.com/msg44070.html http://www.mail-archive.com/digitalmars-d@puremagic.com/msg43769.html http://forum.dlang.org/post/hg62rq$2c2n$1...@digitalmars.com Thank you so much for these links, Jonathan. So fortunately the special role of _const_ ref parameters has been acknowledged. From the 2nd link: The problem with binding rvalues to const ref is that once that is in place you have no way to distinguish an rvalue from a const ref on the callee site. If you do want to distinguish, you must rely on complicated conversion priorities. For example, consider: void foo(ref const Widget); void foo(Widget); You'd sometimes want to do that because you want to exploit an rvalue by e.g. moving its state instead of copying it. However, if rvalues become convertible to ref const, then they are motivated to go either way. A rule could be put in place that gives priority to the second declaration. However, things quickly get complicated in the presence of other applicable rules, multiple parameters etc. Essentially it was impossible for C++ to go this way and that's how rvalue references were born. For D I want to avoid all that aggravation and have a simple rule: rvalues don't bind to references to const. If you don't care, use auto ref. This is a simple rule that works promisingly well in various forwarding scenarios. This is exactly what we propose (to be able to avoid pointer/reference indirection for rvalues in some absolutely performance-critical cases). Unlike Andrei though, I don't find the required overloading rules complicated at all, quite the contrary in fact. From the 3rd link: Binding rvalues to const references was probably the single most hurtful design decisions for C++. I don't have time to explain now, but in short I think all of the problems that were addressed by rvalue references, and most of the aggravation within the design of rvalue references, are owed by that one particular conversion. Here's where I totally disagree with Andrei. C++ rvalue references (T) aren't used to distinguish between lvalues and rvalues when expecting a _const_ reference (I still have to see a use case for 'const T'). They are used for _mutable_ references and primarily to enforce efficient move semantics in C++, i.e., to move _mutable_ rvalue arguments (instead of copying them) and to enforce 'Named Return Value Optimization' when returning lvalues (by using std::move; goal again is to avoid a redundant copy). D fortunately seems to implement move semantics out-of-the-box (at least now in v2.060), in both cases, see Rob T's posts and my replies in this thread. Besides implementing move semantics, C++ with its rvalue refs also implicitly provides a way to distinguish between _mutable_ lvalue and rvalue references and so allows optimized implementations - that is something we'd also need in D, but that's what we've just covered with regard to the 2nd link. So I still don't see a valid reason to preclude binding rvalues to const ref parameters.
Re: Const ref and rvalues again...
11/7/2012 3:54 AM, Manu пишет:
If the compiler started generating 2 copies of all my ref functions, I'd
be rather unimpressed... bloat is already a problem in D. Perhaps this
may be a handy feature, but I wouldn't call this a 'solution' to this issue.
Also, what if the function is external (likely)... auto ref can't work
if the function is external, an implicit temporary is required in that case.
What's wrong with going this route:
void blah(auto ref X stuff){
...lots of code...
}
is magically expanded to:
void blah(ref X stuff){
...that code..
}
and
void blah(X stuff){
.blah(stuff); //now here stuff is L-value so use the ref version
}
Yeah, it looks _almost_ like a template now. But unlike with a template
we can assume it's 2 overloads _always_. External fucntion issue is
then solved by treating it as exactly these 2 overloads (one trampoline,
one real). Basically it becomes one-line declaration of 2 functions.
Given that temporaries are moved anyway the speed should be fine and
there is as much bloat as you'd do by hand.
Also hopefully inliner can be counted on to do its thing in this simple
case.
--
Dmitry Olshansky
Re: Const ref and rvalues again...
11/7/2012 9:04 PM, martin пишет:
On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:
C++:
void f(T a) { // for lvalues
this-resource = a.resource;
a.resetResource();
}
void f(T a) { // for rvalues (moved)
this-resource = a.resource;
a.resetResource();
}
D:
void f(ref T a) { // for lvalues
this.resource = a.resource;
a.resetResource();
}
void f(T a) { // rvalue argument is not copied, but moved
this.resource = a.resource;
a.resetResource();
}
You could probably get away with a single-line overload, both in C++ and D:
C++:
void f(T a) { // for lvalues
// convert a to mutable rvalue reference and
// invoke the main overload f(T)
f(std::move(a));
}
D:
void f(T a) { // rvalue argument is not copied, but moved
// the original argument is now named a (an lvalue)
// invoke the main overload f(ref T)
f(a);
}
Yup, and I'd like auto ref to actually do this r-value trampoline for me.
--
Dmitry Olshansky
Re: Const ref and rvalues again...
On Thursday, 8 November 2012 at 18:28:44 UTC, Dmitry Olshansky
wrote:
What's wrong with going this route:
void blah(auto ref X stuff){
...lots of code...
}
is magically expanded to:
void blah(ref X stuff){
...that code..
}
and
void blah(X stuff){
.blah(stuff); //now here stuff is L-value so use the ref
version
}
Yeah, it looks _almost_ like a template now. But unlike with a
template we can assume it's 2 overloads _always_. External
fucntion issue is then solved by treating it as exactly these 2
overloads (one trampoline, one real). Basically it becomes
one-line declaration of 2 functions.
Given that temporaries are moved anyway the speed should be
fine and there is as much bloat as you'd do by hand.
Also hopefully inliner can be counted on to do its thing in
this simple case.
That second overload for rvalues would be a shortcut to save the
lvalue declarations at each call site - and it really doesn't
matter if the compiler magically added the lvalue declarations
before each call or if it magically added the rvalue overload
(assuming all calls are inlined). But it would create a problem
if there already was an explicit 'void blah(X)' overload in
addition to 'void blah(auto ref X)' (not making much sense
obviously, but this would be something the compiler needed to
handle somehow).
What this 'auto ref' approach (both as currently implemented for
templates and proposed here for non-templated functions) lacks is
the vital distinction between const and mutable parameters.
For the much more common const ref parameters, I repeatedly tried
to explain why I'm absolutely convinced that we don't need
another keyword and that 'in/const ref' is sufficient, safe,
logical and intuitive (coupled with the overload rule that
pass-by-value (moving) is preferred for rvalues). Please prove me
wrong.
For the less common mutable ref parameters, I also repeatedly
tried to explain why I find it dangerous/unsafe to allow rvalues
to be bound to mutable ref parameters. But if there are enough
people wanting that, I'd have no problem with an 'auto ref'
approach for it (only for mutable parameters!). That may actually
be a good compromise, what do you guys think? :)
'auto ref T' for templates expands to 'ref T' (lvalues) and 'T'
(rvalues), duplicating the whole function and providing best
performance - no pointer/reference indirection for rvalues in
contrast to 'auto ref T' (proposed above) for non-templates,
otherwise the concept would be exactly the same. But it's only
for mutable parameters.
Such a templated option may also be worth for const parameters
though (expanding to 'const ref T' and 'const T'), so maybe
something like the (ambiguous) 'in/const auto ref T' wouldn't
actually be that bad (assuming there are only a few use cases,
and only for templates! It'd still be 'in ref T' for
non-templates).
Re: Const ref and rvalues again...
That's cute, but it really feels like a hack.
All of a sudden the debugger doesn't work properly anymore, you need to
step-in twice to enter the function, and it's particularly inefficient in
debug builds (a point of great concern for my industry!).
Please just with the compiler creating a temporary in the caller space.
Restrict is to const ref, or better, in ref (scope seems particularly
important here).
On 8 November 2012 20:28, Dmitry Olshansky dmitry.o...@gmail.com wrote:
11/7/2012 3:54 AM, Manu пишет:
If the compiler started generating 2 copies of all my ref functions, I'd
be rather unimpressed... bloat is already a problem in D. Perhaps this
may be a handy feature, but I wouldn't call this a 'solution' to this
issue.
Also, what if the function is external (likely)... auto ref can't work
if the function is external, an implicit temporary is required in that
case.
What's wrong with going this route:
void blah(auto ref X stuff){
...lots of code...
}
is magically expanded to:
void blah(ref X stuff){
...that code..
}
and
void blah(X stuff){
.blah(stuff); //now here stuff is L-value so use the ref version
}
Yeah, it looks _almost_ like a template now. But unlike with a template we
can assume it's 2 overloads _always_. External fucntion issue is then
solved by treating it as exactly these 2 overloads (one trampoline, one
real). Basically it becomes one-line declaration of 2 functions.
Given that temporaries are moved anyway the speed should be fine and there
is as much bloat as you'd do by hand.
Also hopefully inliner can be counted on to do its thing in this simple
case.
--
Dmitry Olshansky
Re: Const ref and rvalues again...
11/8/2012 11:30 PM, martin пишет:
On Thursday, 8 November 2012 at 18:28:44 UTC, Dmitry Olshansky wrote:
What's wrong with going this route:
void blah(auto ref X stuff){
...lots of code...
}
is magically expanded to:
void blah(ref X stuff){
...that code..
}
and
void blah(X stuff){
.blah(stuff); //now here stuff is L-value so use the ref version
}
Yeah, it looks _almost_ like a template now. But unlike with a
template we can assume it's 2 overloads _always_. External fucntion
issue is then solved by treating it as exactly these 2 overloads (one
trampoline, one real). Basically it becomes one-line declaration of 2
functions.
Given that temporaries are moved anyway the speed should be fine and
there is as much bloat as you'd do by hand.
Also hopefully inliner can be counted on to do its thing in this
simple case.
That second overload for rvalues would be a shortcut to save the lvalue
declarations at each call site - and it really doesn't matter if the
compiler magically added the lvalue declarations before each call or if
it magically added the rvalue overload (assuming all calls are inlined).
The scope. It's all about getting the correct scope, destructor call and
you know, the works. Preferably it can inject it inside temporary scope.
Anticipating bugs in the implementation of this feature let me warn that
re-writing this:
... code here ...
auto r = foo(SomeResource(x, y, ..)); //foo is auto ref
... code here ...
Should not change semantics e.g. imagine the resource is a lock, we'd
better unlock it sooner. That is call destructor right after foo
returns. So we need {} around the call. But this doesn't work as it
traps 'r':
{
auto someRef = SomeResource(x, y, ..);
auto r = foo(someRef);
}
So it's rather something like this:
typeof(foo(...)) r = void;
{
someRef = SomeResource(x, y, ..);
r = foo(someRef); // should in fact construct in place not assign
}
I suspect this is hackable to be more clean inside of the compiler but
not in terms of a re-write.
But it would create a problem if there already was an explicit 'void
blah(X)' overload in addition to 'void blah(auto ref X)' (not making
much sense obviously, but this would be something the compiler needed to
handle somehow).
Aye. But even then there is an ambiguity if there is one version of
function with ref T / T and one with auto ref T.
What this 'auto ref' approach (both as currently implemented for
templates and proposed here for non-templated functions) lacks is the
vital distinction between const and mutable parameters.
For the much more common const ref parameters, I repeatedly tried to
explain why I'm absolutely convinced that we don't need another keyword
and that 'in/const ref' is sufficient, safe, logical and intuitive
(coupled with the overload rule that pass-by-value (moving) is preferred
for rvalues). Please prove me wrong.
I'd rather restrict it to 'auto ref' thingie. Though 'in auto ref'
sounds outright silly.
Simply put const ref implies that callee can save a pointer to it
somewhere (it's l-value). The same risk is with 'auto ref' but at least
there an explicitly written 'disclaimer' by the author of accepting
temporary stuff.
In the ideal world name 'auto ref' would be shorter, logical and more to
the point but we have what we have.
For the less common mutable ref parameters, I also repeatedly tried to
explain why I find it dangerous/unsafe to allow rvalues to be bound to
mutable ref parameters. But if there are enough people wanting that, I'd
have no problem with an 'auto ref' approach for it (only for mutable
parameters!). That may actually be a good compromise, what do you guys
think? :)
I think that function plucked with auto ref is a enough indication that
author is fine with passing to it mutable r-values and not seeing
changes outside and related blah-blah. In most (all?) of cases it means
that parameter is too big to be passed by copy so rather it takes it by ref.
Also certain stuff can't be properly bitwise const because of C-calls
and what not. Logical const is the correct term but in the D world it's
simply mutable.
'auto ref T' for templates expands to 'ref T' (lvalues) and 'T'
(rvalues), duplicating the whole function and providing best performance
- no pointer/reference indirection for rvalues in contrast to 'auto ref
T' (proposed above) for non-templates, otherwise the concept would be
exactly the same. But it's only for mutable parameters.
I'd say that even for templates the speed argument is mostly defeated by
the bloat argument. But that's probably only me.
Such a templated option may also be worth for const parameters though
(expanding to 'const ref T' and 'const T'), so maybe something like the
(ambiguous) 'in/const auto ref T' wouldn't actually be that bad
(assuming there are only a few use cases, and only for templates! It'd
still be 'in ref T' for non-templates).
--
Dmitry Olshansky
Re: Const ref and rvalues again...
On Thursday, 8 November 2012 at 20:15:51 UTC, Dmitry Olshansky
wrote:
The scope. It's all about getting the correct scope, destructor
call and you know, the works. Preferably it can inject it
inside temporary scope.
typeof(foo(...)) r = void;
{
someRef = SomeResource(x, y, ..);
r = foo(someRef); // should in fact construct in place not
assign
}
I suspect this is hackable to be more clean inside of the
compiler but not in terms of a re-write.
Right, I forgot the scope for a moment. I'd illustrate the rvalue
= (const) ref binding to a novice language user as follows:
T const_foo( in ref int x);
T mutable_foo(auto ref int x);
int bar() { return 5; }
T result;
result = const_foo(bar());
/* expanded to:
{
immutable int tmp = bar(); // avoidable for literals
result = const_foo(tmp);
} // destruction of tmp
*/
result = mutable_foo(bar());
/* expanded to:
{
int tmp = bar();
result = mutable_foo(tmp);
} // destruction of tmp
*/
I'd rather restrict it to 'auto ref' thingie. Though 'in auto
ref' sounds outright silly.
Simply put const ref implies that callee can save a pointer to
it somewhere (it's l-value). The same risk is with 'auto ref'
but at least there an explicitly written 'disclaimer' by the
author of accepting temporary stuff.
'in ref' as opposed to 'const ref' should disallow this escaping
issue we've already tackled in this thread, but I'm not sure if
it is already/correctly implemented. Anyway, this issue also
arises with (short-lived) local lvalues at the caller site:
foreach (i; 0 .. 10)
{
int scopedLvalue = i + 2;
foo(scopedLvalue); // passed by ref
} // scopedLvalue is gone
In the ideal world name 'auto ref' would be shorter, logical
and more to the point but we have what we have.
+1, but I don't have a better proposal anyway. ;)
I think that function plucked with auto ref is a enough
indication that author is fine with passing to it mutable
r-values and not seeing changes outside and related blah-blah.
Agreed.
Also certain stuff can't be properly bitwise const because of
C-calls and what not. Logical const is the correct term but in
the D world it's simply mutable.
As you know, I'd definitely allow rvalues to be bound to const
ref parameters as alternative (that would also be useful for a
lot of existing code). People who generally don't use const
(Timon Gehr? :)) are free to only use 'auto ref', I'm most likely
only going to use 'in ref', and there will certainly be people
using both. Sounds like a really good compromise to me.
I'd say that even for templates the speed argument is mostly
defeated by the bloat argument. But that's probably only me.
I haven't performed any benchmarks, but I tend to agree with you,
especially since multiple 'auto ref' parameters lead to
exponential bloating. I could definitely do without a special
role for templates, which would further simplify things
considerably. If performance is really that critical, an explicit
pass-by-value (move) overload for rvalues ought to be enough
flexibility imo.
Re: Const ref and rvalues again...
On 11/08/2012 02:45 AM, martin wrote: On Wednesday, 7 November 2012 at 21:39:52 UTC, Timon Gehr wrote: You can pass him an object that does not support operations you want to preclude. He does not have to _know_, that your book is not changed when he reads it. This is an implementation detail. In fact, you could make the book save away his reading schedule without him noticing. I don't see where you want to go with this. Do you suggest creating tailored objects (book variants) for each function you're gonna pass it to just to satisfy perfect theoretical encapsulation? No. The point is that the language should _support_ what you call 'perfect theoretical encapsulation'. So foo() shouldn't be able to change the author = change from inout author reference to const reference? bar() should only be allowed to read the book title, not the actual book contents = hide that string? ;) For the sake of simplicity, by using const we have the ability to at least control if the object can be modified or not. It is not _just_ the object. Anyway, this is what I stated in my last post. So although my colleague doesn't have to _know_ that he can't modify my book in any way (or know that the book is modifiable in the first place), using const is a primitive but practical way for me to prevent him from doing so. It also weakens encapsulation, which was the point. In the context of this rvalue = (const) ref discussion, const is useful due to a number of reasons. 1) All possible side effects of the function invokation are required to be directly visible by the caller. Some people may find that annoying, but I find it useful, and there's a single-line workaround (lvalue declaration) for the (in my opinion very rare) cases where a potential side-effect is either known not to occur or simply uninteresting (requiring exact knowledge about the function implementation, always, i.e., during the whole life-time of that code!). Wrong. Not everything is a perfect value type. (and anyway, the code that actually will observe the change may be a few frames up the call stack.) 2) Say we pass a literal string (rvalue) to a const ref parameter. The location of the string in memory can then be freely chosen by the compiler, possibly in a static data segment of the binary (literal optimization - only one location for multiple occurrences). If the parameter was a mutable ref, the compiler should probably allocate a copy on the stack before calling the function, otherwise the literal may not be the same when accessed later on, potentially causing funny bugs. Ambiguous to me and all the interpretations are either wrong or irrelevant. 3) Implicit type conversion isn't a problem. Say we pass an int rvalue to a mutable double ref parameter. The parameter will then be a reference to another rvalue (the int cast to a double) and altering it (the hidden double rvalue) may not really be what the coder intended. Afaik D doesn't support implicit casting for user-defined types, so that may not be a problem (for now at least). Maybe you should stop trying to show that 'const' is sufficient for resolving those issues. The point is that it is not _necessary_. It is too strong.
Re: Const ref and rvalues again...
On Thursday, November 08, 2012 21:49:58 Manu wrote: That's cute, but it really feels like a hack. All of a sudden the debugger doesn't work properly anymore, you need to step-in twice to enter the function, and it's particularly inefficient in debug builds (a point of great concern for my industry!). Please just with the compiler creating a temporary in the caller space. Restrict is to const ref, or better, in ref (scope seems particularly important here). I honestly wish that in didn't exist in the language. The fact that it it's an alias two different attributes is confusing, and people keep using it without realizing what they're getting into. If scope worked correctly, you'd only want it in specific circumstances, not in general. And since it doesn't work correctly aside from delegates, once it _does_ work correctly, it'll break code all over the place, because people keep using in, because they like how it corresponds with out or whatever. - Jonathan M Davis
Re: Const ref and rvalues again...
On Thursday, 8 November 2012 at 22:34:03 UTC, Timon Gehr wrote: Ambiguous to me and all the interpretations are either wrong or irrelevant. My point is that it may affect performance. If there was no const, the compiler would need to allocate a dedicated copy of a literal whenever passing it to a mutable ref parameter unless the optimizer worked so well it can prove it's not going to be modified (which I'm sure you'd expect though :D). Maybe you should stop trying to show that 'const' is sufficient for resolving those issues. The point is that it is not _necessary_. It is too strong. In that case it actually is - who cares if the read-only double rvalue the function is passed is the result of an implicit cast (and there's a reason for it being implicit) of the original argument (int rvalue)? Anyway, I think we have moved on in this thread, so maybe you could contribute to trying to settle this rvalue = (const) ref issue once and for all by commenting my latest proposal.
Re: Const ref and rvalues again...
On Thursday, 8 November 2012 at 22:44:26 UTC, Jonathan M Davis wrote: I honestly wish that in didn't exist in the language. The fact that it it's an alias two different attributes is confusing, and people keep using it without realizing what they're getting into. If scope worked correctly, you'd only want it in specific circumstances, not in general. And since it doesn't work correctly aside from delegates, once it _does_ work correctly, it'll break code all over the place, because people keep using in, because they like how it corresponds with out or whatever. I agree that it may likely be a cause for future issues. I wouldn't remove it though, rather relax it to an alias for const only (yes, because I like how it corresponds with out (input only vs. output only) and especially because it is very short - this diff of 3 characters really make a difference in function signatures :D). That'd fortunately still be possible without breaking existing code. So please generalize my countless mentionings of 'in ref' to 'const ref'. ;)
Re: Const ref and rvalues again...
On 11/07/2012 04:13 AM, martin wrote:
On Wednesday, 7 November 2012 at 02:58:57 UTC, Era Scarecrow wrote:
Maybe a very simple change/addition; Like perhaps @ref? (Attribute
ref, Alternate ref, auto ref.. All sorta fit for it's meaning).
So...
void func1(ref int x); //D lvalue-only ref
void func2(@ref int x); //works like c++'s ref
Seems fairly easy to tell apart, and still leaves const-ness as an
option.
Afaik C++ doesn't allow rvalues to be passed to _mutable_ references
(T), only to const references, making perfect sense imo. I really do
not see the point for an additional syntax for C++-like const references
(const T) which would also take rvalues.
You are still missing that const in C++ is different from const in D.
Also, if the point is to have higher speed, why shouldn't the function
be allowed to use an rvalue as scratch space without a _deep copy_ ?
const in C++ does not mean anything. It is just loosely enforced
interface documentation. const in D actually restricts what the callee
can do with the argument, in a transitive fashion.
Please give me an example where you want to pass an rvalue to a
_mutable_ reference parameter.
When my struct does not support any operations that are const, and
creating a mutable copy is not possible due to indirections?
I would simply continue to disallow that,
since that would mean that changes to the referenced rvalue would not be
visible for the caller (a temporary/literal is changed - how could
someone possibly want that?).
The change may well be visible...
class C{
int x;
}
struct W{
C c;
}
W createW(C c){ return W(c); }
void foo(ref W w){ w.c.x = 2; }
void main(){
C c = new C;
assert(c.x==0);
foo(createW(c));
assert(c.x==2);
}
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:
You are still missing that const in C++ is different from const
in D. Also, if the point is to have higher speed, why shouldn't
the function be allowed to use an rvalue as scratch space
without a _deep copy_ ?
const in C++ does not mean anything. It is just loosely
enforced interface documentation. const in D actually restricts
what the callee can do with the argument, in a transitive
fashion.
Also depending on how you think of it, the const ref in C++
logically it didn't make sense to pass a mutable rvalue; Say you
pass the number 3, you can't modify it logically (ie you can't
redefine PI afterall).
When my struct does not support any operations that are const,
and creating a mutable copy is not possible due to indirections?
I would simply continue to disallow that, since that would
mean that changes to the referenced rvalue would not be
visible for the caller (a temporary/literal is changed - how
could someone possibly want that?).
The change may well be visible...
class C{
int x;
}
struct W{
C c;
}
W createW(C c){ return W(c); }
void foo(ref W w){ w.c.x = 2; }
void main(){
C c = new C;
assert(c.x==0);
foo(createW(c));
assert(c.x==2);
}
So basically being named (or not) doesn't change the data, what
it is or how it's used, only how accessible it is.
There's some forms of techniques that don't seem to have any
value until you have a chance to think about it and use them;
Like scopes, overloading, referencing, Exceptions you can easily
do without them all (We have C afterall), but it isn't as much
fun or clean. Some features and uses thereof make more sense for
the back/private implementation rather than the public interface.
The reason I gave my 'divide' example is because a while back I
wrote a BitInt (in C like 10 years ago, and a bit buggy); The
divide function actually calculated the remainder as a side
effect, and using wrappers or writing them as separate portions
would actually be harder and slower than joined as it was.
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 06:13:25 UTC, Rob T wrote:
ref T works fine, but if you wish to use f( ref T a ) on a temp
value returned from another function call, you'll need to
overload f() to pass by value, which means creating a duplicate.
Duplicating the function, yes, but not duplicating the rvalue
since it is moved in D. Isn't this analog to the C++ solution?
C++:
void f(T a) { // for lvalues
this-resource = a.resource;
a.resetResource();
}
void f(T a) { // for rvalues (moved)
this-resource = a.resource;
a.resetResource();
}
D:
void f(ref T a) { // for lvalues
this.resource = a.resource;
a.resetResource();
}
void f(T a) { // rvalue argument is not copied, but moved
this.resource = a.resource;
a.resetResource();
}
T g() {
T temp;
temp.createResource();
return temp; // D already moves temp!
// 'Named Return Value Optimization'
}
I tried f( move(g()) ) but that fails to work. My best guess is
that D does a hidden move of the temp instead of a copy to
value. I can't say for sure because the documentation is not
clear and is missing important details like this. I also cannot
rely on clever compiler optimizations that may or may not be
implemented as a guarantee.
You could implement a copy constructor 'this(this)' in your
struct T and see when it is invoked to check when an instance is
actually copied. I'd expect that invoking 'f(g());' with above
implementation doesn't copy anything.
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:
You are still missing that const in C++ is different from const
in D.
const in C++ does not mean anything. It is just loosely
enforced interface documentation. const in D actually restricts
what the callee can do with the argument, in a transitive
fashion.
I still don't get the big difference (except for transitiveness
for pointers). Given a const reference, I'm only able to invoke
methods decorated with the const keyword (or inout in D) keyword,
both in C++ and D. And I can only pass it as argument by ref to
functions which do not alter it (also taking a const reference,
that is).
Also, if the point is to have higher speed, why shouldn't the
function be allowed to use an rvalue as scratch space without a
_deep copy_ ?
I'm sorry but I don't get what you mean here. Could you please
elaborate on this?
When my struct does not support any operations that are const,
and creating a mutable copy is not possible due to indirections?
If your struct doesn't support any const operations, it most
likely has good reasons not to.
The change may well be visible...
True in this case, but only if you know exactly what foo() does
when you call it inside the main() function. And that is probably
an indicator for bad encapsulation - most of the time, you
shouldn't have the knowledge how foo() is exactly implemented
when using it from the outside.
It is clear that there are some examples where you want to pass
an rvalue argument to a mutable ref parameter if you know exactly
what the function does. But imo these cases are very rare and I
don't really regard it as big issue if you need to add a line
'auto tmp = myRvalue;' before the function call to transform it
to a referenceable lvalue, in these few cases.
Much more commonly, you need a parameter just as read-only input.
Consider a real-word-example of a 4x4 matrix consisting of 16
doubles (128 bytes). Most of the time, you'd only need a
read-only input instance when working with it (combining
matrices, transforming vectors etc.). Given its size (it's not
really huge, I acknowledge that ;)), you probably want to avoid
copying it around and therefore pass it by ref, but want that to
also work for rvalues (produced by matrix combinations like
'viewMatrix * modelMatrix', for example).
struct Matrix
{
double[16] data;
// this op= other
ref Matrix opOpAssign(string op)(in ref Matrix other);
// Matrix result = this op other
Matrix opBinary(string op)(in ref Matrix other) const;
// double4 result = this * vector
// the vector (32 bytes) may be passed by value for AVX
double4 opBinary(string op)(in ref double4 vector) const
if (op == *);
};
Re: Const ref and rvalues again...
On 11/07/2012 04:01 PM, martin wrote:
On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:
You are still missing that const in C++ is different from const in D.
const in C++ does not mean anything. It is just loosely enforced
interface documentation. const in D actually restricts what the callee
can do with the argument, in a transitive fashion.
I still don't get the big difference (except for transitiveness for
pointers).
That is the main thing, but C++ also has a 'mutable' keyword.
Given a const reference, I'm only able to invoke methods
decorated with the const keyword (or inout in D) keyword, both in C++
and D. And I can only pass it as argument by ref to functions which do
not alter it (also taking a const reference, that is).
This is not the case in C++ for the D definition of 'not alter'.
Also, if the point is to have higher speed, why shouldn't the function
be allowed to use an rvalue as scratch space without a _deep copy_ ?
I'm sorry but I don't get what you mean here. Could you please elaborate
on this?
Well, if the changes are not visible, the function can freely change the
memory in order to perform its computations.
When my struct does not support any operations that are const, and
creating a mutable copy is not possible due to indirections?
If your struct doesn't support any const operations, it most likely has
good reasons not to.
Either that, or someone hasn't bothered to annotate.
The change may well be visible...
True in this case, but only if you know exactly what foo() does when you
call it inside the main() function. And that is probably an indicator
for bad encapsulation
The method is called 'foo', that means it is a toy example to show that
the change may be visible. The change does not necessarily have to be
visible to the caller directly.
- most of the time, you shouldn't have the
knowledge how foo() is exactly implemented when using it from the outside.
You got it exactly reverse. const harms encapsulation because it exposes
some details about implementations. It is often not applicable in
sufficiently dynamic code.
It is clear that there are some examples where you want to pass an
rvalue argument to a mutable ref parameter if you know exactly what the
function does. But imo these cases are very rare and I don't really
regard it as big issue if you need to add a line 'auto tmp = myRvalue;'
before the function call to transform it to a referenceable lvalue, in
these few cases.
You do not see issues with changing the interface based on
implementation details?
Much more commonly, you need a parameter just as read-only input.
Consider a real-word-example of a 4x4 matrix consisting of 16 doubles
(128 bytes). Most of the time, you'd only need a read-only input
instance when working with it (combining matrices, transforming vectors
etc.). Given its size (it's not really huge, I acknowledge that ;)), you
probably want to avoid copying it around and therefore pass it by ref,
but want that to also work for rvalues (produced by matrix combinations
like 'viewMatrix * modelMatrix', for example).
struct Matrix
{
double[16] data;
// this op= other
ref Matrix opOpAssign(string op)(in ref Matrix other);
// Matrix result = this op other
Matrix opBinary(string op)(in ref Matrix other) const;
// double4 result = this * vector
// the vector (32 bytes) may be passed by value for AVX
double4 opBinary(string op)(in ref double4 vector) const
if (op == *);
};
I know your use case, and note that I do not have any issues with
rvalues always being allowed to bind to ref parameters, which would
solve your problem in all cases. (others do not like that though, which
is why we have the current situation and your C++ - inspired design is
discussed every second week or so.)
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:
C++:
void f(T a) { // for lvalues
this-resource = a.resource;
a.resetResource();
}
void f(T a) { // for rvalues (moved)
this-resource = a.resource;
a.resetResource();
}
D:
void f(ref T a) { // for lvalues
this.resource = a.resource;
a.resetResource();
}
void f(T a) { // rvalue argument is not copied, but moved
this.resource = a.resource;
a.resetResource();
}
You could probably get away with a single-line overload, both in
C++ and D:
C++:
void f(T a) { // for lvalues
// convert a to mutable rvalue reference and
// invoke the main overload f(T)
f(std::move(a));
}
D:
void f(T a) { // rvalue argument is not copied, but moved
// the original argument is now named a (an lvalue)
// invoke the main overload f(ref T)
f(a);
}
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 16:57:47 UTC, Timon Gehr wrote: That is the main thing, but C++ also has a 'mutable' keyword. Right, I forgot this inconspicuous little keyword. It really is a huge hole in C++'s const system. If your struct doesn't support any const operations, it most likely has good reasons not to. Either that, or someone hasn't bothered to annotate. Hehe, yeah, the latter being more likely. You got it exactly reverse. const harms encapsulation because it exposes some details about implementations. It is often not applicable in sufficiently dynamic code. I don't see it that way. I find it very useful to know that an argument won't be modified. Example: suppose we have a large array (static array or wrapped in a struct) and need it as input for some independent operations. By knowing it won't be touched (passing by const ref) it is immediately clear that the operations can be run in parallel. Otherwise, one would need to go through the operations' implementation to determine if parallelization is possible. You do not see issues with changing the interface based on implementation details? I find it okay to require all possible changes to an argument passed by reference to be required to be directly visible by the caller, just to prevent accidentally missing side effects. Suppose a function takes a const reference parameter and is called at a site using an rvalue. After some time, that function is updated so that its parameter is or may be changed, changing the const ref parameter to a ref parameter. That may lead to big issues for the call sites. A compiler error would make sure the affected call sites are inspected and updated instead of potentially introducing regressions.
Re: Const ref and rvalues again...
On 11/07/2012 06:52 PM, martin wrote: On Wednesday, 7 November 2012 at 16:57:47 UTC, Timon Gehr wrote: That is the main thing, but C++ also has a 'mutable' keyword. Right, I forgot this inconspicuous little keyword. It really is a huge hole in C++'s const system. If your struct doesn't support any const operations, it most likely has good reasons not to. Either that, or someone hasn't bothered to annotate. Hehe, yeah, the latter being more likely. You got it exactly reverse. const harms encapsulation because it exposes some details about implementations. It is often not applicable in sufficiently dynamic code. I don't see it that way. You do not give a justification. I find it very useful to know that an argument won't be modified. That is unrelated and does not necessarily follow, but that it does in some cases is the main reason why const is useful even though it weakens encapsulation. Example: suppose we have a large array (static array or wrapped in a struct) and need it as input for some independent operations. By knowing it won't be touched (passing by const ref) it is immediately clear that the operations can be run in parallel. Otherwise, one would need to go through the operations' implementation to determine if parallelization is possible. You have to do that anyway if it the operation is not at least 'const pure' and you need to guarantee that no other thread is in fact modifying the data. But even then I do not see how that helps encapsulation. Naive parallelization is just one of the examples that shows that encapsulation can be harmful in some specific cases. You do not see issues with changing the interface based on implementation details? I find it okay to require all possible changes to an argument passed by reference to be required to be directly visible by the caller, just to prevent accidentally missing side effects. Suppose a function takes a const reference parameter and is called at a site using an rvalue. After some time, that function is updated so that its parameter is or may be changed, changing the const ref parameter to a ref parameter. That may lead to big issues for the call sites. A compiler error would make sure the affected call sites are inspected and updated instead of potentially introducing regressions. Often there is nothing to be seen for the caller. (lazy initialization, caching, internal state update part of a larger computation, ...)
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 18:07:27 UTC, Timon Gehr wrote: You do not give a justification. I find it very useful to know that an argument won't be modified. That is unrelated and does not necessarily follow, but that it does in some cases is the main reason why const is useful even though it weakens encapsulation. Let's look at encapsulation this way: suppose my function is a colleague and his job is to summarize a book I lend him (my reference parameter). I don't care how he does it as long as the summary (function result) is what I expect. But I do care about my book if I plan on using it afterwards (so in case the argument is an lvalue used later on) - will I have to check it for defects, coffee spillings etc., or can I be sure it's exactly as it was before (const)? This is how I see it. The thing about mutable rvalue references would be unknown or unthought-of side effects: what if the book is the same, but my colleague, a known psychopath, killed the author (a property of the book reference, a reference itself)? I don't use the book anymore, okay (it was an rvalue after all), but it may still be worth knowing that the author has gone. Due to the transitiveness of const, the (const) author would still be alive if the book reference had been const. You have to do that anyway if it the operation is not at least 'const pure' and you need to guarantee that no other thread is in fact modifying the data. I think we can agree that knowing the parameters aren't touched does help a lot in this case.
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:
T g() {
T temp;
temp.createResource();
return temp; // D already moves temp!
// 'Named Return Value Optimization'
}
OK, it seems to be working as you describe.
The main question I have is if I can rely on this behaviour as a
feature provided by the languge specification vs an opportunistic
compiler optimization that may or may not happen depending on the
implementation?
The general problem I'm having with with D, is I need a gaurantee
that certain behaviors will always be performed when moves/copies
are done, but it's just not clear if this is the case or not.
There seems to be a lack of a concise language specification, or
is there one defined somewhere?
You could implement a copy constructor 'this(this)' in your
struct T and see when it is invoked to check when an instance
is actually copied. I'd expect that invoking 'f(g());' with
above implementation doesn't copy anything.
I am doing that, but I cannpt seem to hook into the part where a
move does an init on the struct or class. I sort-of can see it
though, if I set a member value to a known state before the move,
then display it to console or log after the move during
destruction. If it was init'ed then I'll see the default value,
and this seems to be happening as described.
Anyway, what I was hoping for with auto ref was for the
compiler to selectively decide if the ref part should be used or
not depending on the situation, rather than me manually writing
two functions to do the exact same thing. What's discussed in
here about auto ref is something else, although I agree it is
definitely needed.
--rt
Re: Const ref and rvalues again...
On 11/07/2012 07:48 PM, martin wrote: On Wednesday, 7 November 2012 at 18:07:27 UTC, Timon Gehr wrote: You do not give a justification. I find it very useful to know that an argument won't be modified. That is unrelated and does not necessarily follow, but that it does in some cases is the main reason why const is useful even though it weakens encapsulation. Let's look at encapsulation this way: suppose my function is a colleague and his job is to summarize a book I lend him (my reference parameter). I don't care how he does it as long as the summary (function result) is what I expect. But I do care about my book if I plan on using it afterwards (so in case the argument is an lvalue used later on) - will I have to check it for defects, coffee spillings etc., or can I be sure it's exactly as it was before (const)? You can pass him an object that does not support operations you want to preclude. He does not have to _know_, that your book is not changed when he reads it. This is an implementation detail. In fact, you could make the book save away his reading schedule without him noticing. This is how I see it. The thing about mutable rvalue references would be unknown or unthought-of side effects: what if the book is the same, but my colleague, a known psychopath, killed the author (a property of the book reference, a reference itself)? I don't use the book anymore, okay (it was an rvalue after all), but it may still be worth knowing that the author has gone. Due to the transitiveness of const, the (const) author would still be alive if the book reference had been const. I'd assume that the book you pass him would not support changing the author. You have to do that anyway if it the operation is not at least 'const pure' and you need to guarantee that no other thread is in fact modifying the data. I think we can agree that knowing the parameters aren't touched does help a lot in this case. Maybe, but the fact that we know it harms encapsulation.
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 21:39:52 UTC, Timon Gehr wrote: You can pass him an object that does not support operations you want to preclude. He does not have to _know_, that your book is not changed when he reads it. This is an implementation detail. In fact, you could make the book save away his reading schedule without him noticing. I don't see where you want to go with this. Do you suggest creating tailored objects (book variants) for each function you're gonna pass it to just to satisfy perfect theoretical encapsulation? So foo() shouldn't be able to change the author = change from inout author reference to const reference? bar() should only be allowed to read the book title, not the actual book contents = hide that string? ;) For the sake of simplicity, by using const we have the ability to at least control if the object can be modified or not. So although my colleague doesn't have to _know_ that he can't modify my book in any way (or know that the book is modifiable in the first place), using const is a primitive but practical way for me to prevent him from doing so. In the context of this rvalue = (const) ref discussion, const is useful due to a number of reasons. 1) All possible side effects of the function invokation are required to be directly visible by the caller. Some people may find that annoying, but I find it useful, and there's a single-line workaround (lvalue declaration) for the (in my opinion very rare) cases where a potential side-effect is either known not to occur or simply uninteresting (requiring exact knowledge about the function implementation, always, i.e., during the whole life-time of that code!). 2) Say we pass a literal string (rvalue) to a const ref parameter. The location of the string in memory can then be freely chosen by the compiler, possibly in a static data segment of the binary (literal optimization - only one location for multiple occurrences). If the parameter was a mutable ref, the compiler should probably allocate a copy on the stack before calling the function, otherwise the literal may not be the same when accessed later on, potentially causing funny bugs. 3) Implicit type conversion isn't a problem. Say we pass an int rvalue to a mutable double ref parameter. The parameter will then be a reference to another rvalue (the int cast to a double) and altering it (the hidden double rvalue) may not really be what the coder intended. Afaik D doesn't support implicit casting for user-defined types, so that may not be a problem (for now at least).
Re: Const ref and rvalues again...
On Thursday, October 18, 2012 05:07:52 Malte Skarupke wrote: What do you think? Okay. Here are more links to Andrei discussing the problem: http://forum.dlang.org/post/4f83dbe5.20...@erdani.com http://www.mail-archive.com/digitalmars-d@puremagic.com/msg44070.html http://www.mail-archive.com/digitalmars-d@puremagic.com/msg43769.html http://forum.dlang.org/post/hg62rq$2c2n$1...@digitalmars.com He is completely convinced that letting rvalues bind to const in C++ was a huge mistake, and it seems to come down mainly to this: The problem with binding rvalues to const ref is that once that is in place you have no way to distinguish an rvalue from a const ref on the callee site. And apparenly it required adding the concept of rvalue references to C++, which complicated things considerably. It's too bad that he hasn't replied to some of the more detailed questions on the matter in this thread, but if you want rvalues to bind to const ref in D, you're going to have to convince him. At this point, I expect that the most likely solution is that auto ref will continue to work like it does in templates but for non-templated functions it will become like const ref is in C++ and accept rvalues without copying lvalues. Andrei suggests that in at least one of those posts, and AFAIK, it would work just fine. The only real downside that I'm aware of is that then the semantics of auto ref are slightly different for non-templated functions, but you're doing basically the same thing with auto ref in both cases - avoiding copying lvalues and only copying rvalues when you have to - so it really shouldn't be a problem. The main problem is that someone needs to go and implement it. - Jonathan M Davis
Re: Const ref and rvalues again...
Yeah I really don't understand how 'auto ref' entered this conversation? It completely misses the point. The issue as I see it is this: void func(ref in Vector m); func(v1*v2 + Vector(10, 20, 30)); And countless situations like it, all leading to innumerable temp locals with irrelevant names: Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30); func(v1v2PlusSomeStuff); Or even: Vector initialiser1 = Vector(10, 20, 30); // - words can scarcely describe how annoying this is func(initialiser1); Allowing rvalues to be passed by ref is exactly as unsafe as those examples, and that workaround is employed every time anyway, so the restriction is self-defeating. On 5 November 2012 10:01, Jonathan M Davis jmdavisp...@gmx.com wrote: On Monday, November 05, 2012 06:05:07 Rob T wrote: On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis wrote: And when we argued for altering it so that it operated like const ref in C++ (which allows const ref in D to continue to function like it does now), some folks complained, because they've found the current semantics of auto ref to be useful (something to do with propagating the exact, original type, I think). I would expect that auto ref for a template and for a non template should work in exactly the same way, so why would there be a difference? If there must be a difference, there should be different semantics for specifying the difference, otherwise the inconsistent behaviours among identical semantics will only serve to confuse people. auto ref's current semantics can't possibly work with non-templated functions, and it's clear that there are good reasons for keeping auto ref as it is now for templates. So, either we make it work with non-templated functions with different (albeit similar) semantics, or we need a new attribute for doing what we want to do with non-templated functions. - Jonathan M Davis
Re: Const ref and rvalues again...
On Tuesday, November 06, 2012 20:40:38 Manu wrote: Yeah I really don't understand how 'auto ref' entered this conversation? It completely misses the point. The _entire_ reason that auto ref was introduced in the first place was to solve the const ref problem. That's what it's for. It wouldn't exist otherwise. So, it's completely relevant to any discussion of const ref. However, Walter misunderstood what Andrei meant by the feature, so it only got implemented for templates, and given how Walter implemented it, it _can't_ be implemented with non-templated functions. But it could be implemented with slightly different semantics such that it solves the const ref problem as orignally intended. It's just that it would then have slightly different semantics between templated and non-templated functions. - Jonathan M Davis
Re: Const ref and rvalues again...
But it only really makes sense in the context of templates...? Why should something called 'auto ref' provide fabrication of temporaries for the purpose of passing rvalues to functions that receive ref args? How does auto ref (under some different definition/implementation) address the rvalue problem? On 6 November 2012 23:08, Jonathan M Davis jmdavisp...@gmx.com wrote: On Tuesday, November 06, 2012 20:40:38 Manu wrote: Yeah I really don't understand how 'auto ref' entered this conversation? It completely misses the point. The _entire_ reason that auto ref was introduced in the first place was to solve the const ref problem. That's what it's for. It wouldn't exist otherwise. So, it's completely relevant to any discussion of const ref. However, Walter misunderstood what Andrei meant by the feature, so it only got implemented for templates, and given how Walter implemented it, it _can't_ be implemented with non-templated functions. But it could be implemented with slightly different semantics such that it solves the const ref problem as orignally intended. It's just that it would then have slightly different semantics between templated and non-templated functions. - Jonathan M Davis
Re: Const ref and rvalues again...
On Tuesday, 6 November 2012 at 22:32:57 UTC, Manu wrote: But it only really makes sense in the context of templates...? Why should something called 'auto ref' provide fabrication of temporaries for the purpose of passing rvalues to functions that receive ref args? How does auto ref (under some different definition/implementation) address the rvalue problem? The thing is that currently there are 2 workarounds. You described the first one - allocating temporaries manually to obtain referenceable lvalues: void func(ref in Vector m); Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30); func(v1v2PlusSomeStuff); The other, also painfully annoying workaround is overloading func: void func(in ref Vector m); void func(in Vector m); func(v1*v2 + Vector(10, 20, 30)); 'auto ref' implements the second workaround (via a template) and therefore requires a single, but templated func() implementation: void func(T)(in auto ref T m); This template, as I understand it, gets expanded to: void func(T)(in ref T m); // for lvalues void func(T)(in T m); // for rvalues So for non-templated functions, I suggest 2 options: 1) The previously described auto-templates (identical 'auto ref' semantics), where a function with 'auto ref' parameters is treated as implicit template. This may lead to code-bloating (for larger functions) and/or higher performance for rvalue arguments (rvalues passed to value arguments are moved, not copied; we therefore gain nothing by passing a reference, but incur a slight performance hit due to pointer indirection instead of accessing directly the rvalue on the stack). OR 2) Simple under-the-hood temporary lvalue declaration by the compiler (for rvalues passed to 'const ref' parameters) - that would be a handy implementation of the first workaround. I hope you get my point. :)
Re: Const ref and rvalues again...
On Wednesday, November 07, 2012 00:32:48 Manu wrote: But it only really makes sense in the context of templates...? Why should something called 'auto ref' provide fabrication of temporaries for the purpose of passing rvalues to functions that receive ref args? How does auto ref (under some different definition/implementation) address the rvalue problem? The idea behind auto ref was to create a function which would take its argument in the most efficient way possible, whatever that was (be it ref or by value), and that the compiler would figure that out. That way, the function wouldn't care whether it was given an rvalue or lvalue or a value type or reference type or whatever. What actually got implemented was simply duplicating functions with auto ref so that there's a ref version of lvalues and non-ref versions for rvalues, which really doesn't do the job and only works with templates. What we need is an attribute which indicates that you don't care whether the argument is an lvalue or rvalue. You just don't want it to be copied, and you don't want the function to mutate it - which is what const in C++ is normally used for. Whether auto ref is the attribute used for that is more or less irrelevant except for the fact that that's the problem that it was trying to solve in the first place. Making auto ref do that for non-templated functions would solve the problem for non-templated functions but would mean that auto ref on parameters in templated functions and non-templated functions would have different (albeit similar) semantics, which would be a bit of problem and may or may not be acceptable. And I'd have to think it through more to say whether or not auto ref as it is on templated functions really solves the problem well enough for templated functions or not. If it does, then it may be acceptable to just have auto ref on non-templated functions with slightly different semantics. If it doesn't, then we either need to change what auto ref does with templated functions to match what we want for non-templated ones (which could be a problem, since some people have found the current semantics of auto ref useful for other stuff like propagating attributes), or we'd need to use a new attribute rather than auto ref. - Jonathan M Davis
Re: Const ref and rvalues again...
If the compiler started generating 2 copies of all my ref functions, I'd be rather unimpressed... bloat is already a problem in D. Perhaps this may be a handy feature, but I wouldn't call this a 'solution' to this issue. Also, what if the function is external (likely)... auto ref can't work if the function is external, an implicit temporary is required in that case. On 7 November 2012 01:37, martin ki...@libero.it wrote: On Tuesday, 6 November 2012 at 22:32:57 UTC, Manu wrote: But it only really makes sense in the context of templates...? Why should something called 'auto ref' provide fabrication of temporaries for the purpose of passing rvalues to functions that receive ref args? How does auto ref (under some different definition/implementation) address the rvalue problem? The thing is that currently there are 2 workarounds. You described the first one - allocating temporaries manually to obtain referenceable lvalues: void func(ref in Vector m); Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30); func(v1v2PlusSomeStuff); The other, also painfully annoying workaround is overloading func: void func(in ref Vector m); void func(in Vector m); func(v1*v2 + Vector(10, 20, 30)); 'auto ref' implements the second workaround (via a template) and therefore requires a single, but templated func() implementation: void func(T)(in auto ref T m); This template, as I understand it, gets expanded to: void func(T)(in ref T m); // for lvalues void func(T)(in T m); // for rvalues So for non-templated functions, I suggest 2 options: 1) The previously described auto-templates (identical 'auto ref' semantics), where a function with 'auto ref' parameters is treated as implicit template. This may lead to code-bloating (for larger functions) and/or higher performance for rvalue arguments (rvalues passed to value arguments are moved, not copied; we therefore gain nothing by passing a reference, but incur a slight performance hit due to pointer indirection instead of accessing directly the rvalue on the stack). OR 2) Simple under-the-hood temporary lvalue declaration by the compiler (for rvalues passed to 'const ref' parameters) - that would be a handy implementation of the first workaround. I hope you get my point. :)
Re: Const ref and rvalues again...
Thanks Jonathan for the detailed info. So 'auto ref' is implemented the way I thought, by simply duplicating functions. That is actually the only way (I can think of) to solve your problem 'pass this argument in the most efficient way possible (do not copy)'. But that is not the only problem. More commonly, we want to avoid copying lvalues for read-only parameters and therefore pass by reference. At the same time, we want to use the same function for rvalues - we simply can't stand having to declare temporaries manually or having to overload functions! Since the parameter is read-only, we couldn't care less if the argument is actually an lvalue or an rvalue. The only thing is that a dedicated function overload for rvalues would be slightly more efficient (for rvalues only) since the rvalue would be accessed directly on the stack instead of having to access it indirectly via its reference. This problem here would be 'pass this argument as read-only in the most efficient way possible for lvalues (do not copy) but also allow rvalues at a slight cost (compared to a dedicated function overload)'. So we do not really need 'auto ref' for non-templated functions or a new, even more confusing keyword which you, Jonathan, seem to insist on - 'const ref' (or, more elegantly 'in ref') is all we need. We simply want the compiler to automatically declare rvalues passed to 'const ref' parameters as local lvalues - problem solved for the majority of cases. For the remaining few which care about the involved cost for rvalue arguments, allow them to provide a function overload for rvalues (either manually or via 'auto ref' templates) - problem solved completely in my point of view. And that is actually exactly what the thread starter Malte proposes (and I already pointed that out in an earlier post). Again hoping to be clear enough. :)
Re: Const ref and rvalues again...
On Wednesday, November 07, 2012 01:32:03 martin wrote: So we do not really need 'auto ref' for non-templated functions or a new, even more confusing keyword which you, Jonathan, seem to insist on - 'const ref' (or, more elegantly 'in ref') is all we need. No, because that would be doing the same thing as C++, which Walter and Andrei have already rejected. They specifically do _not_ want there to be any ambiguity between whether a const ref variable is an lvalue or rvalue. If they were willing to make const ref work the same as C++'s const, then we would never have had this problem in the first place. We specifically need something other than const ref. The const ref can continue to work as it does now, but we'll have a way to get semantics similar to C++'s const when we want them. - Jonathan M Davis
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 00:47:04 UTC, Jonathan M Davis wrote: No, because that would be doing the same thing as C++, which Walter and Andrei have already rejected. They specifically do _not_ want there to be any ambiguity between whether a const ref variable is an lvalue or rvalue. If they were willing to make const ref work the same as C++'s const, then we would never have had this problem in the first place. We specifically need something other than const ref. The const ref can continue to work as it does now, but we'll have a way to get semantics similar to C++'s const when we want them. Wow, now this finally sheds some light on this topic. Do you have any links regarding their reasoning? I simply and utterly fail to comprehend the need for a compiler to distinguish between a const reference to a named variable and a const reference to an unnamed temporary/literal (which the compiler is free to allocate wherever it makes most sense). Especially since the compiler could simply allocate and name the original rvalue as local variable on the caller's stack before the function call and hence transform it automatically to an lvalue, i.e., do that, what currently most of us have to painfully do by hand.
Re: Const ref and rvalues again...
On Wednesday, November 07, 2012 02:15:26 martin wrote: Wow, now this finally sheds some light on this topic. Do you have any links regarding their reasoning? The most recent discussion where Walter and Andrei were part of the discussion was here: http://forum.dlang.org/post/4f84d6dd.5090...@digitalmars.com A key component of the problem is that conversions cause problems. Also, IIRC allowing const ref to take rvalues causes some issues with overloading. Making some changes to ref with regards to escaping apparently could change the situation a bit, and there's even some discussion in there of allowing ref to take rvalues if certain other things in the language are changed, but it's all up in the air at this point. Certainly, it's not a simple matter of just making const ref work with rvalues like most of the people coming from C++ want and expect. - Jonathan M Davis
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 00:32:04 UTC, martin wrote:
So we do not really need 'auto ref' for non-templated functions
or a new, even more confusing keyword which you, Jonathan, seem
to insist on - 'const ref' (or, more elegantly 'in ref') is all
we need. We simply want the compiler to automatically declare
rvalues passed to 'const ref' parameters as local lvalues -
problem solved for the majority of cases. For the remaining few
which care about the involved cost for rvalue arguments, allow
them to provide a function overload for rvalues (either
manually or via 'auto ref' templates) - problem solved
completely in my point of view. And that is actually exactly
what the thread starter Malte proposes (and I already pointed
that out in an earlier post).
'in ref' may work, but what it implies doesn't quite make sense.
'auto ref' already being used for templates should be able to
carry over to functions, however auto in the same section along
with type declaration may seem a little confusing. Let's glance
at a few of how they would be called.
//either
void func(int x);
void func(const int x);
//lvalue
void func(ref int x);
void func(const ref int x);
//either? auto ref/in ref
void func(auto ref int x);
void func(const auto ref int x);
void func(in ref int x);
Assuming auto can't be used to do type referencing the 'const
auto ref int' seems rather long. alternatively perhaps inout?
(accepts any type, partially skewed?)
void func(inout ref int x);
void func(const inout ref int x);
At this point for const it's just as large as the const auto
ref.. So..
void func(inout ref int x); //mutable
void func(in ref int x);//const version
void func(out ref int x); //optional write-only variable
'out ref int x' could then be legal.. Somehow reminds me of
zlib's compress utility function that returns how many bytes the
output buffer used but sometimes you didn't care. So.. if we
defined and used...
void func(out ref int x) {
x = 100;
}
int x;
func(x);
assert(x == 100);
func(0);//0 and null (and legal) basically ignored.
func(null);
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 01:15:27 UTC, martin wrote: I simply and utterly fail to comprehend the need for a compiler to distinguish between a const reference to a named variable and a const reference to an unnamed temporary/literal (which the compiler is free to allocate wherever it makes most sense). Let me be more precise: I don't see why a called function would need to know whether a passed const reference is named or unnamed at the caller's site. The developer's reasoning for a const reference is simply to avoid costly copying of the argument to be passed (only applicable for lvalues) which is guaranteed not to be modified by the callee. Decorating such a parameter with 'const ref' seems as logical as it gets (don't pass me a copy of the real thing, pass me a reference instead - I'm not going to modify it). The only reason I can think of to distinguish between lvalues and rvalues at the callee is the slight performance hit due to pointer indirection for rvalues. And that would be addressable by preferring by-value-passing of rvalues when selecting between function overloads.
Re: Const ref and rvalues again...
On Tuesday, 6 November 2012 at 23:37:25 UTC, martin wrote: void func(T)(in auto ref T m); This template, as I understand it, gets expanded to: void func(T)(in ref T m); // for lvalues void func(T)(in T m); // for rvalues So for non-templated functions, I suggest 2 options: 1) The previously described auto-templates (identical 'auto ref' semantics), where a function with 'auto ref' parameters is treated as implicit template. This may lead to code-bloating (for larger functions) and/or higher performance for rvalue arguments (rvalues passed to value arguments are moved, not copied; we therefore gain nothing by passing a reference, but incur a slight performance hit due to pointer indirection instead of accessing directly the rvalue on the stack). OR 2) Simple under-the-hood temporary lvalue declaration by the compiler (for rvalues passed to 'const ref' parameters) - that would be a handy implementation of the first workaround. I hope you get my point. :) What about the case where we want to pass a source argument either by reference or as a copy depending on the l/r value situation? eg void f( ref a ); void f( a ); --rt
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 01:33:49 UTC, Jonathan M Davis wrote: The most recent discussion where Walter and Andrei were part of the discussion was here: http://forum.dlang.org/post/4f84d6dd.5090...@digitalmars.com That thread is quite misleading and, I'm sad to say, not very useful (rather damaging to this discussion) in my opinion - especially because the distinction between rvalue = 'const ref' and rvalue = ref is largely neglected, and that distinction is of extremely high importance, I can't stress that enough. Walter's 3 C++ examples (2 of them invalid anyway afaik) don't relate to _const_ references. The implicit type conversion problem in that thread isn't a problem for _const_ references, just to point out one tiny aspect. rvalue = ref/out propagation makes no sense imho, as does treating literals as lvalues (proposed by Walter iirc). The current 'auto ref' semantics also fail to cover the special role of _const_ references for rvalues (also illustrated by Scarecrow's post). Certainly, it's not a simple matter of just making const ref work with rvalues like most of the people coming from C++ want and expect. Well I absolutely do _not_ share this point of view. It just seems so logical to me. I'm still waiting for a plausible argument to prove me wrong. All the required info is in this thread, e.g., we covered the escaping issue you mentioned.
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 02:06:09 UTC, Rob T wrote: What about the case where we want to pass a source argument either by reference or as a copy depending on the l/r value situation? eg void f( ref a ); void f( a ); --rt I don't get what you mean - that's why the 2 overloads are for (you forgot the const/in keyword - beware! :)) void f( in ref T a ); void f( in T a ); rvalue binds to the latter overload (argument not copied, but moved directly). lvalue binds to the first overload (reference). Works with v2.060.
Re: Const ref and rvalues again...
On Tuesday, 6 November 2012 at 23:37:25 UTC, martin wrote: 1) The previously described auto-templates (identical 'auto ref' semantics), where a function with 'auto ref' parameters is treated as implicit template. This may lead to code-bloating (for larger functions) and/or higher performance for rvalue arguments (rvalues passed to value arguments are moved, not copied; we therefore gain nothing by passing a reference, but incur a slight performance hit due to pointer indirection instead of accessing directly the rvalue on the stack). OR 2) Simple under-the-hood temporary lvalue declaration by the compiler (for rvalues passed to 'const ref' parameters) - that would be a handy implementation of the first workaround. Please discard option 1, I'm afraid it creates too much confusion and was not well thought through. The objective was to expand 'auto ref T' to either 'in ref T' for lvalues or 'in T' for rvalues (i.e., only for const parameters!), but then its caption would be horribly misleading (and something like 'in auto ref' would be way too ambiguous).
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 02:45:15 UTC, martin wrote: Please discard option 1, I'm afraid it creates too much confusion and was not well thought through. The objective was to expand 'auto ref T' to either 'in ref T' for lvalues or 'in T' for rvalues (I.e., only for const parameters!), but then its caption would be horribly misleading (and something like 'in auto ref' would be way too ambiguous). Maybe a very simple change/addition; Like perhaps @ref? (Attribute ref, Alternate ref, auto ref.. All sorta fit for it's meaning). So... void func1(ref int x); //D lvalue-only ref void func2(@ref int x); //works like c++'s ref Seems fairly easy to tell apart, and still leaves const-ness as an option.
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 02:58:57 UTC, Era Scarecrow wrote: Maybe a very simple change/addition; Like perhaps @ref? (Attribute ref, Alternate ref, auto ref.. All sorta fit for it's meaning). So... void func1(ref int x); //D lvalue-only ref void func2(@ref int x); //works like c++'s ref Seems fairly easy to tell apart, and still leaves const-ness as an option. Afaik C++ doesn't allow rvalues to be passed to _mutable_ references (T), only to const references, making perfect sense imo. I really do not see the point for an additional syntax for C++-like const references (const T) which would also take rvalues. Please give me an example where you want to pass an rvalue to a _mutable_ reference parameter. I would simply continue to disallow that, since that would mean that changes to the referenced rvalue would not be visible for the caller (a temporary/literal is changed - how could someone possibly want that?).
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 03:13:22 UTC, martin wrote:
void func1(ref int x); //D lvalue-only ref
void func2(@ref int x); //works like c++'s ref
Seems fairly easy to tell apart, and still leaves const-ness
as an option.
Afaik C++ doesn't allow rvalues to be passed to _mutable_
references (T), only to const references, making perfect sense
imo. I really do not see the point for an additional syntax for
C++-like const references (const T) which would also take
rvalues.
Please give me an example where you want to pass an rvalue to a
_mutable_ reference parameter. I would simply continue to
disallow that, since that would mean that changes to the
referenced rvalue would not be visible for the caller (a
temporary/literal is changed - how could someone possibly want
that?).
Still that zlib entry is coming to mind. But more importantly is
that you still need code duplication to get both accessible.
//many possible combinations thereof
int func(const ref x);
int func(int x) {
return func(cast(const int) x);
}
But regarding zlib.. It's function is something like:
int compress(char *output, int *size, char *input, int
inputSize);
So... if we convert that to something similar we get..
enum ZlibEnum {}
ZlibEnum compress(void[] output, void[] input, ref Zlib state)
nothrow pure;
The idea in this case is 'state' would continue to hold the
input/output pointers and all information needed, so you could
continue to use it and if it has any 'unflushed' data (output
size too small?) then it could retain that. however if you knew
you didn't need it you could ignore it.
ZlibEnum could be for output C calling code, so success/failure
needs to be known right away. It makes sense for it to return a
Zlib type as well, but depends on what has higher priority and
why.
string input = something long;
ubyte[1000] output;
ubyte[5] output2;
Zlib state;
compress(output, input, null); //we know the buffer is large
enough
compress(output, input, state);
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 03:13:22 UTC, martin wrote:
void func1(ref int x); //D lvalue-only ref
void func2(@ref int x); //works like c++'s ref
Seems fairly easy to tell apart, and still leaves const-ness
as an option.
Afaik C++ doesn't allow rvalues to be passed to _mutable_
references (T), only to const references, making perfect sense
imo. I really do not see the point for an additional syntax for
C++-like const references (const T) which would also take
rvalues.
Please give me an example where you want to pass an rvalue to a
_mutable_ reference parameter. I would simply continue to
disallow that, since that would mean that changes to the
referenced rvalue would not be visible for the caller (a
temporary/literal is changed - how could someone possibly want
that?).
Still that zlib entry is coming to mind. But more importantly is
that you still need code duplication to get both accessible.
//many possible combinations thereof
int func(const ref x);
int func(int x) {
return func(cast(const int) x);
}
But regarding zlib.. It's function is something like:
int compress(char *output, int *size, char *input, int
inputSize);
So... if we convert that to something similar we get..
enum ZlibEnum {}
ZlibEnum compress(void[] output, void[] input, ref Zlib state)
nothrow pure;
The idea in this case is 'state' would continue to hold the
input/output pointers and all information needed, so you could
continue to use it and if it has any 'unflushed' data (output
size too small?) then it could retain that. however if you knew
you didn't need it you could ignore it.
ZlibEnum could be for output C calling code, so success/failure
needs to be known right away. It makes sense for it to return a
Zlib type as well, but depends on what has higher priority and
why.
string input = something long;
ubyte[1000] output;
ubyte[5] output2;
Zlib state;
compress(output, input, null); //we know the buffer is large
enough
compress(output2, input, state);
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 03:35:19 UTC, Era Scarecrow
wrote:
Still that zlib entry is coming to mind. But more importantly
is that you still need code duplication to get both accessible.
//many possible combinations thereof
int func(const ref x);
int func(int x) {
return func(cast(const int) x);
}
int func(in ref int x);
int func(int x) { return func(x); }
The latter overload is for rvalues (no need to cast to const) and
wouldn't be required if the first one took rvalues directly
(having a _const_ ref parameter). That's not an example for
mutable references though, it's basically a shortcut so as to not
having to declare all rvalues for x manually.
enum ZlibEnum {}
ZlibEnum compress(void[] output, void[] input,
ref Zlib state) nothrow pure;
string input = something long;
ubyte[1000] output;
ubyte[5] output2;
Zlib state;
compress(output, input, null);
compress(output2, input, state);
Where's the rvalue? You're talking about an optional mutable
reference here, which is only doable via a pointer (references
cannot be null, not in C++ and not in D):
ZlibEnum compress(void[] output, void[] input, Zlib* state =
null);
...
compress(output, input);
compress(output2, input, state);
That would be correct. But you're completely missing the point
here. Let's assume you wanted to require a state to be passed:
ZlibEnum compress(void[] output, void[] input, ref Zlib state);
You could now use:
Zlib state; // lvalue
compress(output, input, state);
but not:
compress(output, input, Zlib()); // rvalue
And that is most likely a good thing, since your state would be
lost after the compress() call. If you don't want it, you don't
pass it, and as I said, the only way to do that is to pass a
nullable pointer. So this is off-topic I'm afraid.
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 04:05:32 UTC, martin wrote:
int func(in ref int x);
int func(int x) { return func(x); }
The latter overload is for rvalues (no need to cast to const)
and wouldn't be required if the first one took rvalues directly
(having a _const_ ref parameter). That's not an example for
mutable references though, it's basically a shortcut so as to
not having to declare all rvalues for x manually.
Maybe... But when working with non built-in types can you
guarantee that behavior?
//same as 'const ref' basically, unless 'in ref' is accepted
struct S
//int func(in ref S x); //?
int func(const ref S x);
int func(S x) { return func(x); } //potentially infinitely self
calling
compress(output, input, null);
compress(output2, input, state);
Where's the rvalue? You're talking about an optional mutable
reference here, which is only doable via a pointer (references
cannot be null, not in C++ and not in D):
Perhaps null is wrong in this case, but had it been Zlib.init or
Zlib(), then it would still be applicable. Since it's a reference
I would think it could accept a null pointer and realize to
create a temporary which has it's default values.
That would be correct. But you're completely missing the point
here. Let's assume you wanted to require a state to be passed:
ZlibEnum compress(void[] output, void[] input, ref Zlib state);
You could now use:
Zlib state; // lvalue
compress(output, input, state);
but not:
compress(output, input, Zlib()); // rvalue
And that is most likely a good thing, since your state would be
lost after the compress() call. If you don't want it, you don't
pass it, and as I said, the only way to do that is to pass a
nullable pointer. So this is off-topic I'm afraid.
True, but rather than having to create a temporary you don't
plan on using just to satisfy the signature, and you know you
don't NEED it afterwards, why do you have to go through the extra
steps? How many wrappers are made in general just to work around
minor issues like this?
Quite often we ignore return types if they aren't interesting,
but we can't ignore a rvalue we only need for one call (or just
to satisfy the signature)? Say we have a divide function (who
knows what for), and this is more efficient than normal. So...
//perhaps '@ref int remainder = 0'?
int divide(int number, int divisor, ref int remainder);
Now in this case what if we don't care about the remainder?
Create a temporary (or a wrapper function)?
//satisfy remainder
int divide(int number, int divisor) {
int tmp; return divide(number, divisor, tmp);
}
int modulus(int number, int divisor) {
int tmp; divide(number, divisor, tmp);
return tmp;
}
But if we want the result of the division AND the remainder
should we have to make a separate modulus function when the
original divide can clearly give you the answer? This cuts closer
to the instruction set as an example.
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 04:36:59 UTC, Era Scarecrow
wrote:
Maybe... But when working with non built-in types can you
guarantee that behavior?
Sure, try it out.
What you are referring to here has nothing to do with rvalues. It
is about side results which are only sometimes needed. It is
clear that you have to use lvalue arguments for those in order to
access them after the function call. Using bogus rvalue arguments
in case you're not interested in them wouldn't really help -
you'd save the variable declaration, but the function signature
would still be ugly, and you can't always provide hypothetical
default values to hide them (depending on parameter order). In my
experience, these cases are very rare anyway (unless dealing with
porting old C-style code which you seem to be doing) and function
overloading is the most elegant way to handle it - take a look at
the large number of overloads in modern language libraries such
as .NET.
This is how I would implement it:
// magical, super-fast function ;)
int divide(in int number, in int divisor, int* remainder = null);
auto bla = divide(13, 4); // = 3
// remainder needed as well, hiding the pointer as reference:
int divide(in int number, in int divisor, out int remainder)
{
return divide(number, divisor, remainder);
}
int remainder;
auto bla = divide(13, 4, remainder); // = 3, remainder = 1
// only remainder needed:
int modulus(in int number, in int divisor)
{
int tmp;
divide(number, divisor, tmp);
return tmp;
}
auto bla = modulus(13, 4); // = 1
But this is all really off-topic here, let's stop spamming this
thread.
Re: Const ref and rvalues again...
On Wednesday, 7 November 2012 at 02:34:25 UTC, martin wrote:
On Wednesday, 7 November 2012 at 02:06:09 UTC, Rob T wrote:
What about the case where we want to pass a source argument
either by reference or as a copy depending on the l/r value
situation?
eg
void f( ref a );
void f( a );
--rt
I don't get what you mean - that's why the 2 overloads are for
(you forgot the const/in keyword - beware! :))
void f( in ref T a );
void f( in T a );
rvalue binds to the latter overload (argument not copied, but
moved directly).
lvalue binds to the first overload (reference).
Works with v2.060.
Sorry, my explanation was very poor. I'll try again.
I'm trying to describe the case where you want to modify the ref
argument so that the source is also changed. For example, let's
say you want to move a resource instead of coping it. Moving
means the resource is transfered from one object to another, and
the original is reset without destoying the resource. That way
only one object can have the same resource. So for moving a
resource, you cannot use in ref or const ref, you need to
just use ref.
void f( ref T a ){
// take resource away from a
this.resource = a.resource;
a.resetResource();
// this now owns the resource
}
ref T works fine, but if you wish to use f( ref T a ) on a temp
value returned from another function call, you'll need to
overload f() to pass by value, which means creating a duplicate.
void f( ref T a ){
// take resource away from a
this.resource = a.resource;
a.resetResource();
// this now owns the resource
}
Example:
T g(){
T temp;
// create resource which is stored inside T
temp.createResource();
// move temp contents to return value
return move(temp);
}
f( g() ); // won't compile with ref
It's annoying to have to create overloaded duplicates for a
situation like moving, esp if the duplicate is a lot of code. In
C++ the move problem was solved using move semantics, and this
is what I'm trying to emulate in D but I feel like I'm fighting
with the language.
It is possible that D has a specific way of solving this problem
that I'm not aware of yet, so correct me if I'm asking for
something that is simply not needed.
I tried f( move(g()) ) but that fails to work. My best guess is
that D does a hidden move of the temp instead of a copy to value.
I can't say for sure because the documentation is not clear and
is missing important details like this. I also cannot rely on
clever compiler optimizations that may or may not be implemented
as a guarantee.
--rt
Re: Const ref and rvalues again...
On Monday, November 05, 2012 06:05:07 Rob T wrote: On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis wrote: And when we argued for altering it so that it operated like const ref in C++ (which allows const ref in D to continue to function like it does now), some folks complained, because they've found the current semantics of auto ref to be useful (something to do with propagating the exact, original type, I think). I would expect that auto ref for a template and for a non template should work in exactly the same way, so why would there be a difference? If there must be a difference, there should be different semantics for specifying the difference, otherwise the inconsistent behaviours among identical semantics will only serve to confuse people. auto ref's current semantics can't possibly work with non-templated functions, and it's clear that there are good reasons for keeping auto ref as it is now for templates. So, either we make it work with non-templated functions with different (albeit similar) semantics, or we need a new attribute for doing what we want to do with non-templated functions. - Jonathan M Davis
Re: Const ref and rvalues again...
On Monday, 5 November 2012 at 08:01:57 UTC, Jonathan M Davis
wrote:
auto ref's current semantics can't possibly work with
non-templated functions,
and it's clear that there are good reasons for keeping auto ref
as it is now
for templates. So, either we make it work with non-templated
functions with
different (albeit similar) semantics, or we need a new
attribute for doing what
we want to do with non-templated functions.
I agree that the current auto ref semantics are only useful for
templates since auto ref T is replaced by either a reference
(ref T for lvalues) or a value (T for rvalues), two very
different things.
I don't know how auto ref for templates is currently implemented,
i.e., if, given a function
void foo(T)(auto ref T x, auto ref T y) {}
the compiler would instantiate the template max 2^2 times for
each used type T to cover all lvalue/rvalue combinations for x
and y. In that case, treating a non-templated function
void foo(auto ref MyStruct x, auto ref MyStruct y) {}
as implicit template (without template parameters) would be a
viable option imho, i.e.,
void foo()(auto ref MyStruct x, auto ref MyStruct y) {}
If that kind of auto-templating is not what we want, I'd
definitely opt for allowing rvalues as in ref parameters to
keep things simple.
Re: Const ref and rvalues again...
On Thursday, 18 October 2012 at 03:07:56 UTC, Malte Skarupke wrote: Hello, I realize that this has been discussed before, but so far there is no solution and this really needs to be a high priority: We need a way for a function to declare that it doesn't want it's argument to be copied, but it also doesn't care whether the argument is an rvalue or an lvalue. I'm encountering this problem right now, and found this thread while looking for a solution. In my case I'm forced into doubling up some code, which is not what I want to be doing, so I agree this item should be fixed. It would be nice to have something like the proposed auto ref solution, like we have with return values. --rt
Re: Const ref and rvalues again...
I find it sad that while this topic seems to be of high priority for quite a lot of language users, it is seemingly neglected by the head of language development (Walter, Andrei etc.).
Re: Const ref and rvalues again...
On Monday, 5 November 2012 at 00:58:19 UTC, martin wrote: I find it sad that while this topic seems to be of high priority for quite a lot of language users, it is seemingly neglected by the head of language development (Walter, Andrei etc.). I don't know why this would be ignored, it's rather useful and solves a real-world programming problem. I hope this is just one of those detials that was missed and is only being ignored due to attention from much bigger problems. Do you know if there is already a feature request or bug report on this item?
Re: Const ref and rvalues again...
On 11/4/12 7:58 PM, martin wrote: I find it sad that while this topic seems to be of high priority for quite a lot of language users, it is seemingly neglected by the head of language development (Walter, Andrei etc.). I was hoping auto ref solves this problem. I think it's currently only implemented for templates. Andrei
Re: Const ref and rvalues again...
On Sunday, November 04, 2012 20:43:36 Andrei Alexandrescu wrote: On 11/4/12 7:58 PM, martin wrote: I find it sad that while this topic seems to be of high priority for quite a lot of language users, it is seemingly neglected by the head of language development (Walter, Andrei etc.). I was hoping auto ref solves this problem. I think it's currently only implemented for templates. And when we argued for altering it so that it operated like const ref in C++ (which allows const ref in D to continue to function like it does now), some folks complained, because they've found the current semantics of auto ref to be useful (something to do with propagating the exact, original type, I think). Now, since auto ref currently only works with templates, maybe we can keep its current semantics with templated functions but alter them for non-templated functions so that it works like const ref does in C++. The downside is that the semantics for auto ref between templated functions and non-templated functions are slightly different, but they're close enough that I'm not sure that it matters. - Jonathan M Davis
Re: Const ref and rvalues again...
On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis wrote: On Sunday, November 04, 2012 20:43:36 Andrei Alexandrescu wrote: On 11/4/12 7:58 PM, martin wrote: I find it sad that while this topic seems to be of high priority for quite a lot of language users, it is seemingly neglected by the head of language development (Walter, Andrei etc.). I was hoping auto ref solves this problem. I think it's currently only implemented for templates. And when we argued for altering it so that it operated like const ref in C++ (which allows const ref in D to continue to function like it does now), some folks complained, because they've found the current semantics of auto ref to be useful (something to do with propagating the exact, original type, I think). Now, since auto ref currently only works with templates, maybe we can keep its current semantics with templated functions but alter them for non-templated functions so that it works like const ref does in C++. The downside is that the semantics for auto ref between templated functions and non-templated functions are slightly different, but they're close enough that I'm not sure that it matters. - Jonathan M Davis Yes, please. Auto ref for non-template functions would solve the problem exactly. I also like it because then the intent of the programmer is clear.
Re: Const ref and rvalues again...
On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis wrote: And when we argued for altering it so that it operated like const ref in C++ (which allows const ref in D to continue to function like it does now), some folks complained, because they've found the current semantics of auto ref to be useful (something to do with propagating the exact, original type, I think). I would expect that auto ref for a template and for a non template should work in exactly the same way, so why would there be a difference? If there must be a difference, there should be different semantics for specifying the difference, otherwise the inconsistent behaviours among identical semantics will only serve to confuse people. --rt
Re: Const ref and rvalues again...
On 10/22/2012 12:18 AM, martin wrote: On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote: Const is different in D and in C++. Relating const and rvalues is arbitrary and does not make a lot of sense. It's actually pretty much the same concept in both languages except for the transitiveness in D. Case closed.
Re: Const ref and rvalues again...
On 10/22/2012 11:38 AM, Timon Gehr wrote: On 10/22/2012 12:18 AM, martin wrote: On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote: Const is different in D and in C++. Relating const and rvalues is arbitrary and does not make a lot of sense. It's actually pretty much the same concept in both languages except for the transitiveness in D. Case closed. (But it might be added that C++11 adds a new language feature just in order to be able to distinguish const and rvalue references in the callee, so even with C++'s semantics, this does not seem like a good idea.)
Re: Const ref and rvalues again...
On Monday, 22 October 2012 at 09:41:38 UTC, Timon Gehr wrote:
(But it might be added that C++11 adds a new language feature
just in order to be able to distinguish const and rvalue
references in the callee, so even with C++'s semantics, this
does not seem like a good idea.)
Yeah, and that T syntax is mainly used for _mutable_ rvalue
references in move constructors and assignment operators to move
(hijack) mutable data from the rvalue to another T instance
instead of copying that data, knowing that the rvalue's data is
not going to be accessed anymore anyway. So this is a different
case and does currently not apply to D afaik since there is no
way to forward rvalues:
void foo(ref T lvalue) { }
void foo(T rvalue) { }
In the latter overload for rvalues, you aren't given the original
rvalue, but a copy of it!
Distinguishing between _const_ rvalue and lvalue references
though makes no sense imho, and that is my whole point.
Re: Const ref and rvalues again...
On Monday, 22 October 2012 at 11:59:27 UTC, martin wrote: In the latter overload for rvalues, you aren't given the original rvalue, but a copy of it! I need to correct that after a quick test: the rvalue is passed directly (moved) instead of copying it (well, at least the copy constructor this(this) is not invoked, even in a debug build); that makes perfect sense, is efficient and eliminates the need for C++ rvalue references (T). It doesn't affect the need for an implicit rvalue = const ref propagation though. What I'd like to see is the following passing scheme for function arguments (read-only parameters are denoted by (*)): lvalue: rvalue: T: copy move (*) in T: copy move out T: pass pointer n/a ref T: pass pointer n/a (*) in ref T: pass pointer store on the caller's stack and pass its address So only the rvalue passing rule for the in ref T case would need to be implemented. This would allow to use foo(in ref T bar) for lvalues (eliding a copy expected to be costly) as well as rvalues instead of having to add an overload foo(in T bar) for rvalues. For rvalues, this would actually implicate a performance hit due to pointer indirection, so the compiler could attempt to add an automatic foo(in T bar) overload if not existent. For rvalues, the latter overload (i.e., in T parameters) should be preferred over in ref T parameters - exactly as the thread starter Malte proposes: - Make functions that take ref in arguments also accept rvalues. - The user can still provide an overload that accepts an rvalue, using the in keyword, and that one will be preferred over the ref in version.
Re: Const ref and rvalues again...
On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:
Const is different in D and in C++. Relating const and rvalues
is arbitrary and does not make a lot of sense.
It's actually pretty much the same concept in both languages
except for the transitiveness in D. An implicit rvalue = _const_
ref conversion is something completely different than an rvalue
= mutable ref conversion, and this distinction is sadly largely
lost in the linked thread, which is why I tried to re-initiate
the discussion with another recent thread. Arguments passed by
mutable references are expected to be changed by the called
function, so passing rvalues does indeed not make much sense in
that case (side effects may get lost, quite possibly
unintentionally).
On Friday, 19 October 2012 at 01:26:35 UTC, Malte Skarupke wrote:
This is about finding a way that you can define a function
which safely accepts lvalues and rvalues without having to make
a copy.
EXACTLY! Something telling the compiler: pass the argument by
const reference instead of a costly const copy, if possible. So
for an lvalue, pass directly its address; for an rvalue, allocate
the temporary on the stack and then pass its address:
code
T foo(in ref T bla) { return bla; }
T lvalue;
foo(lvalue); // fine
foo(foo(lvalue)); // error: rvalue!
// required compiler magic:
immutable tmp = foo(lvalue); foo(tmp);
/* possibly in dedicated scope for
immediate destruction of tmp*/
/code
Currently having to allocate all rvalues manually is frankly both
ugly and a pain in the ass, as would be the other alternative:
overloading foo().
On Friday, 19 October 2012 at 03:49:56 UTC, jerro wrote:
My point was that saving an address of a const ref parameter is
already unsafe if you call the function with a local variable
as the parameter. If this behavior seems problematic to you
when it concerns rvalues, it should seem equally problematic
when it comes to local variables. It doesn't make sense to make
passing rvalues as const ref parameters illegal because of this
problem, when passing local variables causes the same problem
and is legal. [...]
The only case I can think of when passing a local variable as
const ref is safe, but passing an rvalue wouldn't be, is when
the called function returns the address of the const parameter
(or assigns it to some other ref parameter).
Exactly my thoughts regarding this escaping aspect, which I don't
see as big issue. The compiler magic illustrated in the previous
example without dedicated scope for temporaries would make
passing rvalues exactly as safe/unsafe as passing local variables.
Re: Const ref and rvalues again...
On Friday, 19 October 2012 at 13:00:52 UTC, Timon Gehr wrote:
On 10/19/2012 09:53 AM, Jacob Carlborg wrote:
On 2012-10-19 04:48, Timon Gehr wrote:
Then how to specify that the value of x cannot be escaped?
I'm in favour of doing it the other way round and disallow
escaping of
ref parameters without an unsafe cast.
scope is supposed to be used to prevent this.
Sure, but how?
void goo(scope int* x){
global0 = x; // should clearly be disallowed
}
void foo(scope ref int*** x){
global1 = x; // ?
global2 = x; // ?
global3 = *x; // ?
globall4 = **x; // ?
}
Maybe we need this:
void foo(scope ref int*** x); // ?
void foo(ref int(***)scope x); // no escaping of x, *x, **x
void foo(ref int*(**)scope x); // may escape **x
void foo(ref int**(*)scope x); // may escape *x, **x
What about x?
No scope should mean that you can not escape the address of
something. So
void goo(scope int* x)
{
global0 = x;
}
Should be allowed. Scope in this case applies to the pointer. Not
to the thing it's pointing to. The scope keyword is not
transitive. That wouldn't make sense. It is perfectly legal to
have a scoped pointer to something allocated on the heap.
Whereas
void goo(scope ref int x)
{
global0 = x;
}
Should NOT be allowed. It looks like it's the same code, but in
this case x is an integer. In the last example x was a pointer to
an integer.
So it comes down to this:
void goo(scope int* x)
{
global0 = x; // copying is allowed
//global1 = x; // referencing is not allowed
}
Re: Const ref and rvalues again...
On 10/20/2012 08:25 PM, Malte Skarupke wrote:
...
So it comes down to this:
void goo(scope int* x)
{
global0 = x; // copying is allowed
//global1 = x; // referencing is not allowed
}
That is useless.
Re: Const ref and rvalues again...
On 2012-10-19 04:48, Timon Gehr wrote: Then how to specify that the value of x cannot be escaped? I'm in favour of doing it the other way round and disallow escaping of ref parameters without an unsafe cast. scope is supposed to be used to prevent this. -- /Jacob Carlborg
Re: Const ref and rvalues again...
On Friday, 19 October 2012 at 07:53:30 UTC, Jacob Carlborg wrote: On 2012-10-19 04:48, Timon Gehr wrote: Then how to specify that the value of x cannot be escaped? I'm in favour of doing it the other way round and disallow escaping of ref parameters without an unsafe cast. scope is supposed to be used to prevent this. I like Timon's idea. scope is a bad fit in that the default should be the safe option. Unfortunately this does have the potential to brake lots of code, perhaps even if we limit it to @safe code. An argument could be made that it's worth the breakage for @safe code to insure better safety.
Re: Const ref and rvalues again...
On 10/19/2012 09:53 AM, Jacob Carlborg wrote:
On 2012-10-19 04:48, Timon Gehr wrote:
Then how to specify that the value of x cannot be escaped?
I'm in favour of doing it the other way round and disallow escaping of
ref parameters without an unsafe cast.
scope is supposed to be used to prevent this.
Sure, but how?
void goo(scope int* x){
global0 = x; // should clearly be disallowed
}
void foo(scope ref int*** x){
global1 = x; // ?
global2 = x; // ?
global3 = *x; // ?
globall4 = **x; // ?
}
Maybe we need this:
void foo(scope ref int*** x); // ?
void foo(ref int(***)scope x); // no escaping of x, *x, **x
void foo(ref int*(**)scope x); // may escape **x
void foo(ref int**(*)scope x); // may escape *x, **x
What about x?
Re: Const ref and rvalues again...
On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis wrote: On Thursday, October 18, 2012 06:24:08 jerro wrote: What would be the problem with const ref taking rvalues? Read the thread that I already linked to: http://forum.dlang.org/thread/4f84d6dd.5090...@digitalmars.com - Jonathan M Davis I read the thread, and not a single one of the problematic cases are actually valid C++. Yes: the faulty MSVC has taught people to do retarded things, or be afraid of things that were illegal to begin with (in particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), such as increment(5). There is actually nothing wrong with creating a temporary when something is bound to a const ref, provided the compiler follows the rules: *Only LValues with an EXACT type match may be passed to a reference. *In regards to *const* references, RValues may be copied in a temporary, and that temporary bound the the ref. I'm not saying we particularly *need* this in D (C++ has a by ref paradigm that makes it more important, but D *rarelly* ever passes by const ref). But if the compiler respects the above two rules (which it should), then RValue to const ref is both perfectly doable and safe (as safe as refs get anyways).
Re: Const ref and rvalues again...
On Thursday, 18 October 2012 at 06:11:26 UTC, monarch_dodra wrote: On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis wrote: On Thursday, October 18, 2012 06:24:08 jerro wrote: What would be the problem with const ref taking rvalues? Read the thread that I already linked to: http://forum.dlang.org/thread/4f84d6dd.5090...@digitalmars.com - Jonathan M Davis I read the thread, and not a single one of the problematic cases are actually valid C++. Yes: the faulty MSVC has taught people to do retarded things, or be afraid of things that were illegal to begin with (in particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), such as increment(5). There is actually nothing wrong with creating a temporary when something is bound to a const ref, provided the compiler follows the rules: *Only LValues with an EXACT type match may be passed to a reference. *In regards to *const* references, RValues may be copied in a temporary, and that temporary bound the the ref. I'm not saying we particularly *need* this in D (C++ has a by ref paradigm that makes it more important, but D *rarelly* ever passes by const ref). But if the compiler respects the above two rules (which it should), then RValue to const ref is both perfectly doable and safe (as safe as refs get anyways). By allowing the the C++ semantics the function looses semantic information - whether the actual parameter was lvalue or rvalue. This semantic info can be used bot for compiler optimizations and move semantics. This is the reason C++11 added references. General question (might not be relevant to current design of D): How about leaving the decision to the compiler and let the programmer only specify usage intent? E.g.: (I'm speaking semantics here, not syntax) void foo(const Type t); // 1. I only read the value void foo (mutate Type t); // 2. I want to also mutate the actual parameter void foo (move Type t); // 3. I want to move the actual parameter In case 1 above, the compiler is free to pass lvalues by const and rvalues by value or perhaps optimize above certain size to const too. In case 2, the compiler passes a ref to lvalue, rvalues are not accepted at CT. If I want move semantics, I can use option 3 which accepts rvalues by ref. btw, what's the correct semantics for lvalues here? What do you think?
Re: Const ref and rvalues again...
On Thursday, 18 October 2012 at 06:11:26 UTC, monarch_dodra wrote: On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis wrote: On Thursday, October 18, 2012 06:24:08 jerro wrote: What would be the problem with const ref taking rvalues? Read the thread that I already linked to: http://forum.dlang.org/thread/4f84d6dd.5090...@digitalmars.com - Jonathan M Davis I read the thread, and not a single one of the problematic cases are actually valid C++. Yes: the faulty MSVC has taught people to do retarded things, or be afraid of things that were illegal to begin with (in particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), such as increment(5). There is actually nothing wrong with creating a temporary when something is bound to a const ref, provided the compiler follows the rules: *Only LValues with an EXACT type match may be passed to a reference. *In regards to *const* references, RValues may be copied in a temporary, and that temporary bound the the ref. I'm not saying we particularly *need* this in D (C++ has a by ref paradigm that makes it more important, but D *rarelly* ever passes by const ref). But if the compiler respects the above two rules (which it should), then RValue to const ref is both perfectly doable and safe (as safe as refs get anyways). The problem with binding rvalues to const ref is that you could take and store the address of it. That's why I'd recommend using in ref instead. @Jonathan: I had already read the linked discussion. There are many valid points in there, but also many invalid ones (as monarch_dodra has pointed out). But I think all problems in that thread should be solved by using in ref instead of const ref because then you'd be sure that the passed-in temporary can not escape the current function. @foobar: I like the idea, but it's probably going to break down in many cases. If you have a non-trivial copy constructor you want the ability to have complete control over when it gets copied and when it doesn't. I just don't trust compilers enough to think that they'd always make the same choice that I'd make. And also about losing semantic information: That's why I proposed the second point: Give the user the option to provide a function which should be preferred for rvalues. That way you don't lose semantic information.
Re: Const ref and rvalues again...
The problem with binding rvalues to const ref is that you could take and store the address of it. That's why I'd recommend using in ref instead. You can also take and store the address of a local variable that was passed as a const ref parameter, and accessing it after the caller exits will result in undefined behavior too. On the other hand, addresses of both local variables and rvalues will be valid at least until the called function returns. Local variables and rvalues are equivalent in regard to this problem.
Re: Const ref and rvalues again...
On 10/19/2012 01:39 AM, Malte Skarupke wrote: On Thursday, 18 October 2012 at 06:11:26 UTC, monarch_dodra wrote: On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis wrote: On Thursday, October 18, 2012 06:24:08 jerro wrote: What would be the problem with const ref taking rvalues? Read the thread that I already linked to: http://forum.dlang.org/thread/4f84d6dd.5090...@digitalmars.com - Jonathan M Davis I read the thread, and not a single one of the problematic cases are actually valid C++. Yes: the faulty MSVC has taught people to do retarded things, or be afraid of things that were illegal to begin with (in particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), such as increment(5). There is actually nothing wrong with creating a temporary when something is bound to a const ref, provided the compiler follows the rules: *Only LValues with an EXACT type match may be passed to a reference. *In regards to *const* references, RValues may be copied in a temporary, and that temporary bound the the ref. I'm not saying we particularly *need* this in D (C++ has a by ref paradigm that makes it more important, but D *rarelly* ever passes by const ref). But if the compiler respects the above two rules (which it should), then RValue to const ref is both perfectly doable and safe (as safe as refs get anyways). The problem with binding rvalues to const ref is that you could take and store the address of it. That's why I'd recommend using in ref instead. @Jonathan: I had already read the linked discussion. There are many valid points in there, but also many invalid ones (as monarch_dodra has pointed out). But I think all problems in that thread should be solved by using in ref instead of const ref because then you'd be sure that the passed-in temporary can not escape the current function. @foobar: I like the idea, but it's probably going to break down in many cases. If you have a non-trivial copy constructor you want the ability to have complete control over when it gets copied and when it doesn't. I just don't trust compilers enough to think that they'd always make the same choice that I'd make. And also about losing semantic information: That's why I proposed the second point: Give the user the option to provide a function which should be preferred for rvalues. That way you don't lose semantic information. Const is different in D and in C++. Relating const and rvalues is arbitrary and does not make a lot of sense. Regarding 'in ref'/'scope ref': What should 'scope' apply to in void foo(scope ref int* x);
Re: Const ref and rvalues again...
On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:
Const is different in D and in C++. Relating const and rvalues
is arbitrary and does not make a lot of sense.
Regarding 'in ref'/'scope ref': What should 'scope' apply to in
void foo(scope ref int* x);
Not sure what you mean with relating. I'm not making any claims
about there being a relationship between rvalues and constness.
This is about finding a way that you can define a function which
safely accepts lvalues and rvalues without having to make a copy.
If we specify the argument as ref in, then we can safely pass
for example the number 5 to it. And this would never break
existing code, so that something like swap(5, 4) would never be
possible code.
For the example that you gave you'd be unable to store the
address of x. So doing
int** storage;
void foo(scope ref int * x)
{
storage = x;
}
would be illegal.
@jerro: the same thing: I'm not trying to fix the problem that
you mention. I'm trying to define a function which can safely
accept rvalues and lvalues without having to make a copy.
Re: Const ref and rvalues again...
On 10/19/2012 03:26 AM, Malte Skarupke wrote:
On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:
Const is different in D and in C++. Relating const and rvalues is
arbitrary and does not make a lot of sense.
Regarding 'in ref'/'scope ref': What should 'scope' apply to in
void foo(scope ref int* x);
Not sure what you mean with relating. I'm not making any claims about
there being a relationship between rvalues and constness.
You do it again right away:
This is about finding a way that you can define a function which safely
accepts lvalues and rvalues without having to make a copy. If we specify
the argument as ref in, then we can safely pass for example the number
^~~~
const
5 to it.
^
rvalue
And this would never break existing code, so that something
like swap(5, 4) would never be possible code.
That does not break existing code except code that checks validity of
code, but every language change does that.
For the example that you gave you'd be unable to store the address of x.
So doing
int** storage;
void foo(scope ref int * x)
{
storage = x;
}
would be illegal.
Then how to specify that the value of x cannot be escaped?
I'm in favour of doing it the other way round and disallow escaping of
ref parameters without an unsafe cast.
@jerro: the same thing: I'm not trying to fix the problem that you
mention. I'm trying to define a function which can safely accept rvalues
and lvalues without having to make a copy.
Re: Const ref and rvalues again...
@jerro: the same thing: I'm not trying to fix the problem that you mention. I'm trying to define a function which can safely accept rvalues and lvalues without having to make a copy. My point was that saving an address of a const ref parameter is already unsafe if you call the function with a local variable as the parameter. If this behavior seems problematic to you when it concerns rvalues, it should seem equally problematic when it comes to local variables. It doesn't make sense to make passing rvalues as const ref parameters illegal because of this problem, when passing local variables causes the same problem and is legal. It would only make sense to introduce in ref whith an intent to solve this problem, if local variables would also have to be passed as in ref (or some kind of scope ref, not necessarily const). But that would break pretty much all code that uses ref. The only case I can think of when passing a local variable as const ref is safe, but passing an rvalue wouldn't be, is when the called function returns the address of the const parameter (or assigns it to some other ref parameter).
