Re: Rvalue references

[Tony via Digitalmars-d-learn]

On Sunday, 14 January 2018 at 00:55:27 UTC, Jonathan M Davis 
wrote:



[...]


It the simplest case, it means that the compiler does a bitwise 
copy rather than a deep copy, but in other cases, it means that 
the compiler is able to use the object in-place rather than 
creating a deep copy that it places elsewhere. If you want to 
know more on the topic, you can always look into C++ move 
constructors. They were added so that C++ could avoid a lot of 
unnecessary copies. D took the approach of requiring that 
structs be moveable (e.g. it's undefined behavior to have a 
struct contain a pointer to itself), which simplifies things 
considerably.


[...]

Thanks!

Re: Rvalue references

[Jonathan M Davis via Digitalmars-d-learn]

On Friday, January 12, 2018 01:59:49 Tony via Digitalmars-d-learn wrote:
> On Monday, 8 January 2018 at 23:31:27 UTC, Jonathan M Davis wrote:
> > auto foo(T)(auto ref T t)
> > {
> >
> > return t;
> >
> > }
> >
> > foo(42);
> >
> > will result in foo being instantiated as
> >
> > int foo(int t)
> > {
> >
> > return t;
> >
> > }
> >
> > whereas
> >
> > int i;
> > foo(i);
> >
> > will result in foo being instantiated as
> >
> > int foo(ref int t)
> > {
> >
> > return t;
> >
> > }
> >
> > So, by using auto ref, a function can accept both lvalues and
> > rvalues. And in D, rvalues get moved, not copied.
>
> What does it mean to "move" a variable/value instead of copying
> it?

It the simplest case, it means that the compiler does a bitwise copy rather
than a deep copy, but in other cases, it means that the compiler is able to
use the object in-place rather than creating a deep copy that it places
elsewhere. If you want to know more on the topic, you can always look into
C++ move constructors. They were added so that C++ could avoid a lot of
unnecessary copies. D took the approach of requiring that structs be
moveable (e.g. it's undefined behavior to have a struct contain a pointer to
itself), which simplifies things considerably.

You can also check out the stackoverflow questions that I linked to before:

https://stackoverflow.com/questions/35120474/does-d-have-a-move-constructor
https://stackoverflow.com/questions/6884996/questions-about-postblit-and-move-semantics

They don't anwser what a move is, but they do give more context.

> Was "auto ref" created for anything besides structs?

auto ref works with any type whatsoever, but as far as efficiency goes, it's
kind of pointless for other types, because most everything else is at most
64 bits, which would be the size of a pointer on a 64-bit system. reals are
larger, and if we ever get cent and ucent, those would be larger, but most
stuff that isn't structs fits in 64 bits, meaning that copying them is not
expensive.

However, if you need to forward the refness of an argument, then auto ref
can still be critical - e.g. std.conv.emplace constructs an object in
memory, and if it couldn't pass the refness of the arguments on to the
constructor that it calls, then constructors with ref parameters wouldn't
work properly. Similarly, in templated code, you can have stuff like front
on a range which may or may not return by ref, so by using auto ref on the
return type of front when wrapping one range with another, the refness of
the underlying range's front can be passed along. Arguably, this aspect of
auto ref matters _way_ more than trying to make it more efficient to copy
structs.

- Jonathan M Davis

Re: Rvalue references

[Tony via Digitalmars-d-learn]

On Monday, 8 January 2018 at 23:31:27 UTC, Jonathan M Davis wrote:



auto foo(T)(auto ref T t)
{
return t;
}

foo(42);

will result in foo being instantiated as

int foo(int t)
{
return t;
}

whereas

int i;
foo(i);

will result in foo being instantiated as

int foo(ref int t)
{
return t;
}

So, by using auto ref, a function can accept both lvalues and 
rvalues. And in D, rvalues get moved, not copied.


What does it mean to "move" a variable/value instead of copying 
it?


Was "auto ref" created for anything besides structs?

Re: Rvalue references

[Steven Schveighoffer via Digitalmars-d-learn]

On 1/10/18 12:59 PM, Dgame wrote:

On Wednesday, 10 January 2018 at 14:41:21 UTC, Steven Schveighoffer wrote:

On 1/10/18 3:08 AM, Dgame wrote:
On Wednesday, 10 January 2018 at 01:56:02 UTC, Steven Schveighoffer 
wrote:

But current auto ref is what we have, so I would recommend using it.


I would recommend to ignore auto ref for rvalue references. It 
generates 2^N functions where N is the amount of auto ref parameters. 
That the most awful template bloat I've ever seen.


It only generates 2^N functions if you call it 2^N different ways. 
Most of the time you call it the same way.




If that would be true we wouldn't need auto ref at all.


The author of the function may not be the one calling it, so he doesn't 
know the way you like to call it.


All I'm saying is to think about that your 2^N prediction requires 2^N 
lines of code, all calling it different ways. It's not likely to happen.


-Steve

Re: Rvalue references

[Dgame via Digitalmars-d-learn]

On Wednesday, 10 January 2018 at 14:41:21 UTC, Steven 
Schveighoffer wrote:

On 1/10/18 3:08 AM, Dgame wrote:
On Wednesday, 10 January 2018 at 01:56:02 UTC, Steven 
Schveighoffer wrote:
But current auto ref is what we have, so I would recommend 
using it.


I would recommend to ignore auto ref for rvalue references. It 
generates 2^N functions where N is the amount of auto ref 
parameters. That the most awful template bloat I've ever seen.


It only generates 2^N functions if you call it 2^N different 
ways. Most of the time you call it the same way.


-Steve


If that would be true we wouldn't need auto ref at all.

Re: Rvalue references

[Steven Schveighoffer via Digitalmars-d-learn]

On 1/10/18 3:08 AM, Dgame wrote:

On Wednesday, 10 January 2018 at 01:56:02 UTC, Steven Schveighoffer wrote:

But current auto ref is what we have, so I would recommend using it.


I would recommend to ignore auto ref for rvalue references. It generates 
2^N functions where N is the amount of auto ref parameters. That the 
most awful template bloat I've ever seen.


It only generates 2^N functions if you call it 2^N different ways. Most 
of the time you call it the same way.


-Steve

Re: Rvalue references

[Dgame via Digitalmars-d-learn]

On Wednesday, 10 January 2018 at 01:56:02 UTC, Steven 
Schveighoffer wrote:
But current auto ref is what we have, so I would recommend 
using it.


I would recommend to ignore auto ref for rvalue references. It 
generates 2^N functions where N is the amount of auto ref 
parameters. That the most awful template bloat I've ever seen.

Re: Rvalue references

[Steven Schveighoffer via Digitalmars-d-learn]

On 1/8/18 6:07 PM, Jiyan wrote:


Sry i know i asked it already in IRC:
Are rvalue references already solved with auto ref?

https://p0nce.github.io/d-idioms/#Rvalue-references:-Understanding-auto-ref-and-then-not-using-it 



Says rvalues are moved!


But an rvalue move is cheaper. You construct it right on the stack where 
it needs to be, and no actual copy is happening. Then inside the 
function, no further indirections are needed, just stack offsets.



The other solution seems not so practical.


The other solution exploits a hole in the "rvalues cannot be references" 
mantra. Because all member functions take 'this' by reference, the 
function call can be used to blur the line between rvalues and lvalues.


It makes for some... interesting things:

struct Int
{
   int value;
   Int opBinary(string op : "+")(ref const Int other) const
   {
   return Int(other.value + value);
   }
}

auto v = Int(5);

auto v2 = s + Int(5); // error
auto v3 = Int(5) + s; // OK!

Is any solution to them intended, or was all the talk about rvalue 
references simply discarded?


auto ref was actually proposed as a non-template solution, but Walter 
implemented it the way it is now. The original proposal would have meant 
that rvalue references like C++ were possible (without conflating it 
with const).


But current auto ref is what we have, so I would recommend using it.

-Steve

Re: Rvalue references

[Jonathan M Davis via Digitalmars-d-learn]

On Monday, January 08, 2018 23:07:52 Jiyan via Digitalmars-d-learn wrote:
> Sry i know i asked it already in IRC:
> Are rvalue references already solved with auto ref?
>
> https://p0nce.github.io/d-idioms/#Rvalue-references:-Understanding-auto-re
> f-and-then-not-using-it
>
> Says rvalues are moved!
>
> The other solution seems not so practical.

rvalue references do not exist in D. auto ref makes it so that the refness
of a parameter or return type of a templated function depends on whether
its' an lvalue or rvalue. e.g.

auto foo(T)(auto ref T t)
{
return t;
}

foo(42);

will result in foo being instantiated as

int foo(int t)
{
return t;
}

whereas

int i;
foo(i);

will result in foo being instantiated as

int foo(ref int t)
{
return t;
}

So, by using auto ref, a function can accept both lvalues and rvalues. And
in D, rvalues get moved, not copied.

https://stackoverflow.com/questions/35120474/does-d-have-a-move-constructor
https://stackoverflow.com/questions/6884996/questions-about-postblit-and-move-semantics

> Is any solution to them intended, or was all the talk about
> rvalue references simply discarded?

Andrei has said on a number of occasions that he does not want rvalue
references in D, and as I understand it, he considers them to have been a
mistake in C++. There has been occasional talk of defining a way to have ref
(or something similar to ref) accept rvalues in addition to lvalues, but it
would have to be done in a way that would satisfy Andrei and his concern
about rvalue references. I don't know what that would look like, and thus
far, no one has create a DIP for such a feature.

I think that the solution that most folks have gone with is simply passing
everything by value unless it needs to be passed by ref so that the argument
can be mutated. And then if profiling indicates that that's a performance
problem in a particular case, then auto ref gets used, or the function gets
overloaded on refness, or the code is refactored in some other way such that
it's not passing around an object that is expensive to copy, or the object
is turned into a reference type.

The fact that D moves rvalues tends to mean that the only cases where
expensive copying would be a problem are with large objects passed as
lvalues or with objects with expensive postblits which are passed by value
(and expensive postblits are generally discouraged). But the most efficient
thing to do there depends on the code and how it interacts with the
optimizer such that assuming what's going to be more efficient can often be
wrong (especially if the object is smaller). In most cases, worrying about
excessive copying seems to tend to be a premature optimization, though if an
object is particularly large, then odds are it should probably be a
reference type. Either way, in most cases, I'd suggest not worrying all that
much about the cost of copying objects unless you have evidence that it's
actually a problem with your code, and when it is, that usually means that
using a reference type would be better, but auto ref is there for those
cases where it makes sense to templatize a function on refness to pass
objects more efficiently (or simply for when refness needs to be passed on
for one reason or another).

- Jonathan M Davis

Re: rvalue references

[Marco Leise via Digitalmars-d]

Am Wed, 19 Oct 2016 11:29:50 +0200
schrieb Timon Gehr :

> Yes, the lack of rvalue references can be annoying but 'const' should be 
> orthogonal to any implemented solution.
> 
> There should be a way to pass rvalues by reference that are not 
> prevented from being mutated, and it should be the same way as for const 
> references. (For example, one possible approach is to just allow rvalues 
> to bind to all 'ref' parameters (as is done for the implicit 'this' 
> reference), alternatively there could be some additional "allow rvalues 
> here" annotation that does not influence mutability.)

Ok, got ya now!

-- 
Marco

Re: rvalue references

[Timon Gehr via Digitalmars-d]

On 19.10.2016 04:58, Marco Leise wrote:

Am Tue, 18 Oct 2016 22:43:01 +0200
schrieb Timon Gehr :


It wouldn't even be the same thing if it was allowed. D const is not C++
const. Enforcing transitive read-only on rvalue references does not make
that much sense.


For me using const ref vs. const is typically entirely a
matter of avoiding a costly copy. The function body shouldn't
care whether the data came from an rvalue or lvalue. It is
exemplified in the following lines where doSomething takes a
const ref and worldTransform returns by value:

  const worldTransform = someObj.worldTransform();
  otherObj.doSomething(arg1, arg2, worldTransform);

I'd prefer to write:

  otherObj.doSomething(arg1, arg2, someObj.worldTransform());

as the temporaries add significant noise in some functions and
make the equivalent C++ code look cleaner.

It may be noteworthy that the lack of rvalue references only
really got annoying for me with matrix calculations, because
there are so many medium sized temporaries. Some of them come
directly from binary operators. Functions often need to
perform a set of computations that bare a lot of similarity,
where visual cues help the understanding. In a reduced form:

  calculate(  matrix);  // works
  calculate(2*matrix);  // doesn't work, requires temporary
  calculate(3*matrix);  //  "

Introducing temporaries makes the similarity of the
calculation less evident. Making calculate take auto-ref would
result in duplicate code and thrash the instruction cache
(especially with 2 or 3 arguments). Dropping "ref" results in
unnecessary matrix copies at this and other call sites.



Yes, the lack of rvalue references can be annoying but 'const' should be 
orthogonal to any implemented solution.


There should be a way to pass rvalues by reference that are not 
prevented from being mutated, and it should be the same way as for const 
references. (For example, one possible approach is to just allow rvalues 
to bind to all 'ref' parameters (as is done for the implicit 'this' 
reference), alternatively there could be some additional "allow rvalues 
here" annotation that does not influence mutability.)

Re: rvalue references

[Marco Leise via Digitalmars-d]

Am Tue, 18 Oct 2016 22:43:01 +0200
schrieb Timon Gehr :

> It wouldn't even be the same thing if it was allowed. D const is not C++ 
> const. Enforcing transitive read-only on rvalue references does not make 
> that much sense.

For me using const ref vs. const is typically entirely a
matter of avoiding a costly copy. The function body shouldn't
care whether the data came from an rvalue or lvalue. It is
exemplified in the following lines where doSomething takes a
const ref and worldTransform returns by value:

  const worldTransform = someObj.worldTransform();
  otherObj.doSomething(arg1, arg2, worldTransform);

I'd prefer to write:

  otherObj.doSomething(arg1, arg2, someObj.worldTransform());

as the temporaries add significant noise in some functions and
make the equivalent C++ code look cleaner.

It may be noteworthy that the lack of rvalue references only
really got annoying for me with matrix calculations, because
there are so many medium sized temporaries. Some of them come
directly from binary operators. Functions often need to
perform a set of computations that bare a lot of similarity,
where visual cues help the understanding. In a reduced form:

  calculate(  matrix);  // works
  calculate(2*matrix);  // doesn't work, requires temporary
  calculate(3*matrix);  //  "

Introducing temporaries makes the similarity of the
calculation less evident. Making calculate take auto-ref would
result in duplicate code and thrash the instruction cache
(especially with 2 or 3 arguments). Dropping "ref" results in
unnecessary matrix copies at this and other call sites.

-- 
Marco

Re: rvalue references

[Stewart Gordon via Digitalmars-d]

On 09/06/2015 13:14, Namespace wrote:
snip

What does this have to do with garbage-collected language?
If I have a big struct, e.g.

struct Matrix {
 float[16] values = [...];
}

I always want to pass it by ref because a move or a copy would be too slow.


That seems to me a matter more of having something that behaves like C++ const T, than of 
rvalue references as such.


Stewart.

--
My email address is valid but not my primary mailbox and not checked regularly.  Please 
keep replies on the 'group where everybody may benefit.

Re: rvalue references

[Namespace via Digitalmars-d]

On Friday, 12 June 2015 at 19:39:25 UTC, Stewart Gordon wrote:

On 09/06/2015 13:14, Namespace wrote:
snip

What does this have to do with garbage-collected language?
If I have a big struct, e.g.

struct Matrix {
 float[16] values = [...];
}

I always want to pass it by ref because a move or a copy would 
be too slow.


That seems to me a matter more of having something that behaves 
like C++ const T, than of rvalue references as such.


Stewart.


Yes, you could say that. It is implemented in this manner.

Re: rvalue references

[bitwise via Digitalmars-d]

On Wed, 10 Jun 2015 04:52:46 -0400, kink no...@nowhere.com wrote:

I know what `in` currently means. Your proposed `in ref T` syntax is imo  
not much better than C++ `const T`, so I'd prefer a simple and  
convenient `in T`. Semantics would be identical to your `in ref` with  
the additional optimization for small POD types. And for beginners, one  
could simply describe it as:
'Use the in keyword for a parameter if you're not going to mutate it.  
Don't rely on its identity as the argument may be passed by value or  
reference, whatever seems more efficient for the compiler and the target  
platform.'


+1

  Bit

Re: rvalue references

[kink via Digitalmars-d]

On Tuesday, 9 June 2015 at 20:25:28 UTC, Namespace wrote:
No opinions on the semantics for `in` I proposed earlier? `in 
T` would be something like a non-escapable `const auto ref T` 
accepting rvalues without codebloat and avoiding the 
indirection for small POD types T. Wouldn't that eliminate 99% 
of the use cases for `auto ref`, improve readability 
substantially and be the solution for all the C++ guys missing 
the rvalue-bindability of `const T`?


'in' means 'const scope' and 'scope ref' together with 'in ref' 
was proposed in DIP 36 which was rejected. Let's see if Andrei 
thinks that my current work is satisfying. I hope so. :)


Can you point me to the justifications for the rejection? It 
should be pretty obvious that something like that is required. 
Personally, I'd go with `ref` being non-escapable by default and 
hence providing implicit rvalue-bindability, and having to use 
`escapable ref` if the parameter may escape and disallowing 
rvalues for these parameters. I know it's never gonna happen, 
it's just what I'd prefer.


I know what `in` currently means. Your proposed `in ref T` syntax 
is imo not much better than C++ `const T`, so I'd prefer a 
simple and convenient `in T`. Semantics would be identical to 
your `in ref` with the additional optimization for small POD 
types. And for beginners, one could simply describe it as:
'Use the in keyword for a parameter if you're not going to mutate 
it. Don't rely on its identity as the argument may be passed by 
value or reference, whatever seems more efficient for the 
compiler and the target platform.'

Re: rvalue references

[Stewart Gordon via Digitalmars-d]

Apologies if I've missed something - I haven't had much time to keep up with the 
discussions lately.


What is the use case for rvalue references in a garbage-collected language?

To me, it sounds like people want this feature for D purely because C++ has it.

Stewart.

--
My email address is valid but not my primary mailbox and not checked regularly.  Please 
keep replies on the 'group where everybody may benefit.

Re: rvalue references

[Namespace via Digitalmars-d]

On Monday, 8 June 2015 at 22:58:15 UTC, bitwise wrote:
On Mon, 08 Jun 2015 16:17:33 -0400, Namespace 
rswhi...@gmail.com wrote:



Yes, the same goes for Dgame.


closed-source projects can also accept rvalue-refs now ;)

  Bit


Dgame is not closed-source. ;)

Re: rvalue references

[kink via Digitalmars-d]

On Tuesday, 9 June 2015 at 13:13:53 UTC, Steven Schveighoffer 
wrote:
It's actually faster to pass an rvalue by value, because it's 
constructed on the stack anyway.


I seriously doubt that's true for a large struct, e.g., something 
containing a large static array. Why move/copy the damn thing if 
I'm only going to read a tiny portion of it?


And please don't forget the ABI in that regard. E.g., on Win64, 
only POD types = 64 bits (and whose size is a power of 2) are 
really passed by value (in a register or on the stack); all other 
types are passed as reference to a dedicated copy allocated on 
the caller's stack. So in this case, the indirection is enforced 
by the ABI anyway. If the callee is not going to modify the 
parameter, the copy should obviously be elided, falling back to 
classical byref passing.

Re: rvalue references

[bitwise via Digitalmars-d]

On Tue, 09 Jun 2015 08:26:46 -0400, kink no...@nowhere.com wrote:


On Monday, 8 June 2015 at 20:16:13 UTC, bitwise wrote:
static Mat4 transform()(const auto ref Vec3 pos, const auto ref Vec3  
scale, const auto ref Quat rot);


Horrific.

static Mat4 transform(in Vec3 pos, in Vec3 scale, in Quat rot);

would be so much better...


How do you figure?

  Bit

Re: rvalue references

[Steven Schveighoffer via Digitalmars-d]

On 6/9/15 10:53 AM, kink wrote:

On Tuesday, 9 June 2015 at 13:13:53 UTC, Steven Schveighoffer wrote:

It's actually faster to pass an rvalue by value, because it's
constructed on the stack anyway.


I seriously doubt that's true for a large struct, e.g., something
containing a large static array. Why move/copy the damn thing if I'm
only going to read a tiny portion of it?


Because it's not moved. It's written in the stack where it will be 
passed to the next function.


Then access to the data is done via stack pointer offsets instead of an 
extra indirection.


-Steve

Re: rvalue references

[bitwise via Digitalmars-d]

On Tue, 09 Jun 2015 07:04:36 -0400, Stewart Gordon smjg_1...@yahoo.com  
wrote:


Apologies if I've missed something - I haven't had much time to keep up  
with the discussions lately.


What is the use case for rvalue references in a garbage-collected  
language?


To me, it sounds like people want this feature for D purely because C++  
has it.


Stewart.




If you have a function that takes a huge struct, like, as Namespace says,  
containing a float[16], you would want to pass by reference.


Currently, given the following code,

struct Mat4 {
float elements[16];
}

void TakeMatrix(ref Mat4 m) { }

Then you must use it like this:

Mat4 tmp = Mat4();
TakeMatrix(tmp);

With r-value references(auto ref), you can do the following, and 'tmp'  
will be 'auto'matically created for you:


TakeMatrix(Mat4());

  Bit

Re: rvalue references

[bitwise via Digitalmars-d]

On Tue, 09 Jun 2015 07:04:36 -0400, Stewart Gordon smjg_1...@yahoo.com  
wrote:
To me, it sounds like people want this feature for D purely because C++  
has it.


Stewart.


That's actually not a bad reason ;)

Suppose we wanted to write bindings for some C++ code which made use of  
const-ref parameters:

http://help.autodesk.com/view/FBX/2015/ENU/?guid=__cpp_ref_class_fbx_a_matrix_html


  Bit

Re: rvalue references

[kink via Digitalmars-d]

On Tuesday, 9 June 2015 at 15:08:07 UTC, Steven Schveighoffer 
wrote:
Because it's not moved. It's written in the stack where it will 
be passed to the next function.


Hmm, you mean the callee's function parameters stack? That's not 
always going to work, e.g., on Win64 the first 4 args are passed 
in registers, always. And, as I said, that ABI doesn't support 
byval passing of types  64 bits (let's exclude vector types 
here), so rvalues  64 bits can sadly not be constructed in-place 
without violating the Win64 ABI - they'll have to be passed byref.

Re: rvalue references

[Steven Schveighoffer via Digitalmars-d]

On 6/9/15 12:23 PM, kink wrote:

On Tuesday, 9 June 2015 at 15:08:07 UTC, Steven Schveighoffer wrote:

Because it's not moved. It's written in the stack where it will be
passed to the next function.


Hmm, you mean the callee's function parameters stack? That's not always
going to work, e.g., on Win64 the first 4 args are passed in registers,
always.


If that's what's required, that's what we do. I'm not strictly saying 
that we will *always* do stack passing in spite of the ABI. But passing 
a large rvalue by value does not involve any moving of data, and can be 
abstracted to a pass by ref if needed. It's never less performing than 
an explicit pass by ref.



And, as I said, that ABI doesn't support byval passing of types
  64 bits (let's exclude vector types here), so rvalues  64 bits can
sadly not be constructed in-place without violating the Win64 ABI -
they'll have to be passed byref.


I don't think we violate the Win64 ABI. So if it requires pass by ref, 
it requires pass by ref. I am not an expert in this matter. You may want 
to try some disassembly and see what D does.


-Steve

Re: rvalue references

[kinke via Digitalmars-d]

On Tuesday, 9 June 2015 at 17:38:00 UTC, Steven Schveighoffer 
wrote:
But passing a large rvalue by value does not involve any moving 
of data, and can be abstracted to a pass by ref if needed. It's 
never less performing than an explicit pass by ref.


Thanks Steve.

The problem I see here on Win64 is that the current `auto ref` 
implementation for templates, i.e., 2^N function instantiations 
for N lvalue/rvalue combinations, seems totally useless for value 
types for which the ABI enforces pass-by-ref anyway.


In essence, `auto ref T` for templated functions on Win64 boils 
down to a simple `ref` accepting rvalue arguments iff T cannot be 
passed by value. All what's needed is constructing the rvalue on 
the caller's stack and destroying it after the call; the callee 
isn't affected at all, so no need for 2^N function versions, 1 is 
enough. Note that I haven't inspected the DMD code yet, but I 
assume it makes no exception for Win64 in that regard.


No opinions on the semantics for `in` I proposed earlier? `in T` 
would be something like a non-escapable `const auto ref T` 
accepting rvalues without codebloat and avoiding the indirection 
for small POD types T. Wouldn't that eliminate 99% of the use 
cases for `auto ref`, improve readability substantially and be 
the solution for all the C++ guys missing the rvalue-bindability 
of `const T`?

Re: rvalue references

[Namespace via Digitalmars-d]

No opinions on the semantics for `in` I proposed earlier? `in 
T` would be something like a non-escapable `const auto ref T` 
accepting rvalues without codebloat and avoiding the 
indirection for small POD types T. Wouldn't that eliminate 99% 
of the use cases for `auto ref`, improve readability 
substantially and be the solution for all the C++ guys missing 
the rvalue-bindability of `const T`?


'in' means 'const scope' and 'scope ref' together with 'in ref' 
was proposed in DIP 36 which was rejected. Let's see if Andrei 
thinks that my current work is satisfying. I hope so. :)

Re: rvalue references

[kink via Digitalmars-d]

On Monday, 8 June 2015 at 20:16:13 UTC, bitwise wrote:
static Mat4 transform()(const auto ref Vec3 pos, const auto ref 
Vec3 scale, const auto ref Quat rot);


Horrific.

static Mat4 transform(in Vec3 pos, in Vec3 scale, in Quat rot);

would be so much better...

Re: rvalue references

[Namespace via Digitalmars-d]

On Tuesday, 9 June 2015 at 11:04:43 UTC, Stewart Gordon wrote:
Apologies if I've missed something - I haven't had much time to 
keep up with the discussions lately.


What is the use case for rvalue references in a 
garbage-collected language?


To me, it sounds like people want this feature for D purely 
because C++ has it.


Stewart.


What does this have to do with garbage-collected language?
If I have a big struct, e.g.

struct Matrix {
float[16] values = [...];
}

I always want to pass it by ref because a move or a copy would be 
too slow.

Re: rvalue references

[Steven Schveighoffer via Digitalmars-d]

On 6/9/15 8:14 AM, Namespace wrote:

On Tuesday, 9 June 2015 at 11:04:43 UTC, Stewart Gordon wrote:

Apologies if I've missed something - I haven't had much time to keep
up with the discussions lately.

What is the use case for rvalue references in a garbage-collected
language?

To me, it sounds like people want this feature for D purely because
C++ has it.

Stewart.


What does this have to do with garbage-collected language?
If I have a big struct, e.g.

struct Matrix {
 float[16] values = [...];
}

I always want to pass it by ref because a move or a copy would be too slow.


It's actually faster to pass an rvalue by value, because it's 
constructed on the stack anyway.


The real reason for this I think is to avoid building multiple functions 
that have identical implementation.


-Steve

Re: rvalue references

[bitwise via Digitalmars-d]

On Mon, 08 Jun 2015 16:17:33 -0400, Namespace rswhi...@gmail.com wrote:


Yes, the same goes for Dgame.


closed-source projects can also accept rvalue-refs now ;)

  Bit

Re: rvalue references

[bitwise via Digitalmars-d]

On Mon, 08 Jun 2015 11:29:56 -0400, bitwise bitwise@gmail.com wrote:


On Mon, 08 Jun 2015 02:44:46 -0400, Namespace rswhi...@gmail.com wrote:


On Sunday, 7 June 2015 at 18:40:42 UTC, bitwise wrote:
On Sat, 06 Jun 2015 14:05:54 -0400, Namespace rswhi...@gmail.com  
wrote:



Finally all green! Now we need a review.


You're my hero.

  Bit


Sounds ironic. o.O



Ironic or Sarcastic?
Neither.

   Bit



One useful case for me:

static Mat4 transform()(const auto ref Vec3 pos, const auto ref Vec3  
scale, const auto ref Quat rot)

{
Mat4 m = Mat4(rot.matrix, 1);

m.m00 *= scale.x;
m.m01 *= scale.x;
m.m02 *= scale.x;

m.m10 *= scale.y;
m.m11 *= scale.y;
m.m12 *= scale.y;

m.m20 *= scale.z;
m.m21 *= scale.z;
m.m22 *= scale.z;

m.m30 = pos.x;
m.m31 = pos.y;
m.m32 = pos.z;

return m;
}

Currently, my choices are template bloat, or pointless copying.

  Bit

Re: rvalue references

[Namespace via Digitalmars-d]

One useful case for me:

static Mat4 transform()(const auto ref Vec3 pos, const auto ref 
Vec3 scale, const auto ref Quat rot)

{
Mat4 m = Mat4(rot.matrix, 1);

m.m00 *= scale.x;
m.m01 *= scale.x;
m.m02 *= scale.x;

m.m10 *= scale.y;
m.m11 *= scale.y;
m.m12 *= scale.y;

m.m20 *= scale.z;
m.m21 *= scale.z;
m.m22 *= scale.z;

m.m30 = pos.x;
m.m31 = pos.y;
m.m32 = pos.z;

return m;
}

Currently, my choices are template bloat, or pointless copying.

  Bit


Yes, the same goes for Dgame.

Re: rvalue references

[Manu via Digitalmars-d]

On 9 June 2015 at 08:58, bitwise via Digitalmars-d
digitalmars-d@puremagic.com wrote:
 On Mon, 08 Jun 2015 16:17:33 -0400, Namespace rswhi...@gmail.com wrote:

 Yes, the same goes for Dgame.


 closed-source projects can also accept rvalue-refs now ;)

How?

Re: rvalue references

[bitwise via Digitalmars-d]

On Mon, 08 Jun 2015 22:16:27 -0400, Manu via Digitalmars-d  
digitalmars-d@puremagic.com wrote:



On 9 June 2015 at 08:58, bitwise via Digitalmars-d
digitalmars-d@puremagic.com wrote:
On Mon, 08 Jun 2015 16:17:33 -0400, Namespace rswhi...@gmail.com  
wrote:



Yes, the same goes for Dgame.



closed-source projects can also accept rvalue-refs now ;)


How?


I mean that if this is implemented, your code doesn't have to be a  
template to use 'auto ref', so the code won't show up in a .di file.


  Bit

Re: rvalue references

[Namespace via Digitalmars-d]

On Sunday, 7 June 2015 at 18:40:42 UTC, bitwise wrote:
On Sat, 06 Jun 2015 14:05:54 -0400, Namespace 
rswhi...@gmail.com wrote:



Finally all green! Now we need a review.


You're my hero.

  Bit


Sounds ironic. o.O

Re: rvalue references

[bitwise via Digitalmars-d]

On Mon, 08 Jun 2015 02:44:46 -0400, Namespace rswhi...@gmail.com wrote:


On Sunday, 7 June 2015 at 18:40:42 UTC, bitwise wrote:
On Sat, 06 Jun 2015 14:05:54 -0400, Namespace rswhi...@gmail.com  
wrote:



Finally all green! Now we need a review.


You're my hero.

  Bit


Sounds ironic. o.O



Ironic or Sarcastic?
Neither.

  Bit

Re: rvalue references

[bitwise via Digitalmars-d]

On Sat, 06 Jun 2015 14:05:54 -0400, Namespace rswhi...@gmail.com wrote:


Finally all green! Now we need a review.


You're my hero.

  Bit

Re: rvalue references

[kinke via Digitalmars-d]

On Wednesday, 3 June 2015 at 01:57:21 UTC, bitwise wrote:
'in' is currently useless because scope is not defined 
properly, and const is too restrictive. Also, because of DIP25, 
it's now even more useless. It seems a pretty sure bet that it 
will either continue to be completely useless or be changed at 
some point anyways, so why not now?

...
So why not re-brand this awesomely concise and convenient 
keyword to be the non-template auto ref?


I see the point in some sort of `auto ref`/`scope ref` parameters 
binding to rvalues in cases where one's not interested in a 
function's side effect applied to a non-escapable rvalue argument.


But what I need 99% of the time is some means to express my 
intention to pass an argument the most efficient way possible as 
the callee is only gonna read from it and won't let it escape. 
This is only relevant for non-primitive value types, i.e., 
structs. Like many others, I love the neat little `in` keyword, 
and would like for it to have exactly these semantics, for both 
templates and normal functions:


* the callee has read-only access to the parameter, i.e.,
  it is const;
* since it's declared as normal function parameter (`in T arg`),
  it cannot escape (`scope` or however we wanna call this);
* depending on type and hardware, value types are either passed
  by value (POD type whose size = [2*]size_t.sizeof) or by
  reference (non-POD type and/or size  threshold);
  reference types (class instances) are always passed by ref
* if a value type is passed by reference, an rvalue argument is
  allowed to safely bind to the const, non-escapable reference.

This would be particularly useful for generic templates, e.g., 
containers, where primitive and small POD types T are ideally 
passed by value, and other Ts by reference.

Re: rvalue references

[Namespace via Digitalmars-d]

Finally all green! Now we need a review.

Re: rvalue references

[Namespace via Digitalmars-d]

I start working on a Pull 
(https://github.com/D-Programming-Language/dmd/pull/4717), but it 
fails the first check. The reason seems to be this: 
https://github.com/D-Programming-Language/dmd/pull/4717/files#diff-ffafa03255a57832dd09031af6cb915dR5945
I guess that this error happens because I cannot distinguish 
between template and non template auto ref. Does anyone have an 
idea?

Re: rvalue references

[Namespace via Digitalmars-d]

On Friday, 5 June 2015 at 21:31:22 UTC, Namespace wrote:
I start working on a Pull 
(https://github.com/D-Programming-Language/dmd/pull/4717), but 
it fails the first check. The reason seems to be this: 
https://github.com/D-Programming-Language/dmd/pull/4717/files#diff-ffafa03255a57832dd09031af6cb915dR5945
I guess that this error happens because I cannot distinguish 
between template and non template auto ref. Does anyone have an 
idea?


Hmm, the only problematic code is std.algorithm.mutation.swap on 
line 1956 - 1959 which calls doesPointTo from std.exception which 
is nothrow pure and @nogc, but shouldn't.

Without it, everything works.

Re: rvalue references

[Timon Gehr via Digitalmars-d]

On 06/03/2015 05:43 AM, bitwise wrote:




Why can't const ref accept rvalues?

const being too restrictive doesn't seem like a real reason, because
although its undesirable for some cases, doesn't mean it can't be useful
in a _lot_ of other cases.


Its a real reason unless you endorse patchwork language design. This is 
an obvious hole. There is absolutely no valid reason to solve only part 
of the issue in a non-orthogonal fashion.

Re: rvalue references

[Timon Gehr via Digitalmars-d]

On 06/02/2015 11:21 PM, Andrei Alexandrescu wrote:




Yah, auto ref for templates is great. We only need to add auto ref for
non-templates with the semantics like ref, except (a) accepts rvalues
on the caller side, (b) does not allow escaping the ref from the function.



What if one wants those semantics for some template function parameters?
It seems arbitrary to introduce this feature for all functions except 
templated ones.

Re: rvalue references

[Namespace via Digitalmars-d]

This code needs to be disallowed under DIP25 (or whatever the 
final DIP will be), of course.


But should work with return ref instead.

Re: rvalue references

[via Digitalmars-d]

On Tuesday, 2 June 2015 at 18:06:32 UTC, Namespace wrote:

On Tuesday, 2 June 2015 at 17:22:07 UTC, Marc Schütz wrote:

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
Thanks to DIP 25 I think it's time to review this again. I 
would implement it (if no one else wants to do it), but there 
are still some unanswered questions:


1. Is 'auto ref' still the chosen syntax (I guess so)?


Why not `scope ref` (or `in ref` == `const scope ref`)?


See DIP 36


I don't see an argument against `scope ref` in DIP36, quite the 
opposite...

Re: rvalue references

[via Digitalmars-d]

On Tuesday, 2 June 2015 at 17:31:56 UTC, Jonathan M Davis wrote:

On Tuesday, 2 June 2015 at 17:22:07 UTC, Marc Schütz wrote:

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
Thanks to DIP 25 I think it's time to review this again. I 
would implement it (if no one else wants to do it), but there 
are still some unanswered questions:


1. Is 'auto ref' still the chosen syntax (I guess so)?


Why not `scope ref` (or `in ref` == `const scope ref`)?


Because scope isn't even properly defined. Certainly, if we 
were to go that route, we'd have to finish working out what the 
heck scope is really supposed to do and mean. And we haven't 
done that yet. As it stands, everyone has their own ideas about 
what it means and/or should mean, but all the spec says for 
scope parameters is that references in the parameter cannot be 
escaped (e.g. assigned to a global variable), and right now, 
the only thing that scope affects is delegate parameters, and 
I'm not sure that even that works correctly yet or is properly 
ironed out.


So, while maybe using scope ref for this would make sense, we 
have a _lot_ to figure out before we can really consider that.




That's what makes it an ideal choice in my opinion :-) Whatever 
semantics we give it is by definition not a breaking change...


Besides, I do have a pretty good idea what `scope` should mean, 
including most of the finer details. I summarized it here:

http://wiki.dlang.org/User:Schuetzm/scope3

The lack of progress on this topic is not because no solution is 
known.



4. What's with this constellation:

struct S { }

void ene(S) { }
void mene(ref S) { }
void muh(auto ref S) { }

should 'mene' (ref) interfere with 'muh' (auto ref)?


If overloading on `scope` is allowed, then yes, else no.


It's not currently allowed. But regardless, if the whole point 
of saying scope ref (or whatever attribute we picked) was to 
indicate that we wanted the parameter to accept both lvalues 
and rvalues, then it makes no sense whatsoever to overload the 
parameter on ref, and it would be ref underneath the hood 
anyway, because that's how it would have to be implemented.


There are use-cases for overloading on scope with value types, 
but there are other ways to achieve the same goals (see here: 
http://wiki.dlang.org/User:Schuetzm/scope3#scope_for_value_types_.26_overloading).

Re: rvalue references

[via Digitalmars-d]

On Wednesday, 3 June 2015 at 07:09:09 UTC, Namespace wrote:

On Wednesday, 3 June 2015 at 03:57:38 UTC, bitwise wrote:

I forgot to mention, in terms of this statement I made:

I can't remember right now what the reasoning was for 'const 
ref' not to take
rvalues in the first place. I think it was that you could 
escape the reference,

but this isn't true anymore with DIP25 right?


I think someone brought this up about a weeks ago,

That was me.

and this was Andrei's response:

Knee-jerk response: if no return attribute on a function it 
should be safeto bind rvalues to ref parameters. Of course 
that's impractical as a default

so explicit auto ref would be needed. -- Andrei


What's impractical?


It's dangerous. If rvalues could be passed to normal ref 
parameter we would have a problem. Consider this:



struct Sprite {
private Texture* _tex;

this(ref Texture tex) {
_tex = tex;
}
}


This works because we accept a valid lvalue and store a pointer 
to it. If the CTor would accept also rvalues, then the pointer 
would be invalid, since the texture was just a temporary 
variable. Not the best example, but I think you understand what 
I mean.


This code needs to be disallowed under DIP25 (or whatever the 
final DIP will be), of course.

Re: rvalue references

[Namespace via Digitalmars-d]

On Wednesday, 3 June 2015 at 10:07:41 UTC, Marc Schütz wrote
I don't see an argument against `scope ref` in DIP36, quite the 
opposite...


I know I was one of the authors. But it was rejected.

Re: rvalue references

[via Digitalmars-d]

On Wednesday, 3 June 2015 at 09:53:36 UTC, Namespace wrote:
This code needs to be disallowed under DIP25 (or whatever the 
final DIP will be), of course.


But should work with return ref instead.


It can even be allowed with an extension to DIP25:

struct Sprite {
private Texture* _tex;

this(ref Texture tex return(this)) {
_tex = tex;
}
}

`return(param)` (or e.g. `out(param)`) means that the reference 
can escape through `param`.


The method can't accept an rvalue ref then, of course.

Re: rvalue references

[Jonathan M Davis via Digitalmars-d]

On Wednesday, 3 June 2015 at 03:43:09 UTC, bitwise wrote:
I can't remember right now what the reasoning was for 'const 
ref' not to take
rvalues in the first place. I think it was that you could 
escape the reference,

but this isn't true anymore with DIP25 right?


The one to ask is Andrei. I can never remember all of the 
details, because it has to do with the details of how it causes 
problems in C++. IIRC, the problem stems from the fact that you 
can no longer tell whether the parameter represents an lvalue or 
a temporary. I don't know how big a deal it really is, but Andrei 
is emphatic about it.


But regardless, because of how restrictive const is in D, 
requiring const to have a parameter accept both lvalues and 
rvalues is just too restrictive anyway. A lot of the same 
developers who are insistent on having rvalue references for 
efficiency are exactly the sort of developers who will avoid 
const, because it's physical const and not logical const and thus 
prevents stuff like caching or lazy loading.


- Jonathan M Davis

Re: rvalue references

[Namespace via Digitalmars-d]

On Wednesday, 3 June 2015 at 03:57:38 UTC, bitwise wrote:

I forgot to mention, in terms of this statement I made:

I can't remember right now what the reasoning was for 'const 
ref' not to take
rvalues in the first place. I think it was that you could 
escape the reference,

but this isn't true anymore with DIP25 right?


I think someone brought this up about a weeks ago,

That was me.

and this was Andrei's response:

Knee-jerk response: if no return attribute on a function it 
should be safeto bind rvalues to ref parameters. Of course 
that's impractical as a default

so explicit auto ref would be needed. -- Andrei


What's impractical?


It's dangerous. If rvalues could be passed to normal ref 
parameter we would have a problem. Consider this:



struct Sprite {
private Texture* _tex;

this(ref Texture tex) {
_tex = tex;
}
}


This works because we accept a valid lvalue and store a pointer 
to it. If the CTor would accept also rvalues, then the pointer 
would be invalid, since the texture was just a temporary 
variable. Not the best example, but I think you understand what I 
mean.

Re: rvalue references

[bitwise via Digitalmars-d]

On Wed, 03 Jun 2015 03:09:06 -0400, Namespace rswhi...@gmail.com wrote:


On Wednesday, 3 June 2015 at 03:57:38 UTC, bitwise wrote:

I forgot to mention, in terms of this statement I made:

I can't remember right now what the reasoning was for 'const ref' not  
to take
rvalues in the first place. I think it was that you could escape the  
reference,

but this isn't true anymore with DIP25 right?


I think someone brought this up about a weeks ago,

That was me.

and this was Andrei's response:

Knee-jerk response: if no return attribute on a function it should  
be safeto bind rvalues to ref parameters. Of course that's impractical  
as a default

so explicit auto ref would be needed. -- Andrei


What's impractical?


It's dangerous. If rvalues could be passed to normal ref parameter we  
would have a problem. Consider this:



struct Sprite {
 private Texture* _tex;

 this(ref Texture tex) {
 _tex = tex;
 }
}


This works because we accept a valid lvalue and store a pointer to it.  
If the CTor would accept also rvalues, then the pointer would be  
invalid, since the texture was just a temporary variable. Not the best  
example, but I think you understand what I mean.



Hmm Indeed. It seems I misunderstood DIP25. I didn't think that was  
allowed.


It appears that even this is allowed:

private Texture* _tex;
struct Sprite {
this(ref Texture tex) {
_tex = tex;
}
}

Is DIP25 intended to eventually prevent these examples?

Wouldn't it have made more sense for DIP25 prevent _all_ escaping,  
including pointers? or is that not possible?

Then, instead of 'return ref' it could be 'escape ref' or something.
Anyways, moot point I suppose.

  Bit

Re: rvalue references

[Namespace via Digitalmars-d]

On Tuesday, 2 June 2015 at 17:22:07 UTC, Marc Schütz wrote:

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
Thanks to DIP 25 I think it's time to review this again. I 
would implement it (if no one else wants to do it), but there 
are still some unanswered questions:


1. Is 'auto ref' still the chosen syntax (I guess so)?


Why not `scope ref` (or `in ref` == `const scope ref`)?


See DIP 36

2. Should auto ref for templates act like auto ref for 
non-templates (creates a temporary and pass it by ref) or 
should the current behaviour stay (duplicate the function 2^N 
times)?


With `scope ref`, obviously it should stay as-is.

3. What's with return ref, should auto ref for non-templates 
include return ref (like auto ref for templates)?


Like above, no.


4. What's with this constellation:

struct S { }

void ene(S) { }
void mene(ref S) { }
void muh(auto ref S) { }

should 'mene' (ref) interfere with 'muh' (auto ref)?


If overloading on `scope` is allowed, then yes, else no.

Re: rvalue references

[Jonathan M Davis via Digitalmars-d]

On Tuesday, 2 June 2015 at 18:05:20 UTC, Namespace wrote:
AFAIK Andrei wanted 'auto ref' as the syntax which accepts 
both, lvalues and rvalues. That's why I'm asking if the current 
behaviour for auto ref for templates should change, or not. If 
not, we have (as you said) two meanings of auto ref, what is 
not optimal, but not so bad either (IMO).
But if I had the choice, I would change it for both, so that 
both create a temporary variable for rvalues.


Andrei originally wanted auto ref to apply to non-templated 
functions, but Walter misunderstood what he meant and created 
what we have now - which is actually incredibly useful, because 
it allows you to forward the attributes of an argument - in 
particular, it's ref-ness. So, we definitely do _not_ want to 
lose that.



So, basically, I think that we have three options:

1. Do nothing. If you want a function parameter to accept both 
lvalues and rvalues efficently, then either duplicate it with 
various overloads to achieve that or templatize it and use 
auto ref so that the compiler will do that for you.
But that means we get (at worst) 2^N functions. And note that 
each function contains the whole body, not just a call to one 
of the previous creations.


Yes. auto ref with templated functions results in a lot of 
template bloat if you use it heavily, which is part of why using 
auto ref with non-templated functions at this point isn't a 
particularly ideal solution, not unless we could actually make it 
so that the compiler could optimize the extraneous instantiations 
and use a temporary value for the rvalues when the attribute 
forwarding isn't actually needed, but I wouldn't really expect us 
to get that.


- Jonathan M Davis

Re: rvalue references

[Namespace via Digitalmars-d]

3. Add a new attribute - e.g. @rvalue ref - which inserts a 
temporary variable for rvalues so that they can be passed by 
ref, and it works with both templated and non-templated 
functions.


We could also somehow use 'return ref' for that purpose, or we 
could get rid of the pointless 'const scope' alias 'in' and use 
'in ref' as a new attribute. But somehow I would prefer 'auto 
ref', it's a more descriptive term.

Re: rvalue references

[Namespace via Digitalmars-d]

auto ref with templated functions needs to retain its current 
behavior. Changing it would not only break existing code, but 
it would lose what we have in terms of perfect forwarding 
(IIRC, it's not quite perfect forwarding, but it's close, and 
we'd be much farther from it without auto ref).

Ok, I thought so too

If we want to have non-templated functions which can accept 
both rvalues and lvalues without copying the lvalues, then we 
need have a way to mark those parameters so that they're ref 
and then have an invisible, temporary variable inserted to hold 
a copy of an rvalue so that it can be passed by ref and 
accepted by the function as well instead of just accepting 
lvalues.
That is what auto ref for non-templates would do (and what I 
planned to do): if the passed argument is not an lvalue, a 
temporary is constructed which is passed by ref.


If we want that to work with both non-templated and templated 
functions, then we need a new syntax. If we're willing to have 
them work with just non-templated functions, then we could 
reuse auto ref for that (and _maybe_ we could have the compiler 
optimize auto ref on templates to mean the same thing when it 
can determine that it's safe to do so and thus avoid extra 
template instantiations, but I question that that will happen). 
But then we only get it for non-templated functions, and auto 
ref means somethings slightly different for templated and 
non-templated functions, which sucks, but I'm not sure that 
it's ultimately all that big a deal.
AFAIK Andrei wanted 'auto ref' as the syntax which accepts both, 
lvalues and rvalues. That's why I'm asking if the current 
behaviour for auto ref for templates should change, or not. If 
not, we have (as you said) two meanings of auto ref, what is not 
optimal, but not so bad either (IMO).
But if I had the choice, I would change it for both, so that both 
create a temporary variable for rvalues.


I _definitely_ think that it would be a huge mistake for ref in 
general to accept rvalues. If we did that, then suddenly, ref 
would be used everywhere, and you couldn't tell when someone 
wanted to actually set the ref argument to something or whether 
they were just trying to avoid extraneous copies of lvalues.

I agree with that 100%.

I'd much rather have no way to have a parameter accept both 
rvalues and lvalues without copying the lvalues with 
non-templated functions than have ref accept rvalues.


So, basically, I think that we have three options:

1. Do nothing. If you want a function parameter to accept both 
lvalues and rvalues efficently, then either duplicate it with 
various overloads to achieve that or templatize it and use auto 
ref so that the compiler will do that for you.
But that means we get (at worst) 2^N functions. And note that 
each function contains the whole body, not just a call to one of 
the previous creations.

That is really huge.
If it would be only a call to one of the previous creations like:

struct S { }

void test()(auto ref S s) {

}

test(S());
S s = S();
test(s);


is expaned to


void test(ref S s) {

}

void test(S s) {
test(s); // Note that
}

it would be fine. But not optimal either, since these functions 
can contain bugs which are never explored if your coverage is not 
100%, because template methods are only instantiated if they are 
called. (Hope you know what I mean, my english is not that 
pretty)


2. Extend auto ref so that it works with non-templated 
functions by inserting a temporary variable for rvalues so that 
they can be passed to the function by ref. Templated functions 
stay as they are.

This is what I wanted to do.

3. Add a new attribute - e.g. @rvalue ref - which inserts a 
temporary variable for rvalues so that they can be passed by 
ref, and it works with both templated and non-templated 
functions.


Right now, we seem to be doing #1, and we have never officially 
decided whether that's permanent. Andrei in particular has 
resisted adding a new attribute, and to some extent, I agree 
that that's undesirable, but it _would_ allow us to solve this 
problem for both templated and non-templated functions, which 
we can't really do otherwise. So, I don't know how reasonable 
or feasible #3 is at this point. #2 probably wouldn't be that 
hard to get in, but then it only works with non-templated 
functions, and it complicates the meaning of auto ref in an 
already complicated language.


Honestly, at this point, I don't know how much this issue 
really matters. It's annoying that we don't have rvalue 
references, but in general, we're living without them just 
fine, and we're heading toward templatizing almost everything 
for ranges anyway, in which case, the current version of auto 
ref will work just fine with most code (though that extraneous 
template bloat _is_ ugly). So, while I'd like to have rvalue 
references, I also think that we can get by just fine without 
them.


If we _were_ going to add rvalue references, at 

Re: rvalue references

[Namespace via Digitalmars-d]

On Tuesday, 2 June 2015 at 21:20:49 UTC, Andrei Alexandrescu 
wrote:
Yah, auto ref for templates is great. We only need to add auto 
ref for non-templates with the semantics like ref, except (a) 
accepts rvalues on the caller side, (b) does not allow escaping 
the ref from the function.
That would work once DIP 25 is fully implemented and if auto ref 
for non-templates would not include return ref. Or am I missing 
something?


Note that __traits(isRef, param) doesn't work in an auto ref 
param in a non-template function (attempt to use must be a 
compile-time error).


That's about it.


Andrei

Re: rvalue references

[Andrei Alexandrescu via Digitalmars-d]

On 6/2/15 11:19 AM, Jonathan M Davis wrote:

On Tuesday, 2 June 2015 at 18:05:20 UTC, Namespace wrote:

AFAIK Andrei wanted 'auto ref' as the syntax which accepts both,
lvalues and rvalues. That's why I'm asking if the current behaviour
for auto ref for templates should change, or not. If not, we have (as
you said) two meanings of auto ref, what is not optimal, but not so
bad either (IMO).
But if I had the choice, I would change it for both, so that both
create a temporary variable for rvalues.


Andrei originally wanted auto ref to apply to non-templated functions,
but Walter misunderstood what he meant and created what we have now -
which is actually incredibly useful, because it allows you to forward
the attributes of an argument - in particular, it's ref-ness. So, we
definitely do _not_ want to lose that.


Yah, auto ref for templates is great. We only need to add auto ref for 
non-templates with the semantics like ref, except (a) accepts rvalues 
on the caller side, (b) does not allow escaping the ref from the function.


Note that __traits(isRef, param) doesn't work in an auto ref param in a 
non-template function (attempt to use must be a compile-time error).


That's about it.


Andrei

Re: rvalue references

[Andrei Alexandrescu via Digitalmars-d]

On 6/2/15 2:28 PM, Namespace wrote:

On Tuesday, 2 June 2015 at 21:20:49 UTC, Andrei Alexandrescu wrote:

Yah, auto ref for templates is great. We only need to add auto ref for
non-templates with the semantics like ref, except (a) accepts rvalues
on the caller side, (b) does not allow escaping the ref from the
function.

That would work once DIP 25 is fully implemented and if auto ref for
non-templates would not include return ref. Or am I missing something?


Sounds about right. -- Andrei

Re: rvalue references

[bitwise via Digitalmars-d]

I forgot to mention, in terms of this statement I made:

I can't remember right now what the reasoning was for 'const ref' not to  
take
rvalues in the first place. I think it was that you could escape the  
reference,

but this isn't true anymore with DIP25 right?


I think someone brought this up about a weeks ago, and this
was Andrei's response:

Knee-jerk response: if no return attribute on a function it should be  
safeto bind rvalues to ref parameters. Of course that's impractical as a  
default

so explicit auto ref would be needed. -- Andrei


What's impractical?

  Bit

Re: rvalue references

[bitwise via Digitalmars-d]

On Tue, 02 Jun 2015 14:06:31 -0400, Namespace rswhi...@gmail.com wrote:


On Tuesday, 2 June 2015 at 17:22:07 UTC, Marc Schütz wrote:

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
Thanks to DIP 25 I think it's time to review this again. I would  
implement it (if no one else wants to do it), but there are still some  
unanswered questions:


1. Is 'auto ref' still the chosen syntax (I guess so)?


Why not `scope ref` (or `in ref` == `const scope ref`)?


See DIP 36



I'm feeling like a bit of a troll here, but I can't help myself.

'in' is currently useless because scope is not defined properly, and const  
is too restrictive. Also, because of DIP25, it's now even more useless. It  
seems a pretty sure bet that it will either continue to be completely  
useless or be changed at some point anyways, so why not now? If it's only  
been around since 2.060, how bad could the code breakage really be?  
Replace All a few times and you're done. A compiler warning could be  
issued for the next few versions to notify users that the meaning of 'in'  
has changed, if needed.


So why not re-brand this awesomely concise and convenient keyword to be  
the non-template auto ref?


  Bit

Re: rvalue references

[Jonathan M Davis via Digitalmars-d]

On Wednesday, 3 June 2015 at 01:57:21 UTC, bitwise wrote:
On Tue, 02 Jun 2015 14:06:31 -0400, Namespace 
rswhi...@gmail.com wrote:



On Tuesday, 2 June 2015 at 17:22:07 UTC, Marc Schütz wrote:

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
Thanks to DIP 25 I think it's time to review this again. I 
would implement it (if no one else wants to do it), but 
there are still some unanswered questions:


1. Is 'auto ref' still the chosen syntax (I guess so)?


Why not `scope ref` (or `in ref` == `const scope ref`)?


See DIP 36



I'm feeling like a bit of a troll here, but I can't help myself.

'in' is currently useless because scope is not defined 
properly, and const is too restrictive. Also, because of DIP25, 
it's now even more useless. It seems a pretty sure bet that it 
will either continue to be completely useless or be changed at 
some point anyways, so why not now? If it's only been around 
since 2.060, how bad could the code breakage really be? Replace 
All a few times and you're done. A compiler warning could be 
issued for the next few versions to notify users that the 
meaning of 'in' has changed, if needed.


So why not re-brand this awesomely concise and convenient 
keyword to be the non-template auto ref?


Where on earth did you get the idea that in was introduced in 
2.060? in existed in D1. Its meaning was slightly different then, 
but it was kept in D2 and its meaning changed in a manner that 
was intended to be reasonably compatible with D1 code. And folks 
_love_ using in, because they view it as the opposite of out. 
Almost always, it's really just const, since scope really only 
applies to delegates at this point, but in is used _heavily_ by 
many folks in the D community.


I keep telling folks not to use in, because scope isn't 
well-defined yet, so who knows what's going to happen when we 
_do_ try and define it properly, but folks keep using it anyway. 
We can't just change the meaning and expect it not to break code. 
That's part of what's going to be so ugly about finishing scope. 
And there are folks who use in fully understanding the risks 
under the assumption that scope will be defined in a way that 
does what they want. So, many folks are relying on in being 
equivalent to const scope, and changing it to mean something else 
is just plain a bad idea.


- Jonathan M Davis

Re: rvalue references

[Manu via Digitalmars-d]

On 3 June 2015 at 07:21, Andrei Alexandrescu via Digitalmars-d
digitalmars-d@puremagic.com wrote:
 On 6/2/15 11:19 AM, Jonathan M Davis wrote:

 On Tuesday, 2 June 2015 at 18:05:20 UTC, Namespace wrote:

 AFAIK Andrei wanted 'auto ref' as the syntax which accepts both,
 lvalues and rvalues. That's why I'm asking if the current behaviour
 for auto ref for templates should change, or not. If not, we have (as
 you said) two meanings of auto ref, what is not optimal, but not so
 bad either (IMO).
 But if I had the choice, I would change it for both, so that both
 create a temporary variable for rvalues.


 Andrei originally wanted auto ref to apply to non-templated functions,
 but Walter misunderstood what he meant and created what we have now -
 which is actually incredibly useful, because it allows you to forward
 the attributes of an argument - in particular, it's ref-ness. So, we
 definitely do _not_ want to lose that.


 Yah, auto ref for templates is great. We only need to add auto ref for
 non-templates with the semantics like ref, except (a) accepts rvalues on
 the caller side, (b) does not allow escaping the ref from the function.

 Note that __traits(isRef, param) doesn't work in an auto ref param in a
 non-template function (attempt to use must be a compile-time error).

 That's about it.


You're killing me! I thought 'scope ref' was more-or-less agreed? auto
ref makes no sense.

Re: rvalue references

[Jonathan M Davis via Digitalmars-d]

On Wednesday, 3 June 2015 at 03:20:48 UTC, bitwise wrote:
Anyways, moving forward with the assumption that the meaning of 
'in' will
not change, I still don't understand. Why couldn't 'in ref' be 
allowed to

accept rvalues in addition to 'auto ref'?


For the same reasons that we can't have const ref accept rvalues, 
especially since in ref basically _is_ const ref in most cases 
right now.


- Jonathan M Davis

Re: rvalue references

[bitwise via Digitalmars-d]

On Tue, 02 Jun 2015 22:28:48 -0400, Jonathan M Davis jmdavisp...@gmx.com  
wrote:

Where on earth did you get the idea that in was introduced in 2.060?


DIP36:
in ref has been allowed from 2.060 :  
http://d.puremagic.com/issues/show_bug.cgi?id=8105;


Reading it more carefully this time, I understand what it's saying now =(


And folks _love_ using in, because they view it as the opposite of out.


To be symmetrical with 'out' shouldn't 'in' == 'const scope ref'?


Almost always, it's really just const, since scope really only applies
to delegates at this point, but in is used _heavily_ by many folks in
the D community.


I still think that it could be put to better use than an alias for const.

I keep telling folks not to use in, because scope isn't well-defined  
yet, so who knows what's going to happen when we _do_ try and define it  
properly, but folks keep using it anyway.


Exactly my point, because it's awesomely convenient, which is why it should
have a more useful meaning :)


We can't just change the meaning and expect it not to break code. That's
part of what's going to be so ugly about finishing scope. And there are
folks who use in fully understanding the risks under the assumption that
scope will be defined in a way that does what they want. So, many folks
are relying on in being equivalent to const scope, and changing it to
mean something else is just plain a bad idea.


I have seen a lot of posts on these forums about people being tired of
breaking changes in D, and I understand that, but I believe 'in' should be
an exception. It's kind of a 'beta' feature, and people should expect that
it's subject to change, especially considering that googling 'dlang in ref'
yields your warning as the second result :)

1. Don't use in.

Anyways, moving forward with the assumption that the meaning of 'in' will
not change, I still don't understand. Why couldn't 'in ref' be allowed to
accept rvalues in addition to 'auto ref'?

  Bit

Re: rvalue references

[bitwise via Digitalmars-d]

On Tue, 02 Jun 2015 23:23:00 -0400, Jonathan M Davis jmdavisp...@gmx.com  
wrote:



On Wednesday, 3 June 2015 at 03:20:48 UTC, bitwise wrote:
Anyways, moving forward with the assumption that the meaning of 'in'  
will
not change, I still don't understand. Why couldn't 'in ref' be allowed  
to

accept rvalues in addition to 'auto ref'?


For the same reasons that we can't have const ref accept rvalues,  
especially since in ref basically _is_ const ref in most cases right now.


- Jonathan M Davis


Why can't const ref accept rvalues?

const being too restrictive doesn't seem like a real reason, because
although its undesirable for some cases, doesn't mean it can't be useful
in a _lot_ of other cases.

Also, with DIP25, isn't 'const ref' the same thing that 'const scope ref'  
was

supposed to be anyways?

I can't remember right now what the reasoning was for 'const ref' not to  
take
rvalues in the first place. I think it was that you could escape the  
reference,

but this isn't true anymore with DIP25 right?

  Bit

Re: rvalue references

[anonymous via Digitalmars-d]

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
Thanks to DIP 25 I think it's time to review this again. I 
would implement it (if no one else wants to do it), but there 
are still some unanswered questions:


1. Is 'auto ref' still the chosen syntax (I guess so)?
2. Should auto ref for templates act like auto ref for 
non-templates (creates a temporary and pass it by ref) or 
should the current behaviour stay (duplicate the function 2^N 
times)?
3. What's with return ref, should auto ref for non-templates 
include return ref (like auto ref for templates)?

4. What's with this constellation:

struct S { }

void ene(S) { }
void mene(ref S) { }
void muh(auto ref S) { }

should 'mene' (ref) interfere with 'muh' (auto ref)?


Is there a final decision about rvalue references?

Background:
Is it still worth finishing and submitting a PR for 
https://issues.dlang.org/show_bug.cgi?id=11529 ?
I did some experiments on how to fix this and then moved to other 
stuff. Now I have time for a second try, but fixing this if 
rvalue reference will be allowed in future is waste of time.


Sorry for hijacking this thread...

Re: rvalue references

[Jonathan M Davis via Digitalmars-d]

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
Thanks to DIP 25 I think it's time to review this again. I 
would implement it (if no one else wants to do it), but there 
are still some unanswered questions:


1. Is 'auto ref' still the chosen syntax (I guess so)?
2. Should auto ref for templates act like auto ref for 
non-templates (creates a temporary and pass it by ref) or 
should the current behaviour stay (duplicate the function 2^N 
times)?
3. What's with return ref, should auto ref for non-templates 
include return ref (like auto ref for templates)?

4. What's with this constellation:

struct S { }

void ene(S) { }
void mene(ref S) { }
void muh(auto ref S) { }

should 'mene' (ref) interfere with 'muh' (auto ref)?


auto ref with templated functions needs to retain its current 
behavior. Changing it would not only break existing code, but it 
would lose what we have in terms of perfect forwarding (IIRC, 
it's not quite perfect forwarding, but it's close, and we'd be 
much farther from it without auto ref).


If we want to have non-templated functions which can accept both 
rvalues and lvalues without copying the lvalues, then we need 
have a way to mark those parameters so that they're ref and then 
have an invisible, temporary variable inserted to hold a copy of 
an rvalue so that it can be passed by ref and accepted by the 
function as well instead of just accepting lvalues.


If we want that to work with both non-templated and templated 
functions, then we need a new syntax. If we're willing to have 
them work with just non-templated functions, then we could reuse 
auto ref for that (and _maybe_ we could have the compiler 
optimize auto ref on templates to mean the same thing when it can 
determine that it's safe to do so and thus avoid extra template 
instantiations, but I question that that will happen). But then 
we only get it for non-templated functions, and auto ref means 
somethings slightly different for templated and non-templated 
functions, which sucks, but I'm not sure that it's ultimately all 
that big a deal.


I _definitely_ think that it would be a huge mistake for ref in 
general to accept rvalues. If we did that, then suddenly, ref 
would be used everywhere, and you couldn't tell when someone 
wanted to actually set the ref argument to something or whether 
they were just trying to avoid extraneous copies of lvalues. I'd 
much rather have no way to have a parameter accept both rvalues 
and lvalues without copying the lvalues with non-templated 
functions than have ref accept rvalues.


So, basically, I think that we have three options:

1. Do nothing. If you want a function parameter to accept both 
lvalues and rvalues efficently, then either duplicate it with 
various overloads to achieve that or templatize it and use auto 
ref so that the compiler will do that for you.


2. Extend auto ref so that it works with non-templated functions 
by inserting a temporary variable for rvalues so that they can be 
passed to the function by ref. Templated functions stay as they 
are.


3. Add a new attribute - e.g. @rvalue ref - which inserts a 
temporary variable for rvalues so that they can be passed by ref, 
and it works with both templated and non-templated functions.


Right now, we seem to be doing #1, and we have never officially 
decided whether that's permanent. Andrei in particular has 
resisted adding a new attribute, and to some extent, I agree that 
that's undesirable, but it _would_ allow us to solve this problem 
for both templated and non-templated functions, which we can't 
really do otherwise. So, I don't know how reasonable or feasible 
#3 is at this point. #2 probably wouldn't be that hard to get in, 
but then it only works with non-templated functions, and it 
complicates the meaning of auto ref in an already complicated 
language.


Honestly, at this point, I don't know how much this issue really 
matters. It's annoying that we don't have rvalue references, but 
in general, we're living without them just fine, and we're 
heading toward templatizing almost everything for ranges anyway, 
in which case, the current version of auto ref will work just 
fine with most code (though that extraneous template bloat _is_ 
ugly). So, while I'd like to have rvalue references, I also think 
that we can get by just fine without them.


If we _were_ going to add rvalue references, at this point, I'd 
probably lean towards a new attribute, because it's cleaner and 
more flexible that way, but it _does_ mean adding a new 
attribute, and I don't know if that's worth it.


- Jonathan M Davis

Re: rvalue references

[via Digitalmars-d]

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
Thanks to DIP 25 I think it's time to review this again. I 
would implement it (if no one else wants to do it), but there 
are still some unanswered questions:


1. Is 'auto ref' still the chosen syntax (I guess so)?


Why not `scope ref` (or `in ref` == `const scope ref`)?

2. Should auto ref for templates act like auto ref for 
non-templates (creates a temporary and pass it by ref) or 
should the current behaviour stay (duplicate the function 2^N 
times)?


With `scope ref`, obviously it should stay as-is.

3. What's with return ref, should auto ref for non-templates 
include return ref (like auto ref for templates)?


Like above, no.


4. What's with this constellation:

struct S { }

void ene(S) { }
void mene(ref S) { }
void muh(auto ref S) { }

should 'mene' (ref) interfere with 'muh' (auto ref)?


If overloading on `scope` is allowed, then yes, else no.

Re: rvalue references

[Jonathan M Davis via Digitalmars-d]

On Tuesday, 2 June 2015 at 17:22:07 UTC, Marc Schütz wrote:

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
Thanks to DIP 25 I think it's time to review this again. I 
would implement it (if no one else wants to do it), but there 
are still some unanswered questions:


1. Is 'auto ref' still the chosen syntax (I guess so)?


Why not `scope ref` (or `in ref` == `const scope ref`)?


Because scope isn't even properly defined. Certainly, if we were 
to go that route, we'd have to finish working out what the heck 
scope is really supposed to do and mean. And we haven't done that 
yet. As it stands, everyone has their own ideas about what it 
means and/or should mean, but all the spec says for scope 
parameters is that references in the parameter cannot be escaped 
(e.g. assigned to a global variable), and right now, the only 
thing that scope affects is delegate parameters, and I'm not sure 
that even that works correctly yet or is properly ironed out.


So, while maybe using scope ref for this would make sense, we 
have a _lot_ to figure out before we can really consider that.



4. What's with this constellation:

struct S { }

void ene(S) { }
void mene(ref S) { }
void muh(auto ref S) { }

should 'mene' (ref) interfere with 'muh' (auto ref)?


If overloading on `scope` is allowed, then yes, else no.


It's not currently allowed. But regardless, if the whole point of 
saying scope ref (or whatever attribute we picked) was to 
indicate that we wanted the parameter to accept both lvalues and 
rvalues, then it makes no sense whatsoever to overload the 
parameter on ref, and it would be ref underneath the hood anyway, 
because that's how it would have to be implemented.


- Jonathan M Davis

Re: rvalue references

[via Digitalmars-d]

On Thursday, 14 May 2015 at 06:56:47 UTC, Namespace wrote:

On Thursday, 14 May 2015 at 00:12:05 UTC, bitwise wrote:
On Tue, 12 May 2015 08:54:15 -0400, Namespace 
rswhi...@gmail.com wrote:


As far as I know, the problem (or at least one of the biggest 
problems) for rvalue references was that they could escape. 
Since DIP25 is approved and already implemented this problem 
should be solved. Would it be possible to allow rvalues 
references now? I'm just curious what the mindfactory of D 
has in mind and what the scheme for that problem is.



Side note: DIP36 seems to be missing the table with the 
authors, status, etc.


 Bit


Huh, DIP36? DIP36 was rejected, but the authors (me and 
Dicebot) are below.


I've heard that DIP69 is [quote]almost dead in favor of 
DIP25[/quote]. That's why I'm a bit confused and would like to 
know the scheme, if any. :)


There's been no official decision, AFAIK. I was just going from 
the impression I got from Walter  Andrei's posts. But `scope` as 
a storage class is still necessary, because there are many more 
kinds of references in D besides `ref`. You can think of `ref` 
implying `scope`. The next steps will be to a) complete the 
implementation of `return ref`, b) extend it to those other 
reference types like pointers, slices, classes and delegates, and 
c) close the holes (see the thread RCArray is unsafe).

Re: rvalue references

[bitwise via Digitalmars-d]

On Thu, 14 May 2015 02:56:46 -0400, Namespace rswhi...@gmail.com wrote:


On Thursday, 14 May 2015 at 00:12:05 UTC, bitwise wrote:



Side note: DIP36 seems to be missing the table with the authors,  
status, etc.


  Bit


Huh, DIP36? DIP36 was rejected, but the authors (me and Dicebot) are  
below.


Hmm.. so they are ;)

I've heard that DIP69 is [quote]almost dead in favor of DIP25[/quote].  
That's why I'm a bit confused and would like to know the scheme, if any.  
:)



Makes sense, but if scope is dead, I assume it would have to be removed at  
some point.


So I guess 'ref' could just take rvalues now that it's escape-proof right?

Also, what becomes of the 'in' storage class? being just another word for  
'const' seems like a waste.


If it were up to me, I would say:

'scope' should be removed as a storage class,
'ref' should be left as is, and
'in' should adopt the behavior of 'ref' post DIP25

  Bit

Re: rvalue references

[Namespace via Digitalmars-d]

On Thursday, 14 May 2015 at 17:50:35 UTC, Andrei Alexandrescu 
wrote:

On 5/14/15 10:15 AM, Namespace wrote:
But interesting question, what will happen with scope, now 
that we have

return ref.


Evaluate its merit once we have everything else in. -- Andrei


Short and meaningful. Thank you.

Re: rvalue references

[Namespace via Digitalmars-d]

On Thursday, 14 May 2015 at 16:53:04 UTC, bitwise wrote:
On Thu, 14 May 2015 02:56:46 -0400, Namespace 
rswhi...@gmail.com wrote:



On Thursday, 14 May 2015 at 00:12:05 UTC, bitwise wrote:



Side note: DIP36 seems to be missing the table with the 
authors, status, etc.


 Bit


Huh, DIP36? DIP36 was rejected, but the authors (me and 
Dicebot) are below.


Hmm.. so they are ;)

I've heard that DIP69 is [quote]almost dead in favor of 
DIP25[/quote]. That's why I'm a bit confused and would like to 
know the scheme, if any. :)



Makes sense, but if scope is dead, I assume it would have to be 
removed at some point.


So I guess 'ref' could just take rvalues now that it's 
escape-proof right?


Also, what becomes of the 'in' storage class? being just 
another word for 'const' seems like a waste.


If it were up to me, I would say:

'scope' should be removed as a storage class,
'ref' should be left as is, and
'in' should adopt the behavior of 'ref' post DIP25

  Bit


As Andrei said several times (recently here: 
http://forum.dlang.org/thread/scaufixfdyyrbmijg...@forum.dlang.org?page=3#post-mit5lp:241kpt:241:40digitalmars.com) 
auto ref will take care about rvalue references. I've only hoped 
that I could get a schedule for that or at least a confirmation. 
:)


But interesting question, what will happen with scope, now that 
we have return ref.

Re: rvalue references

[Andrei Alexandrescu via Digitalmars-d]

On 5/14/15 10:15 AM, Namespace wrote:

But interesting question, what will happen with scope, now that we have
return ref.


Evaluate its merit once we have everything else in. -- Andrei

Re: rvalue references

[Namespace via Digitalmars-d]

Don't ask me, I'm just a little light here.


But I think it's because 'in' means 'const scope' and as you 
said, scope is out. ;)

Re: rvalue references

[bitwise via Digitalmars-d]

On Thu, 14 May 2015 13:15:34 -0400, Namespace rswhi...@gmail.com wrote:

On Thursday, 14 May 2015 at 16:53:04 UTC, bitwise wrote:

If it were up to me, I would say:

'scope' should be removed as a storage class,
'ref' should be left as is, and
'in' should adopt the behavior of 'ref' post DIP25

  Bit


As Andrei said several times (recently here:  
http://forum.dlang.org/thread/scaufixfdyyrbmijg...@forum.dlang.org?page=3#post-mit5lp:241kpt:241:40digitalmars.com)  
auto ref will take care about rvalue references. I've only hoped that I  
could get a schedule for that or at least a confirmation. :)



If I understand Andrei's response correctly though, you will still have to
use 'auto ref' to get ref to bind to rvalues.. no?

So if I'm right about the above, and scope is out, why not just use 'in'  
instead of 'auto ref'?


Bit

Re: rvalue references

[Namespace via Digitalmars-d]

On Thursday, 14 May 2015 at 20:02:29 UTC, bitwise wrote:
On Thu, 14 May 2015 13:15:34 -0400, Namespace 
rswhi...@gmail.com wrote:

On Thursday, 14 May 2015 at 16:53:04 UTC, bitwise wrote:

If it were up to me, I would say:

'scope' should be removed as a storage class,
'ref' should be left as is, and
'in' should adopt the behavior of 'ref' post DIP25

 Bit


As Andrei said several times (recently here: 
http://forum.dlang.org/thread/scaufixfdyyrbmijg...@forum.dlang.org?page=3#post-mit5lp:241kpt:241:40digitalmars.com) 
auto ref will take care about rvalue references. I've only 
hoped that I could get a schedule for that or at least a 
confirmation. :)



If I understand Andrei's response correctly though, you will 
still have to

use 'auto ref' to get ref to bind to rvalues.. no?

Yes, that's what I said. :)
Besides, it would be horrible if rvalues could be bound to normal 
ref.


So if I'm right about the above, and scope is out, why not just 
use 'in' instead of 'auto ref'?


Bit


Don't ask me, I'm just a little light here.

Re: rvalue references

[bitwise via Digitalmars-d]

On Thu, 14 May 2015 16:12:36 -0400, Namespace rswhi...@gmail.com wrote:


Don't ask me, I'm just a little light here.


But I think it's because 'in' means 'const scope' and as you said, scope  
is out. ;)



Which means 'in' is available! ;)

  Bit

Re: rvalue references

[Namespace via Digitalmars-d]

On Thursday, 14 May 2015 at 00:12:05 UTC, bitwise wrote:
On Tue, 12 May 2015 08:54:15 -0400, Namespace 
rswhi...@gmail.com wrote:


As far as I know, the problem (or at least one of the biggest 
problems) for rvalue references was that they could escape. 
Since DIP25 is approved and already implemented this problem 
should be solved. Would it be possible to allow rvalues 
references now? I'm just curious what the mindfactory of D has 
in mind and what the scheme for that problem is.



Side note: DIP36 seems to be missing the table with the 
authors, status, etc.


  Bit


Huh, DIP36? DIP36 was rejected, but the authors (me and Dicebot) 
are below.


I've heard that DIP69 is [quote]almost dead in favor of 
DIP25[/quote]. That's why I'm a bit confused and would like to 
know the scheme, if any. :)

Re: rvalue references

[Namespace via Digitalmars-d]

No draft so far?

Re: rvalue references

[bitwise via Digitalmars-d]

On Tue, 12 May 2015 08:54:15 -0400, Namespace rswhi...@gmail.com wrote:

As far as I know, the problem (or at least one of the biggest problems)  
for rvalue references was that they could escape. Since DIP25 is  
approved and already implemented this problem should be solved. Would it  
be possible to allow rvalues references now? I'm just curious what the  
mindfactory of D has in mind and what the scheme for that problem is.



Wouldn't DIP69 have to be dealt with first before this happens?
There seems to be some overlap between DIP69 and DIP25.

For example, isn't scope unnecessary for this code after DIP25?

func(scope ref T t) {
return t; // already illegal per DIP25, no need for scope
}


  Bit

Re: rvalue references

[bitwise via Digitalmars-d]

On Tue, 12 May 2015 08:54:15 -0400, Namespace rswhi...@gmail.com wrote:

As far as I know, the problem (or at least one of the biggest problems)  
for rvalue references was that they could escape. Since DIP25 is  
approved and already implemented this problem should be solved. Would it  
be possible to allow rvalues references now? I'm just curious what the  
mindfactory of D has in mind and what the scheme for that problem is.



Side note: DIP36 seems to be missing the table with the authors, status,  
etc.


  Bit

Re: rvalue references

[w0rp]

I feel I should mention a detail I observed. I am working on a 
library which generates D code from C++ code with wrappers. I 
found an interesting situation when I realised that I need to do 
something like this.


void setPosition(ref const(QPoint) point) {
 ...
 // Static array.
 auto stack = smokeStack(point);

 cls.classFn(..., stack.ptr);
 ...
}

The above resulting in passing a pointer to a D struct which gets 
a reinterpret_cast on the C++ end and copied from that after 
that. Because I wish to also allow temporaries for this for 
obvious reasons, I must change my function signature to a 
template.


void setPosition()(auto ref const(QPointF) point);

So whatever happens with rvalue references, I would like to see 
the ability to take the address of an rvalue reference in a 
completely unsafe manner. (Or is const an lvalue? I have 
forgetten my Scott Meyers knowledge.) I would make this an error 
in @safe functions, but allowable for @system and @trusted.

Re: rvalue references

[Marco Leise]

Am Wed, 24 Apr 2013 13:41:05 +1000
schrieb Manu turkey...@gmail.com:

 
 
  On 24 April 2013 11:02, Diggory digg...@googlemail.com wrote:
 
  On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:
 
  2013/4/24 Manu turkey...@gmail.com
 
   The r-value being passed is assigned to a stack allocated temporary,
  which has a lifetime that is identical to any other local variable, ie,
  the
  lifetime of the function in which it appears.
  There, I defined it.
 
 
  Good definition. If add more,
  getting address of scope parameter would be disallowed, at least in
  @safe code, because it would be regarded as the escape of stack allocated
  temporary.
 
  Kenji Hara
 
 
  Why does the temporary need to exist any longer than the current
  statement? (the current lifetime of temporaries are the statement or
  expression). Surely any longer is just wasting stack space.
 
 
 scope ref a(scope ref int x) { return x; }
 void b(scope ref int x);
 
 b(a(10));
 
 The temp produced from '10' needs to last longer than a(), because it can
 be returned to b(). Ie, the temp needs the life of any normal local
 declared within that scope.
 
 Does D attempt to recycle stack
 space from previous short-lived locals within a single function?

That's a backend optimization, not something the
language needs to be built for specifically.

-- 
Marco

Re: Rvalue references - The resolution

[Steven Schveighoffer]

On Sun, 26 May 2013 18:56:58 -0400, Timothee Cour  
thelastmamm...@gmail.com wrote:


In fact it's also possible to know that these don't return a reference  
to

their parameter.

Watch out for this:
Struct S {double x;}
ref double foo(ref S a){return a.x;}


That case is covered by the proposal.  It incurs a runtime check (worst  
case, best case it simply doesn't compile).


-Steve

Re: Rvalue references - The resolution

[Martin Nowak]

On 05/05/2013 12:30 AM, Walter Bright wrote:
 On 5/4/2013 3:03 PM, deadalnix wrote:
 Where you miss the point, is that these annotations may be omitted
 (and they
 are most of the time). When nothing is specified, the lifetime of the
 returned
 reference is considered to be the union of the lifetime of parameters
 lifetime, which is what you want in 99% of cases.

 Note : We may also choose the lack of explicit lifetime means runtime
 check as
 proposed, instead of being an error.

 D omits the check when it can prove that the returned ref is not a ref
 to one of the parameters that is local.

ref int foo(ref int a, ref int b);

It's a very nice observation that calling foo with only non-local 
references means that the returned reference is non-local too.

In a way this works like inout but with a safe default so
that no annotation is needed.

In fact it's also possible to know that these don't return a reference 
to their parameter.


ref double foo(ref int a);

Struct S {}
ref double foo(ref S a);

It can become somewhat complicated to check though.

Anyhow I think using flow-analysis to omit runtime checks is a nice 
approach.

Re: Rvalue references - The resolution

[Timothee Cour]

 In fact it's also possible to know that these don't return a reference to
their parameter.

Watch out for this:
Struct S {double x;}
ref double foo(ref S a){return a.x;}

This sounds hacky. I've proposed a general solution here:
http://wiki.dlang.org/DIP38
either with user annotations of ref-return functions (scheme A) (just
distinguishing ref vs scope ref), or with compiler taking care of
annotations (scheme B).

On Sun, May 26, 2013 at 1:21 PM, Martin Nowak c...@dawg.eu wrote:

 On 05/05/2013 12:30 AM, Walter Bright wrote:
  On 5/4/2013 3:03 PM, deadalnix wrote:
  Where you miss the point, is that these annotations may be omitted
  (and they
  are most of the time). When nothing is specified, the lifetime of the
  returned
  reference is considered to be the union of the lifetime of parameters
  lifetime, which is what you want in 99% of cases.
 
  Note : We may also choose the lack of explicit lifetime means runtime
  check as
  proposed, instead of being an error.
 
  D omits the check when it can prove that the returned ref is not a ref
  to one of the parameters that is local.

 ref int foo(ref int a, ref int b);

 It's a very nice observation that calling foo with only non-local
 references means that the returned reference is non-local too.
 In a way this works like inout but with a safe default so
 that no annotation is needed.

 In fact it's also possible to know that these don't return a reference to
 their parameter.

 ref double foo(ref int a);

 Struct S {}
 ref double foo(ref S a);

 It can become somewhat complicated to check though.

 Anyhow I think using flow-analysis to omit runtime checks is a nice
 approach.

Re: Rvalue references - The resolution

[Rainer Schuetze]

On 04.05.2013 20:33, Walter Bright wrote:

Static Compiler Detection (in @safe mode):

1. Do not allow taking the address of a local variable, unless doing a
safe type 'paint' operation.


I'm not exactly sure what a safe type paint operation does, and 
whether the following has already been considered, but I just like to be 
assured it has:


Taking a slice of a stack allocated fixed-size array also includes 
taking its address, so it is also forbidden? This might disallow any 
range based algorithms on the static array.

Re: Rvalue references - The resolution

[Maxim Fomin]

On Saturday, 4 May 2013 at 18:33:04 UTC, Walter Bright wrote:
Thanks to the many recent threads on this, and the dips on it, 
everyone was pretty much up to speed and ready to find a 
resolution. This resolution only deals with the memory safety 
issue.


...

What if an argument is captured by a delegate?

import std.stdio;

alias long[100] T;

T delegate() dg;

//ref T foo(ref T i) @safe
void foo(ref T i) @safe
{
   dg = { return i; } ;
   //return i;
}

//ref T bar()
void bar() @safe
{
   T i = 1;
   //return foo(i);
   foo(i);
}

void rewrite_stack() @safe
{
   T tmp = -1;
}

void main()
{
   //T i = bar();
   bar();
   rewrite_stack();
   writeln(dg());
}

I believe that even taking your runtime solution into account 
there is still flaw in the code which is caused by capturing 
reference (pointer) to passed object. Since definition of 'foo' 
may be unavailable, compiler cannot know during issuing call to 
'bar' whether to allocate argument on the stack or in the heap.


By the way, lazy+delegate is broken.

auto foo(lazy int i) @safe
{
   return { return i; } ;
}

auto bar() @safe
{
   int i = 4;
   return foo(i);
}

void baz() @safe
{
   int[1] arr = 2;
}

void main() @safe
{
   auto x = bar();
   baz();
   assert(x() is 2); // stack value hijacktion
}

First example: http://dpaste.dzfl.pl/4c84a5e4
Second example: http://dpaste.dzfl.pl/9399adc6

Re: Rvalue references - The resolution

[Jonathan M Davis]

On Thursday, May 09, 2013 21:30:00 Rainer Schuetze wrote:
 On 04.05.2013 20:33, Walter Bright wrote:
  Static Compiler Detection (in @safe mode):
  
  1. Do not allow taking the address of a local variable, unless doing a
  safe type 'paint' operation.
 
 I'm not exactly sure what a safe type paint operation does, and
 whether the following has already been considered, but I just like to be
 assured it has:
 
 Taking a slice of a stack allocated fixed-size array also includes
 taking its address, so it is also forbidden? This might disallow any
 range based algorithms on the static array.

Asuming that taking the slice of a static array is treated like ref (as @safe) 
rather than like taking the address of a local variable is (as @system), then 
we'll have to add similar runtime checks for arrays, and that would be way, 
way worse given that without purity, they could be assigned to a global 
dynamic array (or could be assigned to a member variable in a return value 
even with pure functions). It's fairly clean for ref simply because ref is a 
storage class and not a type constructor. Array slices on the other hand could 
escape all over the place.

I'm inclined to believe that taking a slice of a static array should be 
considered @system just like taking the address of a local variable is 
considered @system. If I could, I'd even disallow the implicit slicing of 
static arrays when passing them to functions taking dynamic arrays, but I 
question that Walter would go that far. But I don't know what we can do other 
than making slicing static arrays @system given how difficult it would be to 
have runtime checks catch that.

I'd brought this issue up in the past but had not remembered it during the 
recent discussions on ref safety. Good catch. We don't want any holes like 
this to persist.

- Jonathan M Davis

Re: Rvalue references - The resolution

[Manu]

On 5 May 2013 10:37, Andrei Alexandrescu seewebsiteforem...@erdani.orgwrote:

 On 5/4/13 7:31 PM, Walter Bright wrote:

 On 5/4/2013 3:51 PM, w0rp wrote:

 Does all of this also mean that a
 function with a ref parameter will automagically work with r-values?


 Yes.


 This is new to me. My understanding is that the discussed design addresses
 safety, and leaves the rvalue discussion for a future iteration.


I was left under the same impression that Walter also seems to be under.

Re: Rvalue references - The resolution

[Jonathan M Davis]

On Monday, May 06, 2013 10:16:57 Steven Schveighoffer wrote:
 Could be the time change, haven't rebooted my Mac since flying back. My
 clock is correct, but Opera may be confused.

Oh, the wonders of dealing with time... :)

- Jonathan M Davis

Re: Rvalue references - The resolution

[Zach the Mystic]

On Saturday, 4 May 2013 at 18:33:04 UTC, Walter Bright wrote:

Static Compiler Detection (in @safe mode):

1. Do not allow taking the address of a local variable, unless 
doing a safe type 'paint' operation.


2. In some cases, such as nested, private, and template 
functions, the source is always available so the compiler can 
error on those. Because of the .di file problem, doing this 
with auto return functions is problematic.


3. Issue error on return statements where the expression may 
contain a ref to a local that is going out of scope, taking 
into account the observations.


Runtime Detection

There are still a few cases that the compiler cannot statically 
detect. For these a runtime check is inserted, which compares 
the returned ref pointer to see if it lies within the stack 
frame of the exiting function, and if it does, halts the 
program. The cost will be a couple of CMP instructions and an 
LEA. These checks would be omitted if the -noboundscheck 
compiler switch was provided.


This is a brilliant solution. I'm glad my DIP seems to have 
helped pivot the design process into this superior conclusion, 
which uses something, i.e. runtime checking, I simply didn't 
think of. I guess I didn't realize that the stack has bounds, 
so to say.


I suppose that underneath the hood the compiler will still track 
the state of the return value using something like a 'scope' bit. 
It's just that the user code doesn't need to see this bit, which 
is probably how it should be. And it's great to realize that a 
suitable safety framework - -noboundscheck - has been found which 
already exists to encompass the checking.


I think the main data still to be researched is the slowdown with 
both compile and run times with this checking implemented - not 
that I see how to avoid it, but it's better to know than not to 
know, right?

Re: Rvalue references - The resolution

[Zach the Mystic]

On Sunday, 5 May 2013 at 02:36:45 UTC, Jonathan M Davis wrote:
As it is, we arguably didn't choose the best defaults with the 
attributes that
we have (e.g. @system is the default instead of @safe, and 
impure is the
default instead of pure). The result is that we have to use a 
lot of
annotations if we want to properly take advantage of the 
various language
features, whereas ideally, having to use annotations for stuff 
like @safety or
purity would be the exception. Don was complaining that one 
reason that moving
to D2 at Sociomantic looks unappealing in spite of the benefits 
is the fact
that they're going to have to add so many extra annotations to 
their code.


In the thread which appeared on github someone suggested 
'@infer', which I altered to '@auto', which gets all the 
attributes automatically, and creates the '.di' with the full 
attributes (which might actually be problematic if they change 
too often and force compilation too many times). I'm starting to 
think it might actually be quite valuable to have this annotation 
available to the programmer. What do you think?

Re: Rvalue references - The resolution

[Steven Schveighoffer]

On Sat, 04 May 2013 19:30:21 -0700, Jonathan M Davis jmdavisp...@gmx.com  
wrote:


However, if we had an attribute which explicitly designated that a  
function

accepted both rvalues and lvalues (which is what auto ref was originally
supposed to do as Andrei proposed it), then if you saw

auto foo(ref int i);
auto bar(auto ref int i);

then you could be reasonably certain that foo was intending to alter its
arguments and bar was not.


The counter argument:

foo(makeRvalue()); // error:  cannot pass rvalues to ref

// programmer: WTF?  This is stupid, but ok:

auto x = makeRvalue();
foo(x);

In other words, explicit nops aren't any better than implicit nops.  Even  
if we *require* the user to be explicit (and it's not at all clear from a  
code-review perspective that the auto x line is to circumvent the  
requirements), the fact that this is trivially circumvented makes it a  
useless feature.  It's like having const you can cast away.


I think the larger issue with binding rvalues to refs is this:

int foo(int i);
int foo(ref int i);

what does foo(1) bind to?  It MUST bind to the non-ref, or there is no  
point for it.


If this can be solved, binding rvalues to refs is fine.

-Steve

Re: Rvalue references - The resolution

[Andrei Alexandrescu]

On 5/6/13 12:10 PM, Steven Schveighoffer wrote:

The counter argument:

foo(makeRvalue()); // error: cannot pass rvalues to ref

// programmer: WTF? This is stupid, but ok:

auto x = makeRvalue();
foo(x);

In other words, explicit nops aren't any better than implicit nops. Even
if we *require* the user to be explicit (and it's not at all clear from
a code-review perspective that the auto x line is to circumvent the
requirements), the fact that this is trivially circumvented makes it a
useless feature. It's like having const you can cast away.

I think the larger issue with binding rvalues to refs is this:

int foo(int i);
int foo(ref int i);

what does foo(1) bind to? It MUST bind to the non-ref, or there is no
point for it.

If this can be solved, binding rvalues to refs is fine.


I think we can technically make the overloading work while also allowing 
binding rvalues to ref. But that wouldn't help any. Consider:


ref int min(ref int a, ref int b) { return b  a ? b : a; }
...
int x;
fun(min(x, 100));

Here the result of min may be bound to an lvalue or an rvalue depending 
on a condition. In the latter case, combined with D's propensity to 
destroy temporaries too early (immediately after function calls), the 
behavior is silently undefined; the code may pass unittests.


This is a known issue in C++. Allowing loose binding of rvalues to ref 
not only inherits C++'s mistake, but also adds a fresh one.



Andrei

Re: Rvalue references - The resolution

[Steven Schveighoffer]

On Mon, 06 May 2013 06:43:38 -0700, Andrei Alexandrescu  
seewebsiteforem...@erdani.org wrote:


I think we can technically make the overloading work while also allowing  
binding rvalues to ref. But that wouldn't help any. Consider:


ref int min(ref int a, ref int b) { return b  a ? b : a; }
...
int x;
fun(min(x, 100));

Here the result of min may be bound to an lvalue or an rvalue depending  
on a condition. In the latter case, combined with D's propensity to  
destroy temporaries too early (immediately after function calls), the  
behavior is silently undefined; the code may pass unittests.


Wouldn't the new runtime check fix this?

This is a known issue in C++. Allowing loose binding of rvalues to ref  
not only inherits C++'s mistake, but also adds a fresh one.


I thought C++ would handle this kind of code.  I remember being able to  
use references to rvalues in ways that were unintuitive, but not undefined.


-Steve

Re: Rvalue references - The resolution

[Andrei Alexandrescu]

On 5/6/13 12:48 PM, Steven Schveighoffer wrote:

On Mon, 06 May 2013 06:43:38 -0700, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:


I think we can technically make the overloading work while also
allowing binding rvalues to ref. But that wouldn't help any. Consider:

ref int min(ref int a, ref int b) { return b  a ? b : a; }
...
int x;
fun(min(x, 100));

Here the result of min may be bound to an lvalue or an rvalue
depending on a condition. In the latter case, combined with D's
propensity to destroy temporaries too early (immediately after
function calls), the behavior is silently undefined; the code may pass
unittests.


Wouldn't the new runtime check fix this?


Depends how you define fix. It would be a possibly rare bounds check 
violation on completely innocuous code.



This is a known issue in C++. Allowing loose binding of rvalues to ref
not only inherits C++'s mistake, but also adds a fresh one.


I thought C++ would handle this kind of code. I remember being able to
use references to rvalues in ways that were unintuitive, but not undefined.


template class T const T min(const T a, const T b) {
return b  a ? b : a;
}
...
int x = ...;
auto  weird = min(x, 100);

Have a nice day :o).


Andrei