subject:"DIP 36\: Rvalue References"


On Tuesday, 9 April 2013 at 17:06:47 UTC, Namespace wrote:

http://wiki.dlang.org/DIP36


// Does not pretend to be @safe any more
void test2(ref A a) {
}

I believe 'ref' still *can* be @safe. The unsafe action is really 
when you assign it to a global pointer.


void test2(ref A a) {
  static A* p;
  p = &a; // <-- This is really the only unsafe thing
}

I think for convenience the function must only be considered 
unsafe if it does the above. Even returning the 'ref' is @safe so 
long as the *caller* keeps track of what it passes in, according 
to DIP25.


"Following limitation apply to scope ref function parameters 
(including in ref):
...3. Parameter can only be used as an argument for other 
function if it also accepts scope ref, no implicit casting away."


The DIP fails to mention that 'scope' by itself implies 'ref', 
and (presumably) can be reserved for cases where the caller 
specifically does not want to allow rvalue temps. 'scope' implies 
'ref' because value types are inherently @safe and need no 
attributes. I'm not sure what's so bad about allowing 'scope ref' 
arguments to be passed to 'scope' parameters also. It passes the 
ref it receives, but since it's still 'scope', it's not going 
anywhere.


Also, given that 'scope' implies 'ref', there may be significant 
advantage in simply allowing *all* scope parameters to accept 
rvalue temps. It would be helpful to have a use case where 
'scope' was clearly desired but rvalue temps clearly *not* 
desired. In most cases, it seems, anything marked 'scope' could 
easily accept an rvalue temp no problem.


The 'scope ref' syntax does have a slight conflict with DIP25 and 
my suggested addition DIP35, in that if other means are found to 
making returning 'ref' safe, 'scope ref' as a syntax means that 
no rvalue temporary can ever be returned. Since this may be 
perfectly acceptable from a 'bad practices' point of view, 'scope 
ref' may still be a good syntax. But if it were seen as okay to 
return even a an rvalue temporary, since it is made safe by other 
means, 'scope ref' would either not be usable for this or would 
actually be (confusingly) returnable.

Re: DIP 36: Rvalue References

2013-04-09 Thread kenji hara

2013/4/10 Zach the Mystic 

> On Tuesday, 9 April 2013 at 17:06:47 UTC, Namespace wrote:
>
>> http://wiki.dlang.org/DIP36
>>
>
> // Does not pretend to be @safe any more
> void test2(ref A a) {
> }
>
> I believe 'ref' still *can* be @safe. The unsafe action is really when you
> assign it to a global pointer.
>
> void test2(ref A a) {
>   static A* p;
>   p = &a; // <-- This is really the only unsafe thing
> }
>
> I think for convenience the function must only be considered unsafe if it
> does the above. Even returning the 'ref' is @safe so long as the *caller*
> keeps track of what it passes in, according to DIP25.
>
> "Following limitation apply to scope ref function parameters (including in
> ref):
> ...3. Parameter can only be used as an argument for other function if it
> also accepts scope ref, no implicit casting away."
>
> The DIP fails to mention that 'scope' by itself implies 'ref', and
> (presumably) can be reserved for cases where the caller specifically does
> not want to allow rvalue temps. 'scope' implies 'ref' because value types
> are inherently @safe and need no attributes. I'm not sure what's so bad
> about allowing 'scope ref' arguments to be passed to 'scope' parameters
> also. It passes the ref it receives, but since it's still 'scope', it's not
> going anywhere.
>
> Also, given that 'scope' implies 'ref', there may be significant advantage
> in simply allowing *all* scope parameters to accept rvalue temps. It would
> be helpful to have a use case where 'scope' was clearly desired but rvalue
> temps clearly *not* desired. In most cases, it seems, anything marked
> 'scope' could easily accept an rvalue temp no problem.
>
> The 'scope ref' syntax does have a slight conflict with DIP25 and my
> suggested addition DIP35, in that if other means are found to making
> returning 'ref' safe, 'scope ref' as a syntax means that no rvalue
> temporary can ever be returned. Since this may be perfectly acceptable from
> a 'bad practices' point of view, 'scope ref' may still be a good syntax.
> But if it were seen as okay to return even a an rvalue temporary, since it
> is made safe by other means, 'scope ref' would either not be usable for
> this or would actually be (confusingly) returnable.
>

I think having both 'ref' and 'scope ref' is still reasonable, because:

1. According to DIP25, escaping ref parameter is always safe. But, for
rvalue reference, it allows meaningless code.

// Even if this is allowed, it is safe. BUT will introduce bad practice.
ref int foo(scope ref int x) { return x; }

// safe
ref int bar(ref int x) { return x; }

void test() {
int x;
foo(x) = 10;// may useful but...
foo(1) = 10;// completely useless!

bar(x) = 10;// may useful, and
bar(1) = 10;// disallowed.
}

2. Certainly 'scope' by itself _can_ imply 'ref'. BUT, currently a function
parameter without 'ref' storage class implies having value semantics (==
copy).
If we change the 'scope' meaning to 'implicit ref', it will break the
consistency.

And, 'scope' is alredy used for delegate parameter.

int opApply(scope int delegate(ref int) dg);

Such a semantic modification will silently change the meaning of existing
code. It's not good to me.

Kenji Hara

Re: DIP 36: Rvalue References


On Wednesday, 10 April 2013 at 02:41:10 UTC, kenji hara wrote:
I think having both 'ref' and 'scope ref' is still reasonable, 
because:


1. According to DIP25, escaping ref parameter is always safe.  
But, for

rvalue reference, it allows meaningless code.

// Even if this is allowed, it is safe. BUT will introduce bad 
practice.

ref int foo(scope ref int x) { return x; }

// safe
ref int bar(ref int x) { return x; }

void test() {
int x;
foo(x) = 10;// may useful but...
foo(1) = 10;// completely useless!

bar(x) = 10;// may useful, and
bar(1) = 10;// disallowed.
}


'foo(1) = 10' is indeed completely useless, but I don't think all 
uses are completely useless. But I will concede that allowing 
things like 'foo(1)' may be useless *enough* to disallow it 
without anyone feeling bad. I have made DIP35, 
http://wiki.dlang.org/DIP35 , which utilizes 'scope' to actually 
make DIP25 more flexible, which I think is probably good. The 
example I used was the 'copy' function. 'foo' above, rewritten as 
the copy function would be:


ref int foo(scope ref int x) {
  int* y = new int;
  *y = x;
  return y;
}

DIP25 may *need* to use 'scope' in this way to know that foo() 
will always give a fresh value - so it can treat it like a global 
safely. Thus 'foo(1)' will return a fresh heap allocated copy of 
"1".


ref int test() {
  return foo(1); // allowed
  return bar(1); // disallowed!
}

Without the 'scope' parameter, there would be no way the compiler 
could know. Therefore, it is not completely useless.


2. Certainly 'scope' by itself _can_ imply 'ref'. BUT,  
currently a function
parameter without 'ref' storage class implies having value 
semantics (== copy).


This is slightly wrong, I suspect. 'out' parameters are 
implicitly references.


If we change the 'scope' meaning to 'implicit ref', it will 
break the

consistency.

And, 'scope' is already used for delegate parameter.

int opApply(scope int delegate(ref int) dg);

Such a semantic modification will silently change the meaning 
of existing code. It's not good to me.


Generally, I agree. My argument is for a small convenience of not 
having to type 'ref' when not necessary. It would mean that when 
the type was implicitly a reference, such as a delegate or 
object, 'ref' was *not* implied, whereas it *was* implied with a 
value type. This may or may not wreak havoc on the generic type 
system. I thought there might be an off-chance that it would 
actually be a *convenience* for the type system, since it would 
get the right level of referencing for the type involved.


I believe that the 'scope' keyword may be a necessary addition to 
DIP25 to make 'ref' fully safe and flexible (DIP35). If so, then 
there may be a conflict with using it only to mean rvalue 
temporaries. They are two separate features. I believe that if 
they are to be used together, it will be because it was decided 
that whenever you want to stop a reference from escaping, you 
*always* want to allow rvalue temporaries, and whenever you want 
to allow rvalue temporaries, you *never* want them to be 
returned. Right now, returning an rvalue temporary might seem 
dangerous, but sealed references *are* part of the plan for D, 
and there may be no need to see them as dangerous.

Re: DIP 36: Rvalue References

On Wednesday, 10 April 2013 at 04:46:43 UTC, Zach the Mystic 
wrote:

ref int bar(ref int x) { return x; }

ref int foo(scope ref int x) {
  int* y = new int;
  *y = x;
  return y;
}

ref int test() {
  return foo(1); // allowed
  return bar(1); // disallowed!
}


When said 'bar(1)' was disallowed, I meant that it was considered 
local and not disallowed because it doesn't accept rvalue temps. 
Fix:


ref int test() {
  int y;
  return foo(y); // allowed
  return bar(y); // disallowed!
}

Re: DIP 36: Rvalue References

YES PLEASE!!
I've been saying precisely this for years, I'm glad someone took the time
to write it down all nice and proposal-like :)


On 10 April 2013 03:06, Namespace  wrote:

> http://wiki.dlang.org/DIP36
>

Re: DIP 36: Rvalue References

On 10 April 2013 12:40, kenji hara  wrote:

> 2013/4/10 Zach the Mystic 
>
>> On Tuesday, 9 April 2013 at 17:06:47 UTC, Namespace wrote:
>>
>>> http://wiki.dlang.org/DIP36
>>>
>>
>> // Does not pretend to be @safe any more
>> void test2(ref A a) {
>> }
>>
>> I believe 'ref' still *can* be @safe. The unsafe action is really when
>> you assign it to a global pointer.
>>
>> void test2(ref A a) {
>>   static A* p;
>>   p = &a; // <-- This is really the only unsafe thing
>> }
>>
>> I think for convenience the function must only be considered unsafe if it
>> does the above. Even returning the 'ref' is @safe so long as the *caller*
>> keeps track of what it passes in, according to DIP25.
>>
>> "Following limitation apply to scope ref function parameters (including
>> in ref):
>> ...3. Parameter can only be used as an argument for other function if it
>> also accepts scope ref, no implicit casting away."
>>
>> The DIP fails to mention that 'scope' by itself implies 'ref', and
>> (presumably) can be reserved for cases where the caller specifically does
>> not want to allow rvalue temps. 'scope' implies 'ref' because value types
>> are inherently @safe and need no attributes. I'm not sure what's so bad
>> about allowing 'scope ref' arguments to be passed to 'scope' parameters
>> also. It passes the ref it receives, but since it's still 'scope', it's not
>> going anywhere.
>>
>> Also, given that 'scope' implies 'ref', there may be significant
>> advantage in simply allowing *all* scope parameters to accept rvalue temps.
>> It would be helpful to have a use case where 'scope' was clearly desired
>> but rvalue temps clearly *not* desired. In most cases, it seems, anything
>> marked 'scope' could easily accept an rvalue temp no problem.
>>
>> The 'scope ref' syntax does have a slight conflict with DIP25 and my
>> suggested addition DIP35, in that if other means are found to making
>> returning 'ref' safe, 'scope ref' as a syntax means that no rvalue
>> temporary can ever be returned. Since this may be perfectly acceptable from
>> a 'bad practices' point of view, 'scope ref' may still be a good syntax.
>> But if it were seen as okay to return even a an rvalue temporary, since it
>> is made safe by other means, 'scope ref' would either not be usable for
>> this or would actually be (confusingly) returnable.
>>
>
> I think having both 'ref' and 'scope ref' is still reasonable, because:
>
> 1. According to DIP25, escaping ref parameter is always safe. But, for
> rvalue reference, it allows meaningless code.
>
> // Even if this is allowed, it is safe. BUT will introduce bad practice.
> ref int foo(scope ref int x) { return x; }
>
> // safe
> ref int bar(ref int x) { return x; }
>
> void test() {
> int x;
> foo(x) = 10;// may useful but...
> foo(1) = 10;// completely useless!
>
> bar(x) = 10;// may useful, and
> bar(1) = 10;// disallowed.
> }
>
> 2. Certainly 'scope' by itself _can_ imply 'ref'. BUT, currently a
> function parameter without 'ref' storage class implies having value
> semantics (== copy).
> If we change the 'scope' meaning to 'implicit ref', it will break the
> consistency.
>

Why are you suggesting changing scope to imply ref? This seems wrong. scope
and ref are separate, should remain that way.
Arguments would be scope ref, or in ref, thus allowing rvalues. Without
ref, I would imagine that the meaning is such that, even though the
argument is a copy, it, or members thereof are not allowed to escape.

struct X { int *p; }
int* global;

void func(scope X x) // note: is not ref
{
  global = x.p; // <- error! scope prevents any part of x escaping, even
though it's a copy.
}

This way, scope has meaning with or without ref.


And, 'scope' is alredy used for delegate parameter.
>
> int opApply(scope int delegate(ref int) dg);
>
> Such a semantic modification will silently change the meaning of existing
> code. It's not good to me.
>

What does scope mean in this case? Is this a conflict in some way?

Re: DIP 36: Rvalue References

2013-04-09 Thread kenji hara

2013/4/10 Manu 
>
> Why are you suggesting changing scope to imply ref? This seems wrong.
> scope and ref are separate, should remain that way.
>

I say that 'scope' itself should not imply 'ref'. It is *possible* but
would break existing code meaning, so it should not be applied.


> Arguments would be scope ref, or in ref, thus allowing rvalues. Without
> ref, I would imagine that the meaning is such that, even though the
> argument is a copy, it, or members thereof are not allowed to escape.
>
> struct X { int *p; }
> int* global;
>
> void func(scope X x) // note: is not ref
> {
>   global = x.p; // <- error! scope prevents any part of x escaping, even
> though it's a copy.
> }
>
> This way, scope has meaning with or without ref.
>

That is an enhancement. Similar things is now in my mind, but it is not yet
output.


> And, 'scope' is alredy used for delegate parameter.
>>
>> int opApply(scope int delegate(ref int) dg);
>>
>> Such a semantic modification will silently change the meaning of existing
>> code. It's not good to me.
>>
>
> What does scope mean in this case? Is this a conflict in some way?
>

Compiler guarantees that given delegate to the scope parameter never make a
closure.
It would avoid heap allocation.

Kenji Hara

Re: DIP 36: Rvalue References

On 10 April 2013 15:30, kenji hara  wrote:

> 2013/4/10 Manu 
>>
>> Why are you suggesting changing scope to imply ref? This seems wrong.
>> scope and ref are separate, should remain that way.
>>
>
> I say that 'scope' itself should not imply 'ref'. It is *possible* but
> would break existing code meaning, so it should not be applied.
>

Right. Absolutely agree then! :)



> Arguments would be scope ref, or in ref, thus allowing rvalues. Without
>> ref, I would imagine that the meaning is such that, even though the
>> argument is a copy, it, or members thereof are not allowed to escape.
>>
>> struct X { int *p; }
>> int* global;
>>
>> void func(scope X x) // note: is not ref
>> {
>>   global = x.p; // <- error! scope prevents any part of x escaping, even
>> though it's a copy.
>> }
>>
>> This way, scope has meaning with or without ref.
>>
>
> That is an enhancement. Similar things is now in my mind, but it is not
> yet output.
>

I think this is a tool that will greatly enhance working with @safe in the
future. I look forward to it.


 And, 'scope' is alredy used for delegate parameter.
>>>
>>> int opApply(scope int delegate(ref int) dg);
>>>
>>> Such a semantic modification will silently change the meaning of
>>> existing code. It's not good to me.
>>>
>>
>> What does scope mean in this case? Is this a conflict in some way?
>>
>
> Compiler guarantees that given delegate to the scope parameter never make
> a closure.
> It would avoid heap allocation.
>

Do you have any ideas/solutions that still fit this proposal?
I tend to think the value of this proposal is of extreme significance to D.
It is, without doubt, the single biggest complaint I've heard by virtually
every programmer I've introduced D to, and the future implications for
@safe improvements are valuable too.

Re: DIP 36: Rvalue References


On Wednesday, 10 April 2013 at 05:21:40 UTC, Manu wrote:
Why are you suggesting changing scope to imply ref? This seems 
wrong. scope

and ref are separate, should remain that way.


To be clear, I suggested it, not Kenji. The reason scope could 
imply ref is that no value type is ever unsafe. If you copy the 
value you're passing, there's no way the reference could escape 
the scope, because there's no reference! A delegate has an 
implicit pointer and is inherently a reference type, which is why 
it can work with 'scope'. In all likelihood 'scope' implying 
'ref' would needlessly complicate the type system, in exchange 
for the convenience of only having to type either 'scope' or 
'ref' depending on what you wanted. But I suggested it because I 
at least wanted it to be discussed.

Re: DIP 36: Rvalue References

On 10 April 2013 16:11, Zach the Mystic  wrote:

> On Wednesday, 10 April 2013 at 05:21:40 UTC, Manu wrote:
>
>> Why are you suggesting changing scope to imply ref? This seems wrong.
>> scope
>> and ref are separate, should remain that way.
>>
>
> To be clear, I suggested it, not Kenji. The reason scope could imply ref
> is that no value type is ever unsafe.


It is though, in the case I demonstrated. A value type can aggregate a
reference type, and by-val scope would prohibit any part of the copy from
escaping in turn.
This is tricky, but I think it would be an important enhancement for
non-ref scope variables and @safe-ty.
It might be impractical, but it's something to think about anyway...

If you copy the value you're passing, there's no way the reference could
> escape the scope, because there's no reference! A delegate has an implicit
> pointer and is inherently a reference type, which is why it can work with
> 'scope'. In all likelihood 'scope' implying 'ref' would needlessly
> complicate the type system, in exchange for the convenience of only having
> to type either 'scope' or 'ref' depending on what you wanted. But I
> suggested it because I at least wanted it to be discussed.
>

Fair enough.

Re: DIP 36: Rvalue References

2013-04-10 Thread Zach the Mystic


On Wednesday, 10 April 2013 at 05:21:40 UTC, Manu wrote:

struct X { int *p; }
int* global;

void func(scope X x) // note: is not ref
{
  global = x.p; // <- error! scope prevents any part of x 
escaping, even

though it's a copy.
}

This way, scope has meaning with or without ref.


I think this is a subtle point, but I don't think it's correct. I 
think there's a difference between 'x.p' above, and what '&x.p' 
or '&x' would give. I think 'scope' should guarantee that you 
won't assign the *address* of the passed-in parameter:


void func(scope X x, ref X y) {
  static X* xx = &x; // Error
  static int** xpp = &x.p; // Error
  xx = &y; // unsafe, but pass
  xpp = &y.p; // same
}

I don't think it should guarantee that you won't copy something 
that is *already* a pointer. Copying the pointer may be 
dangerous, but I don't think 'scope' will be very useful if it's 
supposed to track the internal pointers of aggregate types like 
that. I think somebody else has to make sure that pointer is 
safe. I could be wrong.

Re: DIP 36: Rvalue References


On Wednesday, 10 April 2013 at 00:41:54 UTC, kenji hara wrote:
Why test32 does not accept rvalue like test1? I think that 
"scope ref"
should always behave same for rvalue argument - create 
temporary and take
its address to "scope ref" parameter. Consistent behavior with 
any types is

important for the generic programming.


That is me to blame. I have convinced Namespace that this is the 
way to go in private conversation and hope to do the same with 
you ;)


Consistent behavior is important but it does not change the fact 
that there two types of rvalues in the language - raw literals 
(42, A.init) and constructed temporaries (A()). Accepting rvalues 
of the first type in as mutable references may be consistent from 
the point of view of syntax by _extremely_ confusing 
semantics-wise:


void foo(scope ref int x)
{
x = 32;
}

void main()
{
foo(42); // erm, 32 is the new 42 or ref is not actually ref?
}

Beauty of "in ref" is that in that case user can't use it in any 
way that may disclose the fact the temporary variable for 
non-adressable entities is created. For mutable scope ref it is 
not the case and we must leak implementation details, which is 
never good.


Another concern is that if someone tries to pass "42" as a 
mutable ref, most likely he does not really wants it to be 
mutable and "in ref" is a better match.


This may not be consistent from the point of view of generic 
code, but it is now consistent from the point of view of 
programmer : A(...) construct always create temporaries, raw 
value literals never do (well, they will for "in ref" but you 
can't observe it). I think this is much more important.


One extra thing to note is that test32 may actually work if it 
instantiates T as "const int" for 42 (making it "in ref" 
essentially), but I don't know if D rules allow it.

Re: DIP 36: Rvalue References

On Wednesday, 10 April 2013 at 01:40:49 UTC, Zach the Mystic 
wrote:
I think for convenience the function must only be considered 
unsafe if it does the above. Even returning the 'ref' is @safe 
so long as the *caller* keeps track of what it passes in, 
according to DIP25.


Safety of "ref" is not problem that this DIP tries to solve. It 
should be solved with DIP25. Now ref is not safe (despite the 
fact it pretends to) and it still won't be after DIP36 approval, 
not until issue is addressed in DIP25.


No need to mix it.

Re: DIP 36: Rvalue References


On Wednesday, 10 April 2013 at 05:01:12 UTC, Manu wrote:

YES PLEASE!!
I've been saying precisely this for years, I'm glad someone 
took the time

to write it down all nice and proposal-like :)


When I initially volunteered to help formalize it I had no strong 
feelings about this proposal but after taking care of all details 
I really like how it naturally fits into existing D type system, 
with no ugly hacks or weird behavior changes.

Re: DIP 36: Rvalue References

2013-04-10 Thread Zach the Mystic


On Wednesday, 10 April 2013 at 07:46:41 UTC, Dicebot wrote:
On Wednesday, 10 April 2013 at 01:40:49 UTC, Zach the Mystic 
wrote:
I think for convenience the function must only be considered 
unsafe if it does the above. Even returning the 'ref' is @safe 
so long as the *caller* keeps track of what it passes in, 
according to DIP25.


Safety of "ref" is not problem that this DIP tries to solve. It 
should be solved with DIP25. Now ref is not safe (despite the 
fact it pretends to) and it still won't be after DIP36 
approval, not until issue is addressed in DIP25.


No need to mix it.


My proposed DIP35 uses 'scope' to enhance DIP25. I don't believe 
DIP25 is really complete. 'scope' would help it a lot, but that 
would give more meanings to 'scope' which are potentially in 
conflict with it just meaning rvalue temp.

Re: DIP 36: Rvalue References

On Wednesday, 10 April 2013 at 09:26:38 UTC, Zach the Mystic 
wrote:

On Wednesday, 10 April 2013 at 07:46:41 UTC, Dicebot wrote:
On Wednesday, 10 April 2013 at 01:40:49 UTC, Zach the Mystic 
wrote:
I think for convenience the function must only be considered 
unsafe if it does the above. Even returning the 'ref' is 
@safe so long as the *caller* keeps track of what it passes 
in, according to DIP25.


Safety of "ref" is not problem that this DIP tries to solve. 
It should be solved with DIP25. Now ref is not safe (despite 
the fact it pretends to) and it still won't be after DIP36 
approval, not until issue is addressed in DIP25.


No need to mix it.


My proposed DIP35 uses 'scope' to enhance DIP25. I don't 
believe DIP25 is really complete. 'scope' would help it a lot, 
but that would give more meanings to 'scope' which are 
potentially in conflict with it just meaning rvalue temp.


And how it is relevant to DIP36? It solves specific issue. It 
provides some hints how other DIP may use new opportunities. It 
is up to DIP25 and DIP35 to be built on top if this gets accepted 
first. Or other way around.


DIPs are built in context for current language implementation, 
not some potential other DIPs.

Re: DIP 36: Rvalue References

2013-04-10 Thread deadalnix


On Tuesday, 9 April 2013 at 17:06:47 UTC, Namespace wrote:

http://wiki.dlang.org/DIP36


I see no point in adding that much complexity for something that 
can mostly be automated.

Re: DIP 36: Rvalue References


On Wednesday, 10 April 2013 at 11:36:22 UTC, deadalnix wrote:

On Tuesday, 9 April 2013 at 17:06:47 UTC, Namespace wrote:

http://wiki.dlang.org/DIP36


I see no point in adding that much complexity for something 
that can mostly be automated.


Can you explain this a bit more extensively, probably with some 
sort of counter-proposal? I can't see where complexity comes 
from, this DIP introduces literally zero special cases.

Re: DIP 36: Rvalue References

2013-04-10 Thread Manu

On 10 April 2013 21:36, deadalnix  wrote:

> On Tuesday, 9 April 2013 at 17:06:47 UTC, Namespace wrote:
>
>> http://wiki.dlang.org/DIP36
>>
>
> I see no point in adding that much complexity for something that can
> mostly be automated.
>

Where's the complexity? I find it intuitive and super useful.
I personally see this DIP as just fleshing out the functionality of scope
that I imagined/assumed from the moment I learned about the scope keyword.
I was actually quite surprised to learn it wasn't already like this :)

Re: DIP 36: Rvalue References

2013-04-10 Thread Namespace


On Wednesday, 10 April 2013 at 11:37:43 UTC, Dicebot wrote:

On Wednesday, 10 April 2013 at 11:36:22 UTC, deadalnix wrote:

On Tuesday, 9 April 2013 at 17:06:47 UTC, Namespace wrote:

http://wiki.dlang.org/DIP36


I see no point in adding that much complexity for something 
that can mostly be automated.


Can you explain this a bit more extensively, probably with some 
sort of counter-proposal? I can't see where complexity comes 
from, this DIP introduces literally zero special cases.


I think he means that the Compiler could/should decide, if 
something should passed by ref or by value. But I think that 
would be far more complex than scope ref/in ref.

Re: DIP 36: Rvalue References

2013-04-10 Thread Manu

On 10 April 2013 22:27, Namespace  wrote:

> On Wednesday, 10 April 2013 at 11:37:43 UTC, Dicebot wrote:
>
>> On Wednesday, 10 April 2013 at 11:36:22 UTC, deadalnix wrote:
>>
>>> On Tuesday, 9 April 2013 at 17:06:47 UTC, Namespace wrote:
>>>
 http://wiki.dlang.org/DIP36

>>>
>>> I see no point in adding that much complexity for something that can
>>> mostly be automated.
>>>
>>
>> Can you explain this a bit more extensively, probably with some sort of
>> counter-proposal? I can't see where complexity comes from, this DIP
>> introduces literally zero special cases.
>>
>
> I think he means that the Compiler could/should decide, if something
> should passed by ref or by value. But I think that would be far more
> complex than scope ref/in ref.
>

Right. Yeah, seems more complex for sure. It's also
unreliable/unpredictable. Same as with the pure conversation, I really
prefer explicit control of things to at least be an option. I really don't
want to be guessing about things that I may be depending on.

Re: DIP 36: Rvalue References

2013-04-10 Thread Minas Mina


I hope this proposal gets implemented soon :)

Re: DIP 36: Rvalue References

2013-04-10 Thread deadalnix


On Wednesday, 10 April 2013 at 12:40:57 UTC, Manu wrote:
On 10 April 2013 22:27, Namespace  
wrote:



On Wednesday, 10 April 2013 at 11:37:43 UTC, Dicebot wrote:


On Wednesday, 10 April 2013 at 11:36:22 UTC, deadalnix wrote:


On Tuesday, 9 April 2013 at 17:06:47 UTC, Namespace wrote:


http://wiki.dlang.org/DIP36



I see no point in adding that much complexity for something 
that can

mostly be automated.



Can you explain this a bit more extensively, probably with 
some sort of
counter-proposal? I can't see where complexity comes from, 
this DIP

introduces literally zero special cases.



I think he means that the Compiler could/should decide, if 
something
should passed by ref or by value. But I think that would be 
far more

complex than scope ref/in ref.



Right. Yeah, seems more complex for sure. It's also
unreliable/unpredictable. Same as with the pure conversation, I 
really
prefer explicit control of things to at least be an option. I 
really don't

want to be guessing about things that I may be depending on.


It is certainly more complex on the compiler implementation side. 
But the ends result can be much better, and any program would 
benefit from it.


Plus, do you know that depending on the ABI, some pass by value 
are already transformed as pass by ref ? This is already in place 
in some cases, and have been proven to work well.


Back to the DIP, it fail to define how and where lvalue are 
implicitely created, what are their lifetime, etc . . .


Finally, ref and scope would benefit much more from the 
introduction of lifetime into D than hacking around.

Re: DIP 36: Rvalue References


On Wednesday, 10 April 2013 at 13:51:59 UTC, deadalnix wrote:
It is certainly more complex on the compiler implementation 
side. But the ends result can be much better, and any program 
would benefit from it.


Plus, do you know that depending on the ABI, some pass by value 
are already transformed as pass by ref ? This is already in 
place in some cases, and have been proven to work well.


Back to the DIP, it fail to define how and where lvalue are 
implicitely created, what are their lifetime, etc . . .


Finally, ref and scope would benefit much more from the 
introduction of lifetime into D than hacking around.


DIP36 was intentionally tweaked in such way so that compiler is 
free to do any implementation and swap pass by values with pass 
by ref if needed - until it preserves the semantics guarantees.


Introduction of lifetime is very beneficial but not really 
relevant to _this_ DIP. If lifetime is well defined, idea of 
making struct literal lvalues may be discussed, but that is a 
different story.


I don't care about how complex compiler implementation is but 
behavior needs to be predictable by programmer, no observable 
magic. Scope happens to have exactly the semantics needed to make 
stuff work right here and right now, with no relation to possible 
improvements of ref and rvalue/lvalue system.

Re: DIP 36: Rvalue References

2013-04-10 Thread Zach the Mystic


On Wednesday, 10 April 2013 at 09:31:22 UTC, Dicebot wrote:
Safety of "ref" is not problem that this DIP tries to solve. 
It should be solved with DIP25. Now ref is not safe (despite 
the fact it pretends to) and it still won't be after DIP36 
approval, not until issue is addressed in DIP25.


No need to mix it.


My proposed DIP35 uses 'scope' to enhance DIP25. I don't 
believe DIP25 is really complete. 'scope' would help it a lot, 
but that would give more meanings to 'scope' which are 
potentially in conflict with it just meaning rvalue temp.


And how it is relevant to DIP36? It solves specific issue. It 
provides some hints how other DIP may use new opportunities. It 
is up to DIP25 and DIP35 to be built on top if this gets 
accepted first. Or other way around.


DIPs are built in context for current language implementation, 
not some potential other DIPs.


I would say, if you are going to use a keyword ('scope' in this 
case) to mean something, and there are other uses of the keyword 
which are also useful, you have to decide first if there is a 
conflict, and second how it should be resolved. In my opinion 
there is a conflict. The only existing definition of 'scope' 
parameters we have suggets that it means the parameter may not be 
escaped. It turns out that such a parameter is possibly necessary 
to successfully seal references in D. If a different proposal 
wants to use the word to mean something else, then it's important 
to decide either that two different features require two 
different parameter attributes (DIP36 understands this, and 
proposes several alternate possibilities to use), or that each 
feature can always automatically include the other feature 
without any problems (possible but see my other responses), or 
that only one feature is truly important enough to be included in 
the language, allowing you to disregard the other feature 
entirely.

Re: DIP 36: Rvalue References

2013-04-11 Thread Dicebot


On Wednesday, 10 April 2013 at 07:43:57 UTC, Dicebot wrote:

...


Kenji, got any comments/objections on this point of view? It is 
not that late to change it back ;)

Re: DIP 36: Rvalue References

2013-04-11 Thread kenji hara

I think 'scope ref' should always pass the address of given argument to the
function body. Because 'ref' always means "pass the address of given
argument". So, if given argument is an rvalue, compiler will allocate a
temporary on stack, save the rvalue in it, and take its address. If
argument is an lvalue, take its address directly. It should be always true,
eve if the parameter type is small int.

Essentially language semantics should not reflect low level implementation
detail. After DIP36 is accepted, we will get a full set of tools for
efficiency - T, ref T, and scope ref T. From the toolbox, you can choose
the most efficient storage class based on the T.

void foo(MostEfficientParameters!(int, string, S, int[1024]) params) { ... }

Kenji Hara

2013/4/11 Dicebot 

> On Wednesday, 10 April 2013 at 07:43:57 UTC, Dicebot wrote:
>
>> ...
>>
>
> Kenji, got any comments/objections on this point of view? It is not that
> late to change it back ;)
>

Re: DIP 36: Rvalue References

2013-04-11 Thread Dicebot

You don't think exposing creation of temporaries is low level 
implementation

detail?

On Thursday, 11 April 2013 at 08:14:00 UTC, kenji hara wrote:

...

Kenji Hara

Re: DIP 36: Rvalue References

2013-04-11 Thread kenji hara

2013/4/11 Dicebot 

> You don't think exposing creation of temporaries is low level
> implementation
> detail?
>

No. In this case, it is the purpose of 'scope ref', not is the
implementation details.

What is the language semantics and what is the implementation detail - that
varies depending on what is properly supported in the language. D
explicitly distinguishes lvalues and rvalues, and also defines that rvalue
cannot be taken its address (== reference).

The purpose of 'scope ref' is that the parameter can be received both
lvalues and rvalues with one function body. To receive lvalues, scope ref
should take the address of them. As a conclusion, to receive rvalues
temporary creation is necessary in caller site.

Yes, it mostly represents implementation details, but it is still
description of language semantics.
This confusion is almost inevitable. Because the essential purpose of
'scope ref' is already "efficiency".

Furthermore, note that the desired "efficiency" is not "the speed of
argument passing".
I recognize that it is "to avoid code bloating of function body". If you
need "most efficient argument passing", you should choose it from the
argument type. 'scope ref' is not a thing to make the choice automatic.

Kenji Hara

Re: DIP 36: Rvalue References

2013-04-11 Thread Dicebot


On Thursday, 11 April 2013 at 09:27:11 UTC, kenji hara wrote:

No. In this case, it is the purpose of 'scope ref', not is the
implementation details.

What is the language semantics and what is the implementation 
detail - that
varies depending on what is properly supported in the language. 
D
explicitly distinguishes lvalues and rvalues, and also defines 
that rvalue

cannot be taken its address (== reference).

The purpose of 'scope ref' is that the parameter can be 
received both
lvalues and rvalues with one function body. To receive lvalues, 
scope ref
should take the address of them. As a conclusion, to receive 
rvalues

temporary creation is necessary in caller site.

Yes, it mostly represents implementation details, but it is 
still

description of language semantics.
This confusion is almost inevitable. Because the essential 
purpose of

'scope ref' is already "efficiency".

Furthermore, note that the desired "efficiency" is not "the 
speed of

argument passing".
I recognize that it is "to avoid code bloating of function 
body". If you
need "most efficient argument passing", you should choose it 
from the
argument type. 'scope ref' is not a thing to make the choice 
automatic.


Kenji Hara


I trust your judgement then, will update DIP then. But I am still 
afraid of confusion.

Re: DIP 36: Rvalue References

2013-04-11 Thread Jonas Drewsen



Isn's it too strict? The following example would not work because 
you cannot take the address of the parameter. But it would still 
be nice to have it passed by ref and scoped:


struct Item
{
   int data;
   Item* next;
}

void printThem(const scope ref Item first)
{
auto it = &first;
while (it)
{
std.stdio.writeln(*it);
it = it.next;
}
}


/Jonas

Re: DIP 36: Rvalue References

2013-04-11 Thread Zach the Mystic


On Wednesday, 10 April 2013 at 07:43:57 UTC, Dicebot wrote:
Consistent behavior is important but it does not change the 
fact that there two types of rvalues in the language - raw 
literals (42, A.init) and constructed temporaries (A()). 
Accepting rvalues of the first type in as mutable references 
may be consistent from the point of view of syntax by 
_extremely_ confusing semantics-wise:


void foo(scope ref int x)
{
x = 32;
}

void main()
{
foo(42); // erm, 32 is the new 42 or ref is not actually 
ref?

}

Beauty of "in ref" is that in that case user can't use it in 
any way that may disclose the fact the temporary variable for 
non-adressable entities is created. For mutable scope ref it is 
not the case and we must leak implementation details, which is 
never good.


Another concern is that if someone tries to pass "42" as a 
mutable ref, most likely he does not really wants it to be 
mutable and "in ref" is a better match.


This may not be consistent from the point of view of generic 
code, but it is now consistent from the point of view of 
programmer : A(...) construct always create temporaries, raw 
value literals never do (well, they will for "in ref" but you 
can't observe it). I think this is much more important.


One extra thing to note is that test32 may actually work if it 
instantiates T as "const int" for 42 (making it "in ref" 
essentially), but I don't know if D rules allow it.


I think this is another case of two different features which may 
or may not be paired up together. If it is always okay to have A 
accompany B, and B accompany A, it's good from a syntax point of 
view because you only need one syntax for both of them. But if 
you pair them up and later think, I wish I could do A without 
having to do B, or B without having to do A, then you'll regret 
pairing them up because you'll either have to break code or live 
with what you've got.


When you say we shouldn't be able to return an rvalue 
temporaries, even if they're safe, I think it's reasonable, not a 
life or death issue. But if 'scope ref' is used to mean that, 
you're also saying that any *other* feature which wants 'scope', 
such as 'ref' safety (why I created DIP35, to highlight this), 
will automatically also have to take rvalue temps because they're 
bound up with it, it's more risky.


Some of these will be a judgment call - A and B  are sometimes so 
similar that it's better to save on syntax than differentiate 
them. 'const scope' might be the same way.

Re: DIP 36: Rvalue References

2013-04-12 Thread Namespace

Kenji, what do you think, should we / you create a pull request 
for the proposal?
Most seem to be very satisfied with the proposal. In addition, a 
pull request would rather cause that Walter and Andrei join the 
discussion.

Re: DIP 36: Rvalue References

2013-04-17 Thread Namespace


Thanks to Kenji there is now a corresponding pull request:
https://github.com/D-Programming-Language/dmd/pull/1903

Let us hope that it will be quickly merged.

I still hope that Andrei or Walter say something to this topic.
Maybe we have a chance with this pull request. ;)

Re: DIP 36: Rvalue References

2013-04-17 Thread Minas Mina


On Wednesday, 17 April 2013 at 17:50:07 UTC, Namespace wrote:

Thanks to Kenji there is now a corresponding pull request:
https://github.com/D-Programming-Language/dmd/pull/1903

Let us hope that it will be quickly merged.

I still hope that Andrei or Walter say something to this topic.
Maybe we have a chance with this pull request. ;)


Finally, I'm tired of copy-pasting the same code just to change 
to "ref". :)

Re: DIP 36: Rvalue References

2013-04-17 Thread Namespace

Finally, I'm tired of copy-pasting the same code just to change 
to "ref". :)


What do you mean?

Re: DIP 36: Rvalue References

2013-04-17 Thread Minas Mina


On Wednesday, 17 April 2013 at 19:29:51 UTC, Namespace wrote:
Finally, I'm tired of copy-pasting the same code just to 
change to "ref". :)


What do you mean?


This:

/// dot product of two Vector3 vectors
auto dot(T) (Vector3!T u, Vector3!T v) @safe pure nothrow
{
return u.x * v.x + u.y * v.y + u.z * v.z;
}

/// dot product of two Vector3 vectors
auto dot(T) (ref Vector3!T u, ref Vector3!T v) @safe pure nothrow
{
return u.x * v.x + u.y * v.y + u.z * v.z;
}

Re: DIP 36: Rvalue References

2013-04-17 Thread Namespace


On Wednesday, 17 April 2013 at 20:18:56 UTC, Minas Mina wrote:

On Wednesday, 17 April 2013 at 19:29:51 UTC, Namespace wrote:
Finally, I'm tired of copy-pasting the same code just to 
change to "ref". :)


What do you mean?


This:

/// dot product of two Vector3 vectors
auto dot(T) (Vector3!T u, Vector3!T v) @safe pure nothrow
{
return u.x * v.x + u.y * v.y + u.z * v.z;
}

/// dot product of two Vector3 vectors
auto dot(T) (ref Vector3!T u, ref Vector3!T v) @safe pure 
nothrow

{
return u.x * v.x + u.y * v.y + u.z * v.z;
}


Oh, I thought you had anything against the DIP. ^^
Yes that annoys me as well. That's why I really hope that we can 
fix this vexing issue soon. :)

Re: DIP 36: Rvalue References

2013-04-17 Thread Zach the Mystic


On Wednesday, 17 April 2013 at 20:27:27 UTC, Namespace wrote:

Oh, I thought you had anything against the DIP. ^^
Yes that annoys me as well. That's why I really hope that we 
can fix this vexing issue soon. :)


Yes, this temp ref issue is far more important than the ref 
safety issue I've been concerning myself with. I wouldn't mind if 
the objections I've brought up fall in favor of whatever it takes 
to get temp ref solved. ref safety is important, but it's nowhere 
near the shear *annoyingness* factor of rvalue temp refs. But 
someone besides me would need to evaluate both DIP 35 & 36 to see 
if there were any real conflicts there.

Re: DIP 36: Rvalue References

Yes, this temp ref issue is far more important than the ref 
safety issue I've been concerning myself with. I wouldn't mind 
if the objections I've brought up fall in favor of whatever it 
takes to get temp ref solved. ref safety is important, but it's 
nowhere near the shear *annoyingness* factor of rvalue temp 
refs.
That is true, but it makes the impression that, with the 
exception of Kenji, none of the core developers is interested in 
a solution to this problem.
The DIP was not much discussed and the pull request is regularly 
overlooked / ignored.
Also in the discussion on the pull request no comments for Walter 
or Andrei are found. Although the pull is complete and has passed 
all the tests and would be ready to merge.
Also this thread is completely ignored even though both write in 
this forum regularly and participate in other discussions.
To me this makes the impression that they were not interested in 
this problem (or in our solution to the problem). Either they 
want the problem does not solve or try to solve it in their own 
way and that can take a very long time.
At least an annotation what of both is the case or if I see it 
completely wrong, would be polite.


But someone besides me would need to evaluate both DIP 35 & 36 
to see if there were any real conflicts there.

Yes that would be good.

Re: DIP 36: Rvalue References


On Saturday, 20 April 2013 at 14:42:57 UTC, Namespace wrote:
Yes, this temp ref issue is far more important than the ref 
safety issue I've been concerning myself with. I wouldn't mind 
if the objections I've brought up fall in favor of whatever it 
takes to get temp ref solved. ref safety is important, but 
it's nowhere near the shear *annoyingness* factor of rvalue 
temp refs.
That is true, but it makes the impression that, with the 
exception of Kenji, none of the core developers is interested 
in a solution to this problem.
The DIP was not much discussed and the pull request is 
regularly overlooked / ignored.


I don't think adding more to the language is the sane thing to do 
right now.

Re: DIP 36: Rvalue References

I don't think adding more to the language is the sane thing to 
do right now.


Why not? Could you explain this?
This issue is discussed since more than a year and it is a very 
annoying issue.
And even if Walter and Andrei are of this opinion, it would still 
only polite when they explain in detail why they think this.

Re: DIP 36: Rvalue References


On Saturday, 20 April 2013 at 15:17:39 UTC, Namespace wrote:
I don't think adding more to the language is the sane thing to 
do right now.


Why not? Could you explain this?
This issue is discussed since more than a year and it is a very 
annoying issue.
And even if Walter and Andrei are of this opinion, it would 
still only polite when they explain in detail why they think 
this.


Listen, this issue is very real, but it is mostly about 
performance. I'll tell you something : the best performance 
improvement is the one that bring your program from non working 
state to working one. And right now, many existing feature are 
broken.


The let's add whatever feature we have in mind is the very cause 
of the state of the language right now.

Re: DIP 36: Rvalue References

Listen, this issue is very real, but it is mostly about 
performance. I'll tell you something : the best performance 
improvement is the one that bring your program from non working 
state to working one. And right now, many existing feature are 
broken.


The let's add whatever feature we have in mind is the very 
cause of the state of the language right now.


As far as I remember, you were in favor of the introduction of 
the feature "attribute inference for auto functions". Also a new 
feature but none that would solve problems which are discussed 
since more than a year.
Furthermore this issue should be solved with the introduction of 
"auto ref" (AFAIK introduced with dmd 2.035?). So this is not 
like "add whatever feature" but rather "fix an issue, that is 
often discussed and annoys many people".

Re: DIP 36: Rvalue References

2013-04-20 Thread John Colvin


On Saturday, 20 April 2013 at 15:23:35 UTC, deadalnix wrote:

On Saturday, 20 April 2013 at 15:17:39 UTC, Namespace wrote:
I don't think adding more to the language is the sane thing 
to do right now.


Why not? Could you explain this?
This issue is discussed since more than a year and it is a 
very annoying issue.
And even if Walter and Andrei are of this opinion, it would 
still only polite when they explain in detail why they think 
this.


Listen, this issue is very real, but it is mostly about 
performance. I'll tell you something : the best performance 
improvement is the one that bring your program from non working 
state to working one. And right now, many existing feature are 
broken.


The let's add whatever feature we have in mind is the very 
cause of the state of the language right now.


Sadly, I have to agree on this. As nice as many new feature ideas 
are, they are far from priorities when there are multiple core 
mechanics that are broken.

Re: DIP 36: Rvalue References


On Saturday, 20 April 2013 at 15:39:42 UTC, Namespace wrote:
Listen, this issue is very real, but it is mostly about 
performance. I'll tell you something : the best performance 
improvement is the one that bring your program from non 
working state to working one. And right now, many existing 
feature are broken.


The let's add whatever feature we have in mind is the very 
cause of the state of the language right now.


As far as I remember, you were in favor of the introduction of 
the feature "attribute inference for auto functions". Also a 
new feature but none that would solve problems which are 
discussed since more than a year.


Half of the standard lib is broken because it is annotated 
incorrectly.


Furthermore this issue should be solved with the introduction 
of "auto ref" (AFAIK introduced with dmd 2.035?). So this is 
not like "add whatever feature" but rather "fix an issue, that 
is often discussed and annoys many people".


DIP36 discuss the addition of new features.

Re: DIP 36: Rvalue References

2013-04-20 Thread Dicebot

You miss quite an important point - DIP36 does not add new 
feature. It partially defines existing feature (scope) to replace 
an existing but broken solution (auto ref). Nothing new is really 
added to the language, only existing stuff better defined.

Re: DIP 36: Rvalue References

Sadly, I have to agree on this. As nice as many new feature 
ideas are, they are far from priorities when there are multiple 
core mechanics that are broken.


There is no reason to prioritize DIP 36. Kenji, Dicebot and I did 
most of the work. The DIP is written and all necessary 
information are described in detail there with examples. The code 
also exists and there is even a pull request which has passed all 
the tests. Thus, this proposal is linked with not much work. Most 
of it was taken over by others.
Due to this, it really is not asking too much to get a note if 
this pull is accepted or rejected. Of course, with detailed 
justification.

Re: DIP 36: Rvalue References

2013-04-20 Thread Minas Mina


On Saturday, 20 April 2013 at 15:23:35 UTC, deadalnix wrote:

On Saturday, 20 April 2013 at 15:17:39 UTC, Namespace wrote:
I don't think adding more to the language is the sane thing 
to do right now.


Why not? Could you explain this?
This issue is discussed since more than a year and it is a 
very annoying issue.
And even if Walter and Andrei are of this opinion, it would 
still only polite when they explain in detail why they think 
this.


Listen, this issue is very real, but it is mostly about 
performance. I'll tell you something : the best performance 
improvement is the one that bring your program from non working 
state to working one. And right now, many existing feature are 
broken.


The let's add whatever feature we have in mind is the very 
cause of the state of the language right now.



I thought D was driven by its community.

Re: DIP 36: Rvalue References

2013-04-20 Thread John Colvin


On Saturday, 20 April 2013 at 16:11:49 UTC, Namespace wrote:
Sadly, I have to agree on this. As nice as many new feature 
ideas are, they are far from priorities when there are 
multiple core mechanics that are broken.


There is no reason to prioritize DIP 36. Kenji, Dicebot and I 
did most of the work. The DIP is written and all necessary 
information are described in detail there with examples. The 
code also exists and there is even a pull request which has 
passed all the tests. Thus, this proposal is linked with not 
much work. Most of it was taken over by others.
Due to this, it really is not asking too much to get a note if 
this pull is accepted or rejected. Of course, with detailed 
justification.


How about "on hold"? (Not that I have any say in it at all)

The fact is, there's much more to any change than simply 
implementing it. Changes break unexpected things. There are 
always extra corner cases not considered. There are always bugs 
and inconsistencies.


Although it's great that you and some others have done the 
legwork to implement this proposal, it may have to wait until 
other more urgent problems have been fixed.

Re: DIP 36: Rvalue References


How about "on hold"? (Not that I have any say in it at all)
As long as it is implemented in the near future and we must not 
wait another year (not even a half) it is ok.
But the fact is, that we don't know what state it has, because we 
get no response.


The fact is, there's much more to any change than simply 
implementing it. Changes break unexpected things. There are 
always extra corner cases not considered. There are always bugs 
and inconsistencies.
Could be, but I don't see what could be broken by this DIP. All 
contingencies are listed also in the DIP (and that are not many). 
And it passed all tests what is crucial.


Although it's great that you and some others have done the 
legwork to implement this proposal, it may have to wait until 
other more urgent problems have been fixed.
Could be, but I don't know why. Which other fix is necessary for 
this pull?
But also this would be ok, as long as we know, _which_ problems 
must be fixed.

Re: DIP 36: Rvalue References

2013-04-20 Thread Timon Gehr


On 04/20/2013 07:34 PM, Minas Mina wrote:

...

I thought D was driven by its community.


(This is its community.)

Re: DIP 36: Rvalue References

2013-04-20 Thread Timon Gehr


On 04/20/2013 05:56 PM, Dicebot wrote:

You miss quite an important point - DIP36 does not add new feature. It
partially defines existing feature (scope) to replace an existing but
broken solution (auto ref). Nothing new is really added to the language,
only existing stuff better defined.


_New meaning_ is assigned to existing grammar whose original purpose is 
at most loosely related to the _new_ features.


I do not think that making 'scope' indicate an rvalue reference is 
particularly future proof.

Re: DIP 36: Rvalue References

2013-04-20 Thread Zach the Mystic


On Saturday, 20 April 2013 at 14:42:57 UTC, Namespace wrote:
That is true, but it makes the impression that, with the 
exception of Kenji, none of the core developers is interested 
in a solution to this problem.


I don't think that's true.

The DIP was not much discussed and the pull request is 
regularly overlooked / ignored.


Patience!

Also in the discussion on the pull request no comments for 
Walter or Andrei are found. Although the pull is complete and 
has passed all the tests and would be ready to merge.
Also this thread is completely ignored even though both write 
in this forum regularly and participate in other discussions.
To me this makes the impression that they were not interested 
in this problem (or in our solution to the problem). Either 
they want the problem does not solve or try to solve it in 
their own way and that can take a very long time.
At least an annotation what of both is the case or if I see it 
completely wrong, would be polite.


But someone besides me would need to evaluate both DIP 35 & 36 
to see if there were any real conflicts there.

Yes that would be good.


I think you've done a good job with the feature and the 
presentation.


But I personally don't feel like it's my job to rush D's 
development. Part of the reason is that D is already way ahead of 
most if not all of the competition in terms of sheer language 
design. When you extend into unknown territory, it's sometimes 
wise just to stay where you are for a bit, to give yourself time 
to adapt and build up a foundation.


I don't think your work will go unnoticed or unappreciated. Most 
of the time when people don't get back to you, it's because 
they're busy with other things. Probably the best thing you can 
do is say, well, my work on this feature is done, what else can I 
improve around here? This feature and this issue won't go away, 
IMO.

Re: DIP 36: Rvalue References

2013-04-20 Thread Manu

On 21 April 2013 06:51, Timon Gehr  wrote:

> On 04/20/2013 05:56 PM, Dicebot wrote:
>
>> You miss quite an important point - DIP36 does not add new feature. It
>> partially defines existing feature (scope) to replace an existing but
>> broken solution (auto ref). Nothing new is really added to the language,
>> only existing stuff better defined.
>>
>
> _New meaning_ is assigned to existing grammar whose original purpose is at
> most loosely related to the _new_ features.
>
> I do not think that making 'scope' indicate an rvalue reference is
> particularly future proof.
>

That's not what scope does. Scope promises that the variables will not
escape the scope. And as such, just happens to make passing a temporary by
ref safe.
It does not implement r-value ref's. It simply allows refs to temporaries
to be considered a safe operation.

This DIP is actually likely to solve an important source of problems,
consider:

void func(const ref matrix m);

func(x.getMatrix()); // compile error!

// fu*^&%$ing hell! you piece of &%^#≈¿$!
// ...

matrix temp = x.getMatrix();
func(temp); // no more compile error! (but equally unsafe/dangerous)

In this example, the 'solution', which is what everybody does right now, is
exactly as unsafe as the attempted call with the r-value.
ref, as in the language right now, is a fundamentally unsafe operation...
and not only is it technically unsafe, a programmer can't even know if it
is practically unsafe or not.
Since they have a habit of using this hack, they may unknowingly use it in
a call site where it's not *practically* safe to do it. They can't know,
and by habit (and frustration) they're trained to use this hack everywhere.

With 'scope ref' (or 'in ref'), the programmer now has a guide to say
whether it's safe to pass a temporary or not. In the future, if the
function does not receive scope ref, perhaps the programmer will start to
presume that it is NOT safe to pass a temporary, and stop doing so via the
current local-variable hack.

scope was always intended to implement this promise as far as I'm lead to
believe(?).

In my experience showing D to new people, this is the #1 complaint. It's
the first one that comes up, every time (which really doesn't help with
first impressions), and I'm fairly sure every single person I've introduced
to D has complained about this.
It's kind of embarrassing when I'm saying that D is really cool, and then I
have to start making excuses and apologising for this, and assure them that
it's a known issue, and it'll be fixed one day.

Re: DIP 36: Rvalue References


On Saturday, 20 April 2013 at 18:00:50 UTC, Namespace wrote:
The fact is, there's much more to any change than simply 
implementing it. Changes break unexpected things. There are 
always extra corner cases not considered. There are always 
bugs and inconsistencies.
Could be, but I don't see what could be broken by this DIP. All 
contingencies are listed also in the DIP (and that are not 
many). And it passed all tests what is crucial.




The DIP for instance, consider that const scope ref is 
semantically equivalent to pass by value, when it isn't (and not 
only for performance reasons, but for aliasing reasons). Nothing 
is considered about it.

Re: DIP 36: Rvalue References

2013-04-20 Thread Zach the Mystic


On Sunday, 21 April 2013 at 00:51:31 UTC, Manu wrote:
That's not what scope does. Scope promises that the variables 
will not
escape the scope. And as such, just happens to make passing a 
temporary by

ref safe.
It does not implement r-value ref's. It simply allows refs to 
temporaries

to be considered a safe operation.


It's a two-fer! (2 for 1 deal)

This DIP is actually likely to solve an important source of 
problems,

consider:

void func(const ref matrix m);


func(x.getMatrix()); // compile error!

// fu*^&%$ing hell! you piece of &%^#≈¿$!
// ...

matrix temp = x.getMatrix();
func(temp); // no more compile error! (but equally 
unsafe/dangerous)


It's hard to fully understand this example without getMatrix() 
defined, and why func() is unsafe (does it escape the 
reference?). Help!




In my experience showing D to new people, this is the #1 
complaint. It's
the first one that comes up, every time (which really doesn't 
help with
first impressions), and I'm fairly sure every single person 
I've introduced

to D has complained about this.
It's kind of embarrassing when I'm saying that D is really 
cool, and then I
have to start making excuses and apologising for this, and 
assure them that

it's a known issue, and it'll be fixed one day.


Yikes.

Re: DIP 36: Rvalue References

2013-04-20 Thread Manu

On 21 April 2013 11:36, Zach the Mystic  wrote:

> On Sunday, 21 April 2013 at 00:51:31 UTC, Manu wrote:
>
>> That's not what scope does. Scope promises that the variables will not
>> escape the scope. And as such, just happens to make passing a temporary by
>> ref safe.
>> It does not implement r-value ref's. It simply allows refs to temporaries
>> to be considered a safe operation.
>>
>
> It's a two-fer! (2 for 1 deal)
>
>
>  This DIP is actually likely to solve an important source of problems,
>> consider:
>>
>> void func(const ref matrix m);
>>
>>
>> func(x.getMatrix()); // compile error!
>>
>> // fu*^&%$ing hell! you piece of &%^#≈¿$!
>> // ...
>>
>> matrix temp = x.getMatrix();
>> func(temp); // no more compile error! (but equally unsafe/dangerous)
>>
>
> It's hard to fully understand this example without getMatrix() defined,
> and why func() is unsafe (does it escape the reference?). Help!


definition:
  matrix getMatrix(T x); // this is all you know

That's the point of the example. You _don't know_ if func() is unsafe, does
it escape the reference? But you need to pass a temp anyway, you have no
bearing on whether you should just hack it to work, or reconsider the
problem.
And when 99 times out of 100, the correct answer is 'hack it to work',
you're basically asking for a spectacular bug in that other 1% of cases.


 
>> In my experience showing D to new people, this is the #1 complaint. It's
>> the first one that comes up, every time (which really doesn't help with
>> first impressions), and I'm fairly sure every single person I've
>> introduced
>> to D has complained about this.
>> It's kind of embarrassing when I'm saying that D is really cool, and then
>> I
>> have to start making excuses and apologising for this, and assure them
>> that
>> it's a known issue, and it'll be fixed one day.
>>
>
> Yikes.
>

Re: DIP 36: Rvalue References


On Sunday, 21 April 2013 at 02:13:54 UTC, Manu wrote:

definition:
  matrix getMatrix(T x); // this is all you know

That's the point of the example. You _don't know_ if func() is 
unsafe, does
it escape the reference? But you need to pass a temp anyway, 
you have no
bearing on whether you should just hack it to work, or 
reconsider the

problem.
And when 99 times out of 100, the correct answer is 'hack it to 
work',
you're basically asking for a spectacular bug in that other 1% 
of cases.




If that is your concern then the DIP36 is a very bad answer to it 
:
 - It require extra anotation, which wont be added most of the 
time.
 - It is inferior, and impair the introduction of lifetime. This 
isn't even discussed.

Re: DIP 36: Rvalue References

2013-04-20 Thread Zach the Mystic


On Sunday, 21 April 2013 at 02:13:54 UTC, Manu wrote:
 This DIP is actually likely to solve an important source of 
problems,

consider:

void func(const ref matrix m);


func(x.getMatrix()); // compile error!

// fu*^&%$ing hell! you piece of &%^#≈¿$!
// ...

matrix temp = x.getMatrix();
func(temp); // no more compile error! (but equally 
unsafe/dangerous)




It's hard to fully understand this example without getMatrix() 
defined,

and why func() is unsafe (does it escape the reference?). Help!



definition:
  matrix getMatrix(T x); // this is all you know

That's the point of the example. You _don't know_ if func() is 
unsafe, does
it escape the reference? But you need to pass a temp anyway, 
you have no
bearing on whether you should just hack it to work, or 
reconsider the

problem.
And when 99 times out of 100, the correct answer is 'hack it to 
work',
you're basically asking for a spectacular bug in that other 1% 
of cases.


Yeah, that's kind of what I meant when I said it's a two-for-one 
deal. You get to pass temps, and you get safety checking too.

Re: DIP 36: Rvalue References

2013-04-21 Thread Namespace


Well perhaps I am too hasty. I'll wait and see what is happening.

BTW: Thanks for the support. ;)

Re: DIP 36: Rvalue References

2013-04-21 Thread Dicebot


On Sunday, 21 April 2013 at 01:04:04 UTC, deadalnix wrote:

On Saturday, 20 April 2013 at 18:00:50 UTC, Namespace wrote:
The fact is, there's much more to any change than simply 
implementing it. Changes break unexpected things. There are 
always extra corner cases not considered. There are always 
bugs and inconsistencies.
Could be, but I don't see what could be broken by this DIP. 
All contingencies are listed also in the DIP (and that are not 
many). And it passed all tests what is crucial.




The DIP for instance, consider that const scope ref is 
semantically equivalent to pass by value, when it isn't (and 
not only for performance reasons, but for aliasing reasons). 
Nothing is considered about it.


At least some objections that make sense and I'd love to see in 
the very first announcement. If you see any issues that prevent 
treating "const scope int" as "const scope ref int", please tell 
about them and they will be addressed in DIP. Because it is the 
intention. I don't understand what aliasing are you speaking 
about.

Re: DIP 36: Rvalue References

2013-04-21 Thread Dicebot


On Sunday, 21 April 2013 at 02:50:02 UTC, deadalnix wrote:
If that is your concern then the DIP36 is a very bad answer to 
it :
 - It require extra anotation, which wont be added most of the 
time.


Automatic annotations are indicators of broken language. If 
someone will forget this one, he won't be able to use rvalues, no 
problem.


 - It is inferior, and impair the introduction of lifetime. 
This isn't even discussed.


Scope is the absolute minimal lifetime which is needed for this 
DIP. Any other lifetime options are simply irrelevant.

Re: DIP 36: Rvalue References


On Sunday, 21 April 2013 at 02:50:02 UTC, deadalnix wrote:
If that is your concern then the DIP36 is a very bad answer to 
it :
 - It require extra anotation, which wont be added most of the 
time.


It's possible to make it an error to pass a stack variable to a 
non-'scope' ref parameter. Or to make it an error in @safe code 
only. But those may be extreme measures. This also raises the 
issue that some types of escape are more dangerous than others.


ref T func(ref T a) {
  return a; // Escape by return, @safe if tracked at call site
  T* p = &a; // Escape to local pointer, @safe? if *p is tracked
  static T* s = &a; // Escape by global pointer, @system
}

In this sense, 'scope' is a blunt instrument, because you might 
want to say, 'returnable, but not globally assignable'.


'scope' is also the kind of thing which might be better as the 
default than as something you must specifically add. But then 
we'd need an attribute '@noscope' or something like that. Ugh.


 - It is inferior, and impair the introduction of lifetime. 
This isn't even discussed.


You've mentioned lifetime a couple times. Can you show what 
lifetime is and show how it contrasts with the current proposal, 
particularly DIP25?

Re: DIP 36: Rvalue References


On Sunday, 21 April 2013 at 10:55:37 UTC, Dicebot wrote:
The DIP for instance, consider that const scope ref is 
semantically equivalent to pass by value, when it isn't (and 
not only for performance reasons, but for aliasing reasons). 
Nothing is considered about it.


At least some objections that make sense and I'd love to see in 
the very first announcement. If you see any issues that prevent 
treating "const scope int" as "const scope ref int", please 
tell about them and they will be addressed in DIP. Because it 
is the intention. I don't understand what aliasing are you 
speaking about.


The only thing I could think of was if you want to 'ref' a 
reference type.


void func(ref Object o, scope Object o2, scope ref Object o3) {}

Would there be a difference between the type of o and o2, or of 
o2 and o3? There is inconsistency, but it would be nice to find 
use cases which make this inconsistency truly unworkable with 
other things in the language.

Re: DIP 36: Rvalue References


On Sunday, 21 April 2013 at 16:08:55 UTC, Zach the Mystic wrote:
Would there be a difference between the type of o and o2, or of 
o2 and o3? There is inconsistency, but it would be nice to find 
use cases which make this inconsistency truly unworkable with 
other things in the language.


('nice' in the sense that it would prove the suggestion to be 
unworkable as opposed to nice that it actually is unworkable...)

Re: DIP 36: Rvalue References

2013-04-21 Thread deadalnix


On Sunday, 21 April 2013 at 16:02:35 UTC, Zach the Mystic wrote:

On Sunday, 21 April 2013 at 02:50:02 UTC, deadalnix wrote:
If that is your concern then the DIP36 is a very bad answer to 
it :
- It require extra anotation, which wont be added most of the 
time.


It's possible to make it an error to pass a stack variable to a 
non-'scope' ref parameter. Or to make it an error in @safe code 
only. But those may be extreme measures.


No they aren't.

This also raises the issue that some types of escape are more 
dangerous than others.


This is why the notion of lifetime is needed.

'scope' is also the kind of thing which might be better as the 
default than as something you must specifically add. But then 
we'd need an attribute '@noscope' or something like that. Ugh.




Inference is always a solution.

- It is inferior, and impair the introduction of lifetime. 
This isn't even discussed.


You've mentioned lifetime a couple times. Can you show what 
lifetime is and show how it contrasts with the current 
proposal, particularly DIP25?


Here is a good link to start :
http://smallcultfollowing.com/babysteps/blog/2012/07/17/borrowed-pointer-tutorial/

Re: DIP 36: Rvalue References

2013-04-21 Thread Timon Gehr


On 04/21/2013 02:51 AM, Manu wrote:

On 21 April 2013 06:51, Timon Gehr mailto:timon.g...@gmx.ch>> wrote:

On 04/20/2013 05:56 PM, Dicebot wrote:

You miss quite an important point - DIP36 does not add new
feature. It
partially defines existing feature (scope) to replace an
existing but
broken solution (auto ref). Nothing new is really added to the
language,
only existing stuff better defined.


_New meaning_ is assigned to existing grammar whose original purpose
is at most loosely related to the _new_ features.

I do not think that making 'scope' indicate an rvalue reference is
particularly future proof.


That's not what scope does. Scope promises that the variables will not
escape the scope.


That is the intention, but this formulation is awfully imprecise. Hence 
nothing has been implemented.


void foo(scope ref int x){ ... }
void foo(scope int* x){ ... }
void foo(scope ref int* x){ ... }

// ???

struct S{
scope S* x; // ???
}


And as such, just happens to make passing a temporary
by ref safe.
...


But this is not about safety! Passing an rvalue by ref is disallowed 
even in @system code. Hence 'scope' is assigned a meaning different from 
its intention. The DIP also aims to add more special behaviour to 
built-in literals.

Re: DIP 36: Rvalue References

2013-04-21 Thread Timon Gehr


On 04/21/2013 06:27 AM, Zach the Mystic wrote:

...
Yeah, that's kind of what I meant when I said it's a two-for-one deal.
You get to pass temps, and you get safety checking too.


I prefer to buy separate products separately.

Re: DIP 36: Rvalue References

2013-04-21 Thread Manu

On 22 April 2013 03:39, Timon Gehr  wrote:

> On 04/21/2013 02:51 AM, Manu wrote:
>
>> On 21 April 2013 06:51, Timon Gehr > > wrote:
>>
>> On 04/20/2013 05:56 PM, Dicebot wrote:
>>
>> You miss quite an important point - DIP36 does not add new
>> feature. It
>> partially defines existing feature (scope) to replace an
>> existing but
>> broken solution (auto ref). Nothing new is really added to the
>> language,
>> only existing stuff better defined.
>>
>>
>> _New meaning_ is assigned to existing grammar whose original purpose
>> is at most loosely related to the _new_ features.
>>
>> I do not think that making 'scope' indicate an rvalue reference is
>> particularly future proof.
>>
>>
>> That's not what scope does. Scope promises that the variables will not
>> escape the scope.
>>
>
> That is the intention, but this formulation is awfully imprecise. Hence
> nothing has been implemented.
>
> void foo(scope ref int x){ ... }
> void foo(scope int* x){ ... }
> void foo(scope ref int* x){ ... }
>

What's the problem here?

// ???
>
> struct S{
> scope S* x; // ???
> }
>

I don't think scope on a struct member makes a lot of sense.
scope on a local variable declaration makes sense though, it would be able
to be assigned from another scope variable.
Although I would say the same about ref on a local variable declaration,
which is disallowed for completely unknown reasons.

 And as such, just happens to make passing a temporary
>> by ref safe.
>> ...
>>
>
> But this is not about safety! Passing an rvalue by ref is disallowed even
> in @system code. Hence 'scope' is assigned a meaning different from its
> intention. The DIP also aims to add more special behaviour to built-in
> literals.
>

Passing an r-value by ref is disallowed for arbitrary reasons. It can
easily create a temporary on the spot, but it doesn't, because it's not
considered good form to pass a temp to a function by ref.
This offers the possibility to pass a temp safely, and consequently, the
compiler may generate the temp for you.

Explain to me what meaning 'scope' has been assigned that is different from
it's intention? And what 'special behaviour'?

I honestly can't understand the objections, by you or deadalnix. Can
someone actually explain the problem clearly?

Re: DIP 36: Rvalue References

2013-04-21 Thread Manu

On 22 April 2013 04:23, Timon Gehr  wrote:

> On 04/21/2013 06:27 AM, Zach the Mystic wrote:
>
>> ...
>>
>> Yeah, that's kind of what I meant when I said it's a two-for-one deal.
>> You get to pass temps, and you get safety checking too.
>>
>
> I prefer to buy separate products separately.
>

I'm confused, what 2 products are you getting? Which of said products do
you actually object to?

Re: DIP 36: Rvalue References

2013-04-21 Thread Timon Gehr


On 04/21/2013 10:38 PM, Manu wrote:

...

That's not what scope does. Scope promises that the variables
will not
escape the scope.


That is the intention, but this formulation is awfully imprecise.
Hence nothing has been implemented.

void foo(scope ref int x){ ... }
void foo(scope int* x){ ... }
void foo(scope ref int* x){ ... }


What's the problem here?



What does 'scope' bind to? How to make it bind to something else (if at 
all)?


Does the proposal lock down 'scope' to always refer to every single 
pointer in the argument?




// ???

struct S{
 scope S* x; // ???
}


I don't think scope on a struct member makes a lot of sense.


void main()@safe{
S s;
auto t = S(&t);
}


scope on a local variable declaration makes sense though, it would be
able to be assigned from another scope variable.


'scope' shouldn't restrict the abstraction capability of structs.


Although I would say the same about ref on a local variable declaration,
which is disallowed for completely unknown reasons.


And as such, just happens to make passing a temporary
by ref safe.
...


But this is not about safety! Passing an rvalue by ref is disallowed
even in @system code. Hence 'scope' is assigned a meaning different
from its intention. The DIP also aims to add more special behaviour
to built-in literals.


Passing an r-value by ref is disallowed for arbitrary reasons.


Reasons not closely related to lack of 'scope'. So why bind the new rule 
to scope?



It can easily create a temporary on the spot, but it doesn't, because it's not
considered good form to pass a temp to a function by ref.  ...


Yet this is what everyone appears to want to do.


Explain to me what meaning 'scope' has been assigned that is different
from it's intention?


Implicit binding of (some) rvalues to ref.


And what 'special behaviour'?



If I read the ambiguous wording of the proposal correctly, the following 
code will be disallowed:


void foo1(scope ref int x) { }

void main(){
foo1(1); // error
}

But this will be fine:

void foo1(scope ref int x) { }
int bar(){ return 1; }

void main(){
foo1(bar());
}


I honestly can't understand the objections, by you or deadalnix. Can
someone actually explain the problem clearly?


Having a lot of ad-hoc programming language rules leads to a clumsy 
design. Eg. see the evolution of C++.

Re: DIP 36: Rvalue References


On Sunday, 21 April 2013 at 22:16:14 UTC, Timon Gehr wrote:
What does 'scope' bind to? How to make it bind to something 
else (if at all)?


I agree this is an issue. We need to enumerate the use cases. Is 
'scope' transitive or does it only apply to the first thing it 
hits?



Passing an r-value by ref is disallowed for arbitrary reasons.


Reasons not closely related to lack of 'scope'. So why bind the 
new rule to scope?


I think that it's possible to consider binding it as a convenient 
benefit of what 'scope' is generally meant to do anyway. I think 
the crossover of the two features is quite high. But it is a 
judgment call. Sometimes you may want to allow rvalue refs 
without prohibiting escaping them, or prohibit escaping them 
without allowing rvalues refs. But the point of 'scope' is that 
it safely takes stack references, which is exactly what rvalue 
temps are.


Maybe getting the two features separate is good, but since it 
will require another parameter attribute, it suggests a need for 
increased reliance on attribute inference to spare the programmer 
the trouble. The main reason to consider combining is that it 
saves on syntax.


If I read the ambiguous wording of the proposal correctly, the 
following code will be disallowed:


void foo1(scope ref int x) { }

void main(){
foo1(1); // error
}


My impression is that this would be allowed, not disallowed.


But this will be fine:

void foo1(scope ref int x) { }
int bar(){ return 1; }

void main(){
foo1(bar());
}


I think this would also be allowed.

Re: DIP 36: Rvalue References

2013-04-22 Thread Namespace

If I read the ambiguous wording of the proposal correctly, the 
following code will be disallowed:


void foo1(scope ref int x) { }

void main(){
   foo1(1); // error
}


My impression is that this would be allowed, not disallowed.


But this will be fine:

void foo1(scope ref int x) { }
int bar(){ return 1; }

void main(){
   foo1(bar());
}


I think this would also be allowed.


Yes, both cases are allowed. Dicebot initially wanted that scope 
ref does not accept literals (only in ref should this do), but 
Kenji has convinced him that this is unnecessary.

In the current pull of Kenji both cases works fine.
I also have taken the time and added the overload rules to the 
DIP.

Re: DIP 36: Rvalue References

2013-04-22 Thread Diggory

I realise I'm new here but this seems to be suggesting a whole 
load of changes and special cases for something that can be done 
in (IMHO) a much simpler way:


Why not simply make escaping a "ref" pointer an unsafe operation. 
The compiler should be able to detect this and report it without 
any changes to the syntax.


This should cover 99% of cases with no extra attributes required 
and no limitations on what you can do with a "ref" within the 
function. In the 1% of cases that a pointer needs to be escaped 
safely you can add an attribute (maybe "heap ref" or something, 
although perhaps some existing syntax could be used) that 
requires the input to have been allocated on the heap.


In the case that a stack variable is passed as a "heap ref" 
parameter the compiler can automatically promote it where 
possible, or if that's not possible, such as the variable being 
marked "scope" (existing meaning) then it should complain.


All values (including literals and temporaries) should then be 
able to be passed to a "ref const" parameter.


As far as I'm aware the only real purpose for R-value references 
is to implement move semantics. This could already be done using 
"ref in" syntax, as that makes no guarantees that the value is 
not going to be destroyed, only says that the value should be 
initialised prior to it being passed in. The only change would be 
to allow passing temporaries as "ref in".


To promise that the variable is not going to be modified "ref 
const" or "ref const in" can be used.

Re: DIP 36: Rvalue References

2013-04-22 Thread Namespace


On Monday, 22 April 2013 at 12:22:14 UTC, Diggory wrote:
I realise I'm new here but this seems to be suggesting a whole 
load of changes and special cases for something that can be 
done in (IMHO) a much simpler way:


Why not simply make escaping a "ref" pointer an unsafe 
operation. The compiler should be able to detect this and 
report it without any changes to the syntax.


This should cover 99% of cases with no extra attributes 
required and no limitations on what you can do with a "ref" 
within the function. In the 1% of cases that a pointer needs to 
be escaped safely you can add an attribute (maybe "heap ref" or 
something, although perhaps some existing syntax could be used) 
that requires the input to have been allocated on the heap.


In the case that a stack variable is passed as a "heap ref" 
parameter the compiler can automatically promote it where 
possible, or if that's not possible, such as the variable being 
marked "scope" (existing meaning) then it should complain.


All values (including literals and temporaries) should then be 
able to be passed to a "ref const" parameter.


As far as I'm aware the only real purpose for R-value 
references is to implement move semantics. This could already 
be done using "ref in" syntax, as that makes no guarantees that 
the value is not going to be destroyed, only says that the 
value should be initialised prior to it being passed in. The 
only change would be to allow passing temporaries as "ref in".


To promise that the variable is not going to be modified "ref 
const" or "ref const in" can be used.


This DIP already suggest 'in ref' besides 'scope ref'. 'in' is a 
shortcut for const scope.
And Andrei already reject variants which _only_ consist of 
non-mutable rvalue ref's (like const ref), because the const 
system in D is physical.
But this could be read on the wiki. You should read the DIP 
before you write here. ;)

Re: DIP 36: Rvalue References

2013-04-22 Thread Diggory


On Monday, 22 April 2013 at 13:17:36 UTC, Namespace wrote:

On Monday, 22 April 2013 at 12:22:14 UTC, Diggory wrote:
I realise I'm new here but this seems to be suggesting a whole 
load of changes and special cases for something that can be 
done in (IMHO) a much simpler way:


Why not simply make escaping a "ref" pointer an unsafe 
operation. The compiler should be able to detect this and 
report it without any changes to the syntax.


This should cover 99% of cases with no extra attributes 
required and no limitations on what you can do with a "ref" 
within the function. In the 1% of cases that a pointer needs 
to be escaped safely you can add an attribute (maybe "heap 
ref" or something, although perhaps some existing syntax could 
be used) that requires the input to have been allocated on the 
heap.


In the case that a stack variable is passed as a "heap ref" 
parameter the compiler can automatically promote it where 
possible, or if that's not possible, such as the variable 
being marked "scope" (existing meaning) then it should 
complain.


All values (including literals and temporaries) should then be 
able to be passed to a "ref const" parameter.


As far as I'm aware the only real purpose for R-value 
references is to implement move semantics. This could already 
be done using "ref in" syntax, as that makes no guarantees 
that the value is not going to be destroyed, only says that 
the value should be initialised prior to it being passed in. 
The only change would be to allow passing temporaries as "ref 
in".


To promise that the variable is not going to be modified "ref 
const" or "ref const in" can be used.


This DIP already suggest 'in ref' besides 'scope ref'. 'in' is 
a shortcut for const scope.
And Andrei already reject variants which _only_ consist of 
non-mutable rvalue ref's (like const ref), because the const 
system in D is physical.


The definition was somewhat hidden in the language reference and 
the first reference I found on the forum said that "in" simply 
meant that the value should be initialised by the caller (in the 
same way that "out" means that the callee initialises the value).


But this could be read on the wiki. You should read the DIP 
before you write here. ;)


I have but the DIP isn't consistent, for example in test1 it says 
R-value temporaries are allowed:

void test1(scope ref A a)


And in test32 it says they are not "Only adressable parameters 
are valid, not temporaries, no extra template instances.":

void test32(T)(scope ref T id)


And then:
test32(1337); // Fine, temporary mutable int variable is 
created with value 1337


Some other problems:
- Taking the address of a "scope ref" variable is prohibited even 
if the compiler can prove that it's safe.

- Implementing "forward" in the following is impossible:
doSomething(forward())
Obviously a function that simply returns what is passed in seems 
pointless but it may be very useful if sometimes that function 
should do something else.


All the restrictions on "scope ref" are exactly the same as the 
restrictions on "ref" if DIP25 is implemented, except that "scope 
ref" variables can't be returned (why is this the case when by 
following the same rules as DIP25 it can be safe?)


In which case why not allow passing R-value references as normal 
"ref" parameters? Why the need for "scope" when the only new 
restriction it introduces can (and should) be avoided.


DIP25 says that the return value of a ref function must not be 
assumed to have a lifetime longer than the shortest lifetime of 
any of its ref parameters. This formulation works just as well 
for an R-value reference as it does for a reference to a local 
variable, just that the first is slightly shorter.

Re: DIP 36: Rvalue References

2013-04-22 Thread Namespace


I have but the DIP isn't consistent
I must have overlooked this all the time. I will clean up this 
inconsistency.
The statement that only adressable elements are taken by scope 
ref is not true anymore. Both, scope ref and in ref can take any 
values. If it is an rvalue, a temporary is created and this is 
passed to the function.

Thanks for the hint.

In which case why not allow passing R-value references as 
normal "ref" parameters? Why the need for "scope" when the only 
new restriction it introduces can (and should) be avoided.


DIP25 says that the return value of a ref function must not be 
assumed to have a lifetime longer than the shortest lifetime of 
any of its ref parameters. This formulation works just as well 
for an R-value reference as it does for a reference to a local 
variable, just that the first is slightly shorter.


You should search for "rvalue references". 'ref' that takes both, 
lvalues and revalues was suggested and AFAIK rejected.
Otherwise you could open a new DIP or first a thread and ask for 
this. I'm curious of the answers.
This issue is very old (Exactly one year ago it was officially 
discussed for the first time). But the core problem exist since 
2.038. There was 'auto ref' introduced. This was the first 
attempt to solve this problem. Long long time ago and it annoys 
us still today.

Re: DIP 36: Rvalue References

2013-04-22 Thread Andrei Alexandrescu


On 4/20/13 11:17 AM, Namespace wrote:

I don't think adding more to the language is the sane thing to do
right now.


Why not? Could you explain this?
This issue is discussed since more than a year and it is a very annoying
issue.
And even if Walter and Andrei are of this opinion, it would still only
polite when they explain in detail why they think this.


In a way I wish my not being polite would be the main bottleneck for 
this. I've been extremely busy at work, then preparing for DConf 2013, 
and last week has been quite distracting what with the Boston bombings 
three miles away from where we live and all that. It strikes me as odd 
to be obligated to spend time on something just because someone else 
did, and framed as impolite if I don't.


That being said, I have made a pass through this DIP and I have the 
following concerns about it.


1. It defines a new language feature instead of improving the existing 
ones. At this point in the development of the language, our preference 
should be putting the existing features in good order.


2. The proposal is sketchy and does not give many details, such as the 
lifetime of temporaries bound to scope ref objects.


3. The relationship with auto ref is insufficiently described, e.g. 
there should be clarification on why auto ref cannot be improved to 
fulfill the desired role.


4. Above all this is a new language feature and again we want to resort 
to adding new feature only if it is clear that the existing features are 
insufficient and cannot be made sufficient. In particular we are much 
more inclined to impart real, demonstrable safety to "ref" and to make 
"auto ref" work as a reference that can bind to rvalues as well as lvalues.



Andrei

Re: DIP 36: Rvalue References

2013-04-22 Thread Zach the Mystic


On Monday, 22 April 2013 at 12:22:14 UTC, Diggory wrote:
Why not simply make escaping a "ref" pointer an unsafe 
operation. The compiler should be able to detect this and 
report it without any changes to the syntax.


In other words, 'scope' would be the default and require no 
explicit attribute. The first thing about this is the question of 
how much code it would break. I don't know the answer. Maybe a 
mockup of this idea could be used to create a sense of how badly 
it breaks existing code. The second question is how desirable is 
it as a feature. I think it may be quite desirable to have all 
refs be 'scope' by default, because the most common case will be 
the default, that refs will not be assigned to globals. Since 
there is a difference between the safety of returning by ref and 
the safety of assigning to heap or global addresses, there may 
need to be a distinction made between these two types of 'escape'.


This should cover 99% of cases with no extra attributes 
required and no limitations on what you can do with a "ref" 
within the function. In the 1% of cases that a pointer needs to 
be escaped safely you can add an attribute (maybe "heap ref" or 
something, although perhaps some existing syntax could be used) 
that requires the input to have been allocated on the heap.


'@noscope ref' has also been suggested, since the reference in 
question would also need to include static global data.


In the case that a stack variable is passed as a "heap ref" 
parameter the compiler can automatically promote it where 
possible, or if that's not possible, such as the variable being 
marked "scope" (existing meaning) then it should complain.


This is interesting to me. The only drawback I can think of is 
the aversion people have to introducing code which could secretly 
allocate, since many people want to avoid the garbage collector. 
It would only be an issue if the automatic promotion looked so 
much like a normal function call that it was extraordinarily 
difficult to distinguish it.


As far as I'm aware the only real purpose for R-value 
references is to implement move semantics. This could already 
be done using "ref in" syntax, as that makes no guarantees that 
the value is not going to be destroyed, only says that the 
value should be initialised prior to it being passed in. The 
only change would be to allow passing temporaries as "ref in".


The only existing definition of 'in' is that it means 'const 
scope', so it already means scope. I don't know if that is a good 
definition or not. I think that there remains a significant fear 
in this community of introducing changes which will break 
existing code. I don't myself know how justified this fear is, 
but it is nonetheless something to take very seriously when 
proposing new ideas.



I realise I'm new here


I'm new too. I'm glad to be able to have this discussion with you 
despite both of our being new. :-)

Re: DIP 36: Rvalue References

2013-04-22 Thread Tove

On Monday, 22 April 2013 at 20:02:12 UTC, Andrei Alexandrescu 
wrote:
4. Above all this is a new language feature and again we want 
to resort to adding new feature only if it is clear that the 
existing features are insufficient and cannot be made 
sufficient. In particular we are much more inclined to impart 
real, demonstrable safety to "ref" and to make "auto ref" work 
as a reference that can bind to rvalues as well as lvalues.


Why isn't DIP36 "scope ref" be future compatible with a future 
safe "auto ref"?


... and if in the future the compiler would be able to infer 
"scope ref" from "auto ref", this entire DIP could be reused, 
with the benefit that people could start using this functionality 
already today(there is a full pull request with a very small 
delta), before the auto inference is in place.

In a way I wish my not being polite would be the main
bottleneck for this. I've been extremely busy at work, then
preparing for DConf 2013, and last week has been quite
distracting what with the Boston bombings three miles away from
where we live and all that. It strikes me as odd to be
obligated to spend time on something just because someone else
did, and framed as impolite if I don't.

That being said, I have made a pass through this DIP and I have
the following concerns about it.

1. It defines a new language feature instead of improving the
existing ones. At this point in the development of the
language, our preference should be putting the existing
features in good order.

2. The proposal is sketchy and does not give many details, such
as the lifetime of temporaries bound to scope ref objects.

3. The relationship with auto ref is insufficiently described,
e.g. there should be clarification on why auto ref cannot be
improved to fulfill the desired role.

4. Above all this is a new language feature and again we want
to resort to adding new feature only if it is clear that the
existing features are insufficient and cannot be made
sufficient. In particular we are much more inclined to impart
real, demonstrable safety to "ref" and to make "auto ref" work
as a reference that can bind to rvalues as well as lvalues.

Andrei

I'm sorry that you take it so personally, but I know from
experience that some topics often get forgotten or ignored if you
don't ask about it more then once.
And that we have made so much effort should show that we work for
D and not against it and that we don't want to annoy you.

But you are right, scope ref is in comparison to auto ref a new
feature, but it is one with little sideeffects and minimized
implementation effort.
And Jonathan described here
(http://forum.dlang.org/thread/uswucstsooghescof...@forum.dlang.org?page=2#post-mailman.293.1364249651.4724.digitalmars-d-learn:40puremagic.com)
very well and in detail why auto ref is no solution for
non-templates.

In short:
It cannot work the same way for non-templates as it does for
template functions, because that would mean that non-templates
are also doubled 2^n times.
And the way scope ref would work, would mean, that auto ref works
different for templates and non-template functions.

But you'd better read Jonathan's answer.

Re: DIP 36: Rvalue References

On Monday, 22 April 2013 at 20:02:12 UTC, Andrei Alexandrescu 
wrote:
In a way I wish my not being polite would be the main 
bottleneck for this. I've been extremely busy at work, then 
preparing for DConf 2013, and last week has been quite 
distracting what with the Boston bombings three miles away from 
where we live and all that. It strikes me as odd to be 
obligated to spend time on something just because someone else 
did, and framed as impolite if I don't.


Thanks for your time. Sometimes even simple "Have seen it, will 
answer later / after DConf is priceless".


That being said, I have made a pass through this DIP and I have 
the following concerns about it.


1. It defines a new language feature instead of improving the 
existing ones. At this point in the development of the 
language, our preference should be putting the existing 
features in good order.


You consider "scope" a new language feature? And consider "auto 
ref" improvable solution. The very point of this DIP is to not 
create syntax-driven features, instead better define existing 
ones that make sense on their own so they can be used for same 
purpose.


2. The proposal is sketchy and does not give many details, such 
as the lifetime of temporaries bound to scope ref objects.


It can't because lifetime of temporaries is not defined in D at 
all and suck stuff needs to be consistent. It is not really 
different from a lifetime of struct literal temporary used for 
pass-by-value.


3. The relationship with auto ref is insufficiently described, 
e.g. there should be clarification on why auto ref cannot be 
improved to fulfill the desired role.


auto ref is a template-world entity. If by "improved" you mean 
"completely reworked" than sure, I can add this rationale. Will 
do today.


4. Above all this is a new language feature and again we want 
to resort to adding new feature only if it is clear that the 
existing features are insufficient and cannot be made 
sufficient. In particular we are much more inclined to impart 
real, demonstrable safety to "ref" and to make "auto ref" work 
as a reference that can bind to rvalues as well as lvalues.


Can't agree. "scope" is defined (but not implemented, a pity). 
Creating temporaries already exists for some cases. No really new 
language feature is added. No special syntax created. "scope ref" 
is still "scope" and "ref" with all semantics this imposes, 
rvalue references are just a side-effect. Contrary to this, "auto 
ref" IS a new feature created specifically for syntax hack. That 
is a very problem you are speaking about - introducing new 
concepts instead of making use of ones that exist for ages.

Re: DIP 36: Rvalue References


On Sunday, 21 April 2013 at 22:16:14 UTC, Timon Gehr wrote:

Passing an r-value by ref is disallowed for arbitrary reasons.


Reasons not closely related to lack of 'scope'. So why bind the 
new rule to scope?


I had quite an impression this was prohibited mostly because of 
@safe issues.

Re: DIP 36: Rvalue References


On Sunday, 21 April 2013 at 16:08:55 UTC, Zach the Mystic wrote:
The only thing I could think of was if you want to 'ref' a 
reference type.


void func(ref Object o, scope Object o2, scope ref Object o3) {}

Would there be a difference between the type of o and o2, or of 
o2 and o3? There is inconsistency, but it would be nice to find 
use cases which make this inconsistency truly unworkable with 
other things in the language.


Taking consistent approach with "ref", they all should have same 
type but bpth "ref" and "scope" differences are mangled into func 
name so that caller can take notion of proper behavior. I don't 
see an issue here.

Re: DIP 36: Rvalue References

2013-04-23 Thread Mehrdad

On Wednesday, 10 April 2013 at 06:11:29 UTC, Zach the Mystic 
wrote:

no value type is ever unsafe.



Delegates?

Re: DIP 36: Rvalue References

On Wednesday, 10 April 2013 at 07:39:25 UTC, Zach the Mystic 
wrote:

...


Kind of. I am thinking that scope is best when transitive for 
aggregation but not for indirection:


struct S { int a; int *b; }

void foo(scope S s)
{
static int *x1 = &(s.a); // error
static int *x2 = s.b; // fine
}

Re: DIP 36: Rvalue References

2013-04-23 Thread Timon Gehr


On 04/23/2013 10:10 AM, Dicebot wrote:

On Sunday, 21 April 2013 at 22:16:14 UTC, Timon Gehr wrote:

Passing an r-value by ref is disallowed for arbitrary reasons.


Reasons not closely related to lack of 'scope'. So why bind the new
rule to scope?


I had quite an impression this was prohibited mostly because of @safe
issues.


It is disallowed in @system code.

Re: DIP 36: Rvalue References

2013-04-23 Thread Timon Gehr


On 04/23/2013 10:16 AM, Dicebot wrote:

On Wednesday, 10 April 2013 at 07:39:25 UTC, Zach the Mystic wrote:

...


Kind of. I am thinking that scope is best when transitive for
aggregation but not for indirection:

struct S { int a; int *b; }

void foo(scope S s)
{
 static int *x1 = &(s.a); // error
 static int *x2 = s.b; // fine
}


(Both of these are in error because static variables are initialized at 
compile time, but I get what you mean.)


Why would 'scope' have any effect on the validity of the first? s.a 
belongs to the callee exclusively. Adding modular annotations to prevent 
taking its address has no point.

Re: DIP 36: Rvalue References


On Tuesday, 23 April 2013 at 08:19:43 UTC, Timon Gehr wrote:

It is disallowed in @system code.


Ye, I know and always took it as yet another "we care mostly for 
@safe" issue. What is the real rationale for this restriction?

Re: DIP 36: Rvalue References


On Tuesday, 23 April 2013 at 08:25:18 UTC, Timon Gehr wrote:
(Both of these are in error because static variables are 
initialized at compile time, but I get what you mean.)


Shame on me! (blush)

Why would 'scope' have any effect on the validity of the first? 
s.a belongs to the callee exclusively. Adding modular 
annotations to prevent taking its address has no point.


Because it is an attempt to take/store address of a scope 
variable, which is prohibited by scope definition. I think (but I 
am not sure!) that it makes sense to make scope transitive for 
aggregation, because automatic destruction upon leaving the scope 
happens for aggregated entities. Thus, nothing can be known about 
s.a lifetime, same as s lifetime. This is one step towards 
consistent "scope" behavior that will allow safety desired by 
this DIP.

Re: DIP 36: Rvalue References

2013-04-23 Thread Andrei Alexandrescu


On 4/23/13 4:08 AM, Dicebot wrote:

You consider "scope" a new language feature?


Yes.


And consider "auto ref"
improvable solution.


Yes.


The very point of this DIP is to not create
syntax-driven features, instead better define existing ones that make
sense on their own so they can be used for same purpose.


It's a new feature, no two ways about it. It overlaps ref and auto ref 
without any palpable benefit and defines yet another way to achieve the 
same thing as auto ref. On this ground alone the proposal has a large 
problem.



2. The proposal is sketchy and does not give many details, such as the
lifetime of temporaries bound to scope ref objects.


It can't because lifetime of temporaries is not defined in D at all and
suck stuff needs to be consistent. It is not really different from a
lifetime of struct literal temporary used for pass-by-value.


A proposal aimed at binding rvalues to references must address lifetime 
of temporaries as a central concern.



3. The relationship with auto ref is insufficiently described, e.g.
there should be clarification on why auto ref cannot be improved to
fulfill the desired role.


auto ref is a template-world entity. If by "improved" you mean
"completely reworked" than sure, I can add this rationale. Will do today.


I think we should focus on 
http://d.puremagic.com/issues/show_bug.cgi?id=9238 and on making ref safe.



4. Above all this is a new language feature and again we want to
resort to adding new feature only if it is clear that the existing
features are insufficient and cannot be made sufficient. In particular
we are much more inclined to impart real, demonstrable safety to "ref"
and to make "auto ref" work as a reference that can bind to rvalues as
well as lvalues.


Can't agree. "scope" is defined (but not implemented, a pity). Creating
temporaries already exists for some cases. No really new language
feature is added. No special syntax created. "scope ref" is still
"scope" and "ref" with all semantics this imposes, rvalue references are
just a side-effect. Contrary to this, "auto ref" IS a new feature
created specifically for syntax hack. That is a very problem you are
speaking about - introducing new concepts instead of making use of ones
that exist for ages.


You are of course to disagree, but that would make you wrong.


Andrei

Re: DIP 36: Rvalue References