Re: Multiple return values

[Sean Kelly]

It's easy enough with Tuple, though better language support would be nice. 

Sent from my iPhone

On Jan 3, 2012, at 2:40 PM, Manu  wrote:

> Why doesn't D support multiple return values like many other modern languages?
> 
> Clearly the same syntax as Go wouldn't work, but I'm sure a neat and tidy 
> syntax could be invented?
> I constantly want to be able to return x,y from a function, or 
> retVal,errorCode and I want the language to make some rough ABI guarantees, 
> like multiple return values will be returned in consecutive registers, rather 
> than a single return value register like C/C++, avoiding the need to pass 
> output addresses through ref function parameters (slow!).

Re: Multiple return values

[Manu]

Does returning a tuple give any ABI guarantees? How can I be sure multiple
return values will return in consecutive registers?
What if the return types are of different types, a float and an int... can
I expect each to return in their own register types respectively?
This needs to be defined and loosely guaranteed (within reason) so people
can expect multiple return values to behave as expected on any architecture.

On 4 January 2012 01:02, Sean Kelly  wrote:

> It's easy enough with Tuple, though better language support would be nice.
>
> Sent from my iPhone
>
> On Jan 3, 2012, at 2:40 PM, Manu  wrote:
>
> > Why doesn't D support multiple return values like many other modern
> languages?
> >
> > Clearly the same syntax as Go wouldn't work, but I'm sure a neat and
> tidy syntax could be invented?
> > I constantly want to be able to return x,y from a function, or
> retVal,errorCode and I want the language to make some rough ABI guarantees,
> like multiple return values will be returned in consecutive registers,
> rather than a single return value register like C/C++, avoiding the need to
> pass output addresses through ref function parameters (slow!).
>

Re: Multiple return values

[Sean Kelly]

Tuple is a struct, so it's returned according to those rules. 

Sent from my iPhone

On Jan 3, 2012, at 3:09 PM, Manu  wrote:

> Does returning a tuple give any ABI guarantees? How can I be sure multiple 
> return values will return in consecutive registers?
> What if the return types are of different types, a float and an int... can I 
> expect each to return in their own register types respectively?
> This needs to be defined and loosely guaranteed (within reason) so people can 
> expect multiple return values to behave as expected on any architecture.
> 
> On 4 January 2012 01:02, Sean Kelly  wrote:
> It's easy enough with Tuple, though better language support would be nice.
> 
> Sent from my iPhone
> 
> On Jan 3, 2012, at 2:40 PM, Manu  wrote:
> 
> > Why doesn't D support multiple return values like many other modern 
> > languages?
> >
> > Clearly the same syntax as Go wouldn't work, but I'm sure a neat and tidy 
> > syntax could be invented?
> > I constantly want to be able to return x,y from a function, or 
> > retVal,errorCode and I want the language to make some rough ABI guarantees, 
> > like multiple return values will be returned in consecutive registers, 
> > rather than a single return value register like C/C++, avoiding the need to 
> > pass output addresses through ref function parameters (slow!).
> 

Re: Multiple return values

[Mail Mantis]

2012/1/4 Manu 

> Does returning a tuple give any ABI guarantees? How can I be sure multiple
> return values will return in consecutive registers?
> What if the return types are of different types, a float and an int... can
> I expect each to return in their own register types respectively?
> This needs to be defined and loosely guaranteed (within reason) so people
> can expect multiple return values to behave as expected on any architecture.
>
>
> On 4 January 2012 01:02, Sean Kelly  wrote:
>
>> It's easy enough with Tuple, though better language support would be nice.
>>
>> Sent from my iPhone
>>
>> On Jan 3, 2012, at 2:40 PM, Manu  wrote:
>>
>> > Why doesn't D support multiple return values like many other modern
>> languages?
>> >
>> > Clearly the same syntax as Go wouldn't work, but I'm sure a neat and
>> tidy syntax could be invented?
>> > I constantly want to be able to return x,y from a function, or
>> retVal,errorCode and I want the language to make some rough ABI guarantees,
>> like multiple return values will be returned in consecutive registers,
>> rather than a single return value register like C/C++, avoiding the need to
>> pass output addresses through ref function parameters (slow!).
>>
>
> Returning multiple values in registers might not always be the best idea
from the performance point of view. The caller must make sure the
appropriate registers don't hold any intermediate values before calling the
function, so overall complier optimization may be poorer.
Of course, this only applies when function is actually called. When it is
inlined, having multiple return values could result in better optimization.

Re: Multiple return values

[Manu]

And those rules aren't really type aware... as far as I can tell it wont
split the int/float/etc args into separate register types...
Multiple return values via tuples are not really addressing the problem,
and miss the point.

On 4 January 2012 01:13, Sean Kelly  wrote:

> Tuple is a struct, so it's returned according to those rules.
>
> Sent from my iPhone
>
> On Jan 3, 2012, at 3:09 PM, Manu  wrote:
>
> Does returning a tuple give any ABI guarantees? How can I be sure multiple
> return values will return in consecutive registers?
> What if the return types are of different types, a float and an int... can
> I expect each to return in their own register types respectively?
> This needs to be defined and loosely guaranteed (within reason) so people
> can expect multiple return values to behave as expected on any architecture.
>
> On 4 January 2012 01:02, Sean Kelly  wrote:
>
>> It's easy enough with Tuple, though better language support would be nice.
>>
>> Sent from my iPhone
>>
>> On Jan 3, 2012, at 2:40 PM, Manu  wrote:
>>
>> > Why doesn't D support multiple return values like many other modern
>> languages?
>> >
>> > Clearly the same syntax as Go wouldn't work, but I'm sure a neat and
>> tidy syntax could be invented?
>> > I constantly want to be able to return x,y from a function, or
>> retVal,errorCode and I want the language to make some rough ABI guarantees,
>> like multiple return values will be returned in consecutive registers,
>> rather than a single return value register like C/C++, avoiding the need to
>> pass output addresses through ref function parameters (slow!).
>>
>
>

Re: Multiple return values

[Timon Gehr]

On 01/04/2012 12:09 AM, Manu wrote:
Does returning a tuple give any ABI guarantees?  How can I be sure multiple return values will return in consecutive 

registers?

  * 1, 2 and 4 byte structs are returned in EAX.
  * 8 byte structs are returned in EDX,EAX, where EDX gets the most 
significant half.
  * For other struct sizes, the return value is stored through a hidden 
pointer passed as an argument to the function.





What if the return types are of different types, a float and an int...
can I expect each to return in their own register types respectively?


No.


This needs to be defined and loosely guaranteed (within reason) so
people can expect multiple return values to behave as expected on any
architecture.


A possibility would be to allow TypeTuple return types.




On 4 January 2012 01:02, Sean Kelly mailto:s...@invisibleduck.org>> wrote:

It's easy enough with Tuple, though better language support would be
nice.

Sent from my iPhone

On Jan 3, 2012, at 2:40 PM, Manu mailto:turkey...@gmail.com>> wrote:

 > Why doesn't D support multiple return values like many other
modern languages?
 >
 > Clearly the same syntax as Go wouldn't work, but I'm sure a neat
and tidy syntax could be invented?
 > I constantly want to be able to return x,y from a function, or
retVal,errorCode and I want the language to make some rough ABI
guarantees, like multiple return values will be returned in
consecutive registers, rather than a single return value register
like C/C++, avoiding the need to pass output addresses through ref
function parameters (slow!).

Re: Multiple return values

[Manu]

When a function is inlined, virtually everything regarding parameters
passing is irrelevant, the compiler can usually do what's best in this case.

In the former case, it's extremely rare that the calling function making
the register available for the return value could negatively affect
performance.. if it wants the returned value (that was alternatively
written to the stack), it needs to allocate a register for it anyway, why
not the one that allows it to return directly. Additionally, if the
function receives any args, then the registers for return values are
probably already allocated to the function call to pass the args when
calling.

On 4 January 2012 01:17, Mail Mantis  wrote:

> 2012/1/4 Manu 
>
>> Does returning a tuple give any ABI guarantees? How can I be sure
>> multiple return values will return in consecutive registers?
>> What if the return types are of different types, a float and an int...
>> can I expect each to return in their own register types respectively?
>> This needs to be defined and loosely guaranteed (within reason) so people
>> can expect multiple return values to behave as expected on any architecture.
>>
>>
>> On 4 January 2012 01:02, Sean Kelly  wrote:
>>
>>> It's easy enough with Tuple, though better language support would be
>>> nice.
>>>
>>> Sent from my iPhone
>>>
>>> On Jan 3, 2012, at 2:40 PM, Manu  wrote:
>>>
>>> > Why doesn't D support multiple return values like many other modern
>>> languages?
>>> >
>>> > Clearly the same syntax as Go wouldn't work, but I'm sure a neat and
>>> tidy syntax could be invented?
>>> > I constantly want to be able to return x,y from a function, or
>>> retVal,errorCode and I want the language to make some rough ABI guarantees,
>>> like multiple return values will be returned in consecutive registers,
>>> rather than a single return value register like C/C++, avoiding the need to
>>> pass output addresses through ref function parameters (slow!).
>>>
>>
>> Returning multiple values in registers might not always be the best idea
> from the performance point of view. The caller must make sure the
> appropriate registers don't hold any intermediate values before calling the
> function, so overall complier optimization may be poorer.
> Of course, this only applies when function is actually called. When it is
> inlined, having multiple return values could result in better optimization.
>

Re: Multiple return values

[Manu]

This is precisely my understanding, and the reason I post the topic.
Tuples are not a reasonable solution.

On 4 January 2012 01:17, Timon Gehr  wrote:

> On 01/04/2012 12:09 AM, Manu wrote:
>
>> Does returning a tuple give any ABI guarantees?  How can I be sure
>> multiple return values will return in consecutive
>>
> registers?
>
>  * 1, 2 and 4 byte structs are returned in EAX.
>  * 8 byte structs are returned in EDX,EAX, where EDX gets the most
> significant half.
>  * For other struct sizes, the return value is stored through a hidden
> pointer passed as an argument to the function.
>
>
>
>
>  What if the return types are of different types, a float and an int...
>> can I expect each to return in their own register types respectively?
>>
>
> No.
>
>
>  This needs to be defined and loosely guaranteed (within reason) so
>> people can expect multiple return values to behave as expected on any
>> architecture.
>>
>
> A possibility would be to allow TypeTuple return types.
>
>
>
>> On 4 January 2012 01:02, Sean Kelly > > wrote:
>>
>>It's easy enough with Tuple, though better language support would be
>>nice.
>>
>>Sent from my iPhone
>>
>>On Jan 3, 2012, at 2:40 PM, Manu >> wrote:
>>
>> > Why doesn't D support multiple return values like many other
>>modern languages?
>> >
>> > Clearly the same syntax as Go wouldn't work, but I'm sure a neat
>>and tidy syntax could be invented?
>> > I constantly want to be able to return x,y from a function, or
>>retVal,errorCode and I want the language to make some rough ABI
>>guarantees, like multiple return values will be returned in
>>consecutive registers, rather than a single return value register
>>like C/C++, avoiding the need to pass output addresses through ref
>>function parameters (slow!).
>>
>>
>>
>

Re: Multiple return values

[Jonathan M Davis]

On Wednesday, January 04, 2012 00:40:13 Manu wrote:
> Why doesn't D support multiple return values like many other modern
> languages?

I'm not saying that being able to return multiple values wouldn't be nice, but 
in my experience, that's actually a very rare thing for languages to be able 
to do - especially among C-based languages. Regardless, std.typecons.Tuple is 
D's solution.

- Jonathan M Davis

Re: Multiple return values

[bearophile]

Jonathan M Davis:

> I'm not saying that being able to return multiple values wouldn't be nice, 
> but 
> in my experience, that's actually a very rare thing for languages to be able 
> to do - especially among C-based languages.

How do you count how rare is a feature need in languages that don't support 
that certain feature? I think you have no proof of what you are saying.
On the contrary, the only evidence I see (looking at lot of real code written 
by other people), with languages that do offer multiple return values, like 
Python, Go and others, show that people do use this feature very often and 
appreciate it.


> Regardless, std.typecons.Tuple is D's solution.

Maybe, but we'll need better unpacking syntax at the calling/use point:

auto (x, y) = foo();
foreach (auto (x, y); genPairs()) {

There is a pull request, plus a good amount of discussions, on this topic. 

Bye,
bearophile

Re: Multiple return values

[Nick Sabalausky]

"Manu"  wrote in message 
news:mailman.2114.1325632682.24802.digitalmar...@puremagic.com...
> And those rules aren't really type aware... as far as I can tell it wont
> split the int/float/etc args into separate register types...
> Multiple return values via tuples are not really addressing the problem,
> and miss the point.
>

I realize this probably isn't quite the answer you want, but I bet you could 
probably create custom calling conventions in D via naked inline asm and 
metaprogramming. That could be an interesting exercise for anyone who wanted 
to try it.

Re: Multiple return values

[Jonathan M Davis]

On Tuesday, January 03, 2012 19:00:05 bearophile wrote:
> Jonathan M Davis:
> > I'm not saying that being able to return multiple values wouldn't be
> > nice, but in my experience, that's actually a very rare thing for
> > languages to be able to do - especially among C-based languages.
> 
> How do you count how rare is a feature need in languages that don't support
> that certain feature? I think you have no proof of what you are saying. On
> the contrary, the only evidence I see (looking at lot of real code written
> by other people), with languages that do offer multiple return values, like
> Python, Go and others, show that people do use this feature very often and
> appreciate it.

I'm not saying anything about the value of the feature. I'm just saying that 
given D's lineage, it's not at all suprising that it doesn't support multiple 
return values. In fact, it would be very abnormal if it did. The OP seems to 
expect that modern languages would support such a feature, whereas many do not 
- particularly the C-based ones.

Whether it would be worth having is an entirely different issue.

- Jonathan M Davis

Re: Multiple return values

[bearophile]

Jonathan M Davis:

> I'm not saying anything about the value of the feature. I'm just saying that 
> given D's lineage, it's not at all suprising that it doesn't support multiple 
> return values. In fact, it would be very abnormal if it did. The OP seems to 
> expect that modern languages would support such a feature, whereas many do 
> not 
> - particularly the C-based ones.

OK.
For fun here is a slightly modified version of your text:

"I'm not saying anything about the value of the feature. I'm just saying that 
given D's lineage, it's not at all suprising that it doesn't support nested 
functions. In fact, it would be very abnormal if it did. The OP seems to expect 
that modern languages would support such a feature, whereas many do not - 
particularly the C-based ones."

:-)

Bye,
bearophile

Re: Multiple return values

[Jonathan M Davis]

On Tuesday, January 03, 2012 19:48:53 bearophile wrote:
> Jonathan M Davis:
> > I'm not saying anything about the value of the feature. I'm just saying
> > that given D's lineage, it's not at all suprising that it doesn't
> > support multiple return values. In fact, it would be very abnormal if
> > it did. The OP seems to expect that modern languages would support such
> > a feature, whereas many do not - particularly the C-based ones.
> 
> OK.
> For fun here is a slightly modified version of your text:
> 
> "I'm not saying anything about the value of the feature. I'm just saying
> that given D's lineage, it's not at all suprising that it doesn't support
> nested functions. In fact, it would be very abnormal if it did. The OP
> seems to expect that modern languages would support such a feature, whereas
> many do not - particularly the C-based ones."
> :-)

Well, yeah. Having nested functions in C-based languages is not normal, and I 
don't believe that most other "modern" languages have them either, so it's 
arguably unreasonable to _expect_ that D would have nested functions, because 
it's a modern language. That doesn't mean that it _won't_ have nested 
functions (or multiple return values), but _expecting_ that it will simply 
because it's a modern programming language is arguably unreasonable.

- Jonathan M Davis

Re: Multiple return values

[Andrew Wiley]

On Tue, Jan 3, 2012 at 6:48 PM, bearophile  wrote:
> Jonathan M Davis:
>
>> I'm not saying anything about the value of the feature. I'm just saying that
>> given D's lineage, it's not at all suprising that it doesn't support multiple
>> return values. In fact, it would be very abnormal if it did. The OP seems to
>> expect that modern languages would support such a feature, whereas many do 
>> not
>> - particularly the C-based ones.
>
> OK.
> For fun here is a slightly modified version of your text:
>
> "I'm not saying anything about the value of the feature. I'm just saying that 
> given D's lineage, it's not at all suprising that it doesn't support nested 
> functions. In fact, it would be very abnormal if it did. The OP seems to 
> expect that modern languages would support such a feature, whereas many do 
> not - particularly the C-based ones."
>

While that's true, it's completely tangential to the argument at hand.

Re: Multiple return values

[Manu]

I only base my expectation on the fact that D is an open source language
maintained by a community that comes from many other areas of programming,
has a tendency to respond to user feedback, and generally seems to do it's
best to cherry pick the best bits from virtually every other language I can
think of. I'm just surprised this isn't constantly requested, and I see no
harm in adding it.
I'm sure an unambiguous syntax can be invented (although I don't like the
current proposal using EVEN MORE parentheses)... D has a bit of an
obsession with parenthesis... ;)

Regardless, I'd like to put a strong +1 towards proper multiple return
values. The point is NOT just to have the syntactical feature (ie. struct
return value semantics via tuples). The point is a language guarantee that
the architecture will do its best job to return each value in consecutive
registers OF ITS OWN TYPE, avoiding allocation&writing to the stack. This
is a common problem in C where the only solution is to make functions
inline or pay the cost of hitting the stack (LHS hazard, very expensive on
many RISC architectures).
It would lead to many optimisations in lots of code in my experience.

On 4 January 2012 05:46, Jonathan M Davis  wrote:

> On Tuesday, January 03, 2012 19:48:53 bearophile wrote:
> > Jonathan M Davis:
> > > I'm not saying anything about the value of the feature. I'm just saying
> > > that given D's lineage, it's not at all suprising that it doesn't
> > > support multiple return values. In fact, it would be very abnormal if
> > > it did. The OP seems to expect that modern languages would support such
> > > a feature, whereas many do not - particularly the C-based ones.
> >
> > OK.
> > For fun here is a slightly modified version of your text:
> >
> > "I'm not saying anything about the value of the feature. I'm just saying
> > that given D's lineage, it's not at all suprising that it doesn't support
> > nested functions. In fact, it would be very abnormal if it did. The OP
> > seems to expect that modern languages would support such a feature,
> whereas
> > many do not - particularly the C-based ones."
> > :-)
>
> Well, yeah. Having nested functions in C-based languages is not normal,
> and I
> don't believe that most other "modern" languages have them either, so it's
> arguably unreasonable to _expect_ that D would have nested functions,
> because
> it's a modern language. That doesn't mean that it _won't_ have nested
> functions (or multiple return values), but _expecting_ that it will simply
> because it's a modern programming language is arguably unreasonable.
>
> - Jonathan M Davis
>

Re: Multiple return values

[bearophile]

deadalnix:

> I do agree with Andrei. Those are 2 orthogonals problems. The 
> optimization problem is clearly something that can be improved in D ABI. 
> Then, returning a tuple could be improved using thoses rules.

I agree.


> The multiple retur value problem is syntaxic sugar. It may be 
> interesting, but not #1 priority.

I don't agree. In this moment of the D development it's still more important to 
design D well than to squeeze every bit of performance out of the reference 
implementation.

Example: currently vector operations have some small syntax (and maybe 
semantic) problem, plus performance problems. I think fixing their syntax is 
currently more important.

Fixing performance problems is possible to do later in GDC/LDC if the language 
design is good, but fixing the syntax later is quite harder to do.

Bye,
bearophile

Re: Multiple return values

[Manu]

>
> > The multiple retur value problem is syntaxic sugar. It may be
> > interesting, but not #1 priority.
>
> I don't agree. In this moment of the D development it's still more
> important to design D well than to squeeze every bit of performance out of
> the reference implementation.
>
> Example: currently vector operations have some small syntax (and maybe
> semantic) problem, plus performance problems. I think fixing their syntax
> is currently more important.
>
> Fixing performance problems is possible to do later in GDC/LDC if the
> language design is good, but fixing the syntax later is quite harder to do.
>

I'm not clear what side of the fence you sit... I raise this issue because
I feel fixing multiple return values IS a syntactic problem. If the syntax
was good, ie. capable of expressing what you actually want from multiple
return values, but the code gen was not, I wouldn't care.. The problem is D
has no way to express this concept efficiently. That seems like an
immediate syntax problem.

It's the same with __forceinline, or __restrict... I don't really care if
they work right now, but the fact that they're missing from the language
spec is a worry, since programmers will need them.

Re: Multiple return values

[deadalnix]

Le 04/01/2012 15:23, Andrei Alexandrescu a écrit :

On 1/4/12 2:36 AM, Manu wrote:

Regardless, I'd like to put a strong +1 towards proper multiple return
values. The point is NOT just to have the syntactical feature (ie.
struct return value semantics via tuples). The point is a language
guarantee that the architecture will do its best job to return each
value in consecutive registers OF ITS OWN TYPE, avoiding
allocation&writing to the stack. This is a common problem in C where the
only solution is to make functions inline or pay the cost of hitting the
stack (LHS hazard, very expensive on many RISC architectures).
It would lead to many optimisations in lots of code in my experience.


Why make those optimizations only for multiple return values and not for
every single returned structure?

Andrei


I do agree with Andrei. Those are 2 orthogonals problems. The 
optimization problem is clearly something that can be improved in D ABI. 
Then, returning a tuple could be improved using thoses rules.


The multiple retur value problem is syntaxic sugar. It may be 
interesting, but not #1 priority.

Re: Multiple return values

[Andrei Alexandrescu]

On 1/4/12 2:36 AM, Manu wrote:

Regardless, I'd like to put a strong +1 towards proper multiple return
values. The point is NOT just to have the syntactical feature (ie.
struct return value semantics via tuples). The point is a language
guarantee that the architecture will do its best job to return each
value in consecutive registers OF ITS OWN TYPE, avoiding
allocation&writing to the stack. This is a common problem in C where the
only solution is to make functions inline or pay the cost of hitting the
stack (LHS hazard, very expensive on many RISC architectures).
It would lead to many optimisations in lots of code in my experience.


Why make those optimizations only for multiple return values and not for 
every single returned structure?


Andrei

Re: Multiple return values

[Manu]

On 4 January 2012 16:23, Andrei Alexandrescu
wrote:

> On 1/4/12 2:36 AM, Manu wrote:
>
>> Regardless, I'd like to put a strong +1 towards proper multiple return
>> values. The point is NOT just to have the syntactical feature (ie.
>> struct return value semantics via tuples). The point is a language
>> guarantee that the architecture will do its best job to return each
>> value in consecutive registers OF ITS OWN TYPE, avoiding
>> allocation&writing to the stack. This is a common problem in C where the
>> only solution is to make functions inline or pay the cost of hitting the
>> stack (LHS hazard, very expensive on many RISC architectures).
>> It would lead to many optimisations in lots of code in my experience.
>>
>
> Why make those optimizations only for multiple return values and not for
> every single returned structure?


I think multiple return values and returning a struct (tuple) are
fundamentally different things...
Multiple return values are just that, and it implies nothing about the
arrangement/'structure' of those values, or how/where they are stored..
they are simply returned in their respective register types.

A struct (tuple) on the other hand must be expected that it has some
'structure', the layout is known, addressable, and you can perform memory
manipulation on it.
I can't imagine any situation where returning a struct could NOT be
required to reserve stack space for the struct in the appropriate layout,
and results written, even if they were also returned in appropriate
registers...
Implementing multiple return values seems far simpler and more elegant, not
to mention, more predictable and intuitive in terms of ABI and register
usage. Surely it would be easier/faster to implement, and I'd argue the
result would be more 'proper'...

Re: Multiple return values

[Manu]

On 5 January 2012 00:22, Manu  wrote:

> > The multiple retur value problem is syntaxic sugar. It may be
>> > interesting, but not #1 priority.
>>
>> I don't agree. In this moment of the D development it's still more
>> important to design D well than to squeeze every bit of performance out of
>> the reference implementation.
>>
>> Example: currently vector operations have some small syntax (and maybe
>> semantic) problem, plus performance problems. I think fixing their syntax
>> is currently more important.
>>
>> Fixing performance problems is possible to do later in GDC/LDC if the
>> language design is good, but fixing the syntax later is quite harder to do.
>>
>
> I'm not clear what side of the fence you sit... I raise this issue because
> I feel fixing multiple return values IS a syntactic problem. If the syntax
> was good, ie. capable of expressing what you actually want from multiple
> return values, but the code gen was not, I wouldn't care.. The problem is D
> has no way to express this concept efficiently. That seems like an
> immediate syntax problem.
>
> It's the same with __forceinline, or __restrict... I don't really care if
> they work right now, but the fact that they're missing from the language
> spec is a worry, since programmers will need them.
>

*** bearophile: sorry, for some reason I didn't receive Andrei or
deadalnix's emails before getting yours. It's clear what you're saying, and
I completely agree.

Re: Multiple return values

[F i L]

bearophile wrote:

deadalnix:

I do agree with Andrei. Those are 2 orthogonals problems. The 
optimization problem is clearly something that can be improved 
in D ABI. Then, returning a tuple could be improved using 
thoses rules.


I agree.


The multiple retur value problem is syntaxic sugar. It may be 
interesting, but not #1 priority.


I don't agree. In this moment of the D development it's still 
more important to design D well than to squeeze every bit of 
performance out of the reference implementation.


Example: currently vector operations have some small syntax 
(and maybe semantic) problem, plus performance problems. I 
think fixing their syntax is currently more important.


Fixing performance problems is possible to do later in GDC/LDC 
if the language design is good, but fixing the syntax later is 
quite harder to do.


Bye,
bearophile


+1 to this. Performance is key to D's success of course, but what 
will attract more developers is clean and bug free aesthetics. 
DMD already outperforms MS/Mono C# in all of my modest 
benchmarks. GDC and LDC much more so.


The first two things that really made me want to use D where 
reading about it's built-in Arrays and Unittests. Multiple return 
syntax could only make D feel more hip. A complaint I've heard 
before is how C-like D tries to be with it's library naming (eg, 
std.writeln vs Console.WriteLine), so I don't think tradition or 
heritage are good ways to increase support. While D's naming 
seems natural enough to me or anyone who's written C/C++ code, 
from an outside perspective you might wounder why there was 
module dealing with venereal disease.


I know you D developers, have a lot on your plates. Thank you all.

Re: Multiple return values

[Jacob Carlborg]

On 2012-01-05 00:31, F i L wrote:

bearophile wrote:

deadalnix:


I do agree with Andrei. Those are 2 orthogonals problems. The
optimization problem is clearly something that can be improved in D
ABI. Then, returning a tuple could be improved using thoses rules.


I agree.



The multiple retur value problem is syntaxic sugar. It may be
interesting, but not #1 priority.


I don't agree. In this moment of the D development it's still more
important to design D well than to squeeze every bit of performance
out of the reference implementation.

Example: currently vector operations have some small syntax (and maybe
semantic) problem, plus performance problems. I think fixing their
syntax is currently more important.

Fixing performance problems is possible to do later in GDC/LDC if the
language design is good, but fixing the syntax later is quite harder
to do.

Bye,
bearophile


+1 to this. Performance is key to D's success of course, but what will
attract more developers is clean and bug free aesthetics. DMD already
outperforms MS/Mono C# in all of my modest benchmarks. GDC and LDC much
more so.

The first two things that really made me want to use D where reading
about it's built-in Arrays and Unittests. Multiple return syntax could
only make D feel more hip. A complaint I've heard before is how C-like D
tries to be with it's library naming (eg, std.writeln vs
Console.WriteLine), so I don't think tradition or heritage are good ways
to increase support. While D's naming seems natural enough to me or
anyone who's written C/C++ code, from an outside perspective you might
wounder why there was module dealing with venereal disease.

I know you D developers, have a lot on your plates. Thank you all.



C#, Java and similar language doesn't support free functions like D 
does. Then they have to resort to "hacks" like static methods, i.e. 
Console.WriteLine.


--
/Jacob Carlborg

Re: Multiple return values

[F i L]

Jacob Carlborg wrote:
C#, Java and similar language doesn't support free functions 
like D does. Then they have to resort to "hacks" like static 
methods, i.e. Console.WriteLine.


D's free functions and simple module design are definitely points 
in it's favor. However that wasn't really my point. I was saying 
that from an non-C/C++/D perspective, "Console.WriteLine()" makes 
more immediate sense than "std.writeln()" albeit only marginally. 
Both Java and C# use a less cryptic naming convention in their 
standard libraries which I think helps when learning a language. 
For instance, "std.container" makes me think of a cup, while 
"System.Collections" makes me think of a bundle. It's probably 
just me, but for awhile I thought D didn't have a standard linked 
list structure because I kept looking in std.array for it.

Re: Multiple return values

[Andrew Wiley]

On Thu, Jan 5, 2012 at 2:47 AM, F i L  wrote:
> Jacob Carlborg wrote:
>>
>> C#, Java and similar language doesn't support free functions like D does.
>> Then they have to resort to "hacks" like static methods, i.e.
>> Console.WriteLine.
>
>
> D's free functions and simple module design are definitely points in it's
> favor. However that wasn't really my point. I was saying that from an
> non-C/C++/D perspective, "Console.WriteLine()" makes more immediate sense
> than "std.writeln()" albeit only marginally. Both Java and C# use a less
> cryptic naming convention in their standard libraries which I think helps
> when learning a language. For instance, "std.container" makes me think of a
> cup, while "System.Collections" makes me think of a bundle. It's probably
> just me, but for awhile I thought D didn't have a standard linked list
> structure because I kept looking in std.array for it.
>

You're pointing out two completely different things here. C#
PascalCases methods and D camelCases them. That's a completely
seperate issue from the actual names.
std.writeln() is incorrect, you would just use writeln(), which looks
perfectly sane if you're used to seeing free functions. It will look
strange initially if you're from a Java/C# background, but it's a free
function because conceptually it *should* be a free function. If you
want to force yourself to qualify your code with namespaces, use
static imports and write std.stdio.writeln().
As for the names, you'll find they relate more to C and C++ libraries
than to Java and C#, which I would argue is a good thing because the
design ideals are closer to C and C++.

Re: Multiple return values

[F i L]

Andrew Wiley wrote:

You're pointing out two completely different things here. C#
PascalCases methods and D camelCases them. That's a completely
seperate issue from the actual names.


I think you're reading too much into what I wrote, I wasn't 
talking about the case or modules vs static methods. I'm simply 
using C#'s libraries as an example of what's easier to understand 
from and basic English perspective. I could have wrote: "import 
system.console : writeLine" or "import system.io : writeLine" the 
point is those are less cryptic (initially) than "import 
std.stdio : writeln", etc...


I'm perfectly fine with how D's libraries are worded personally, 
and my points here are very minor.



As for the names, you'll find they relate more to C and C++ 
libraries
than to Java and C#, which I would argue is a good thing 
because the design ideals are closer to C and C++.


Sure, I've written some C++ programs in the past so I get the 
nostalgia, but first impression make a difference and not every 
potential D programmer is going to have a background in C. Nor 
should they need to, and that's my only argument here.

Re: Multiple return values...

[H. S. Teoh]

On Thu, Mar 08, 2012 at 10:08:41PM +0200, Manu wrote:
> I find myself really wishing for proper multiple return values almost every
> day, particularly when I work with maths heavy code, and especially for
> efficiently returning error codes in functions I'd rather not throw from.
> Many maths-y functions return some sort of pair; intersections return (ray,
> t) or something of that type.
> I'm finding HEAPS of SIMD functions want to return pairs (unpacks in
> particular): int4 (low, hight) = unpack(someShort8);
> Currently I have to duplicate everyting: int4 low =
> unpackLow(someShort8); int4 high = unpackHigh(someShort8);
> I'm getting really sick of that, it feels so... last millennium.
> 
> The point of 'proper' multiple return values is to return each value in
> registers, in its own register type, using exactly the same register
> assignment pattern as when passing args TO functions.
> I don't think this causes any side effects to the ABI, since the arg
> registers are already volatile across function calls in the first place.
> It just means that the returned-to function can find its return
> values already conveniently in an appropriate register, avoiding memory
> access.
> 
> People argue I should return a tuple, but this isn't really the same, it
> has hidden implications that complicate the optimisation potential. For
> instance, tuples have an implicit structure/memory layout which can't be
> ignored, whereas return values don't have memory allocated, ie, you can't
> take the address of a return value without first assigning it to some local
> syntactically.
[...]

Couldn't you just use ref parameters? Or does the compiler turn them
into pointers?


T

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula: pi zz a. -- Wouter Verhelst

Re: Multiple return values...

[Manu]

On 8 March 2012 22:13, H. S. Teoh  wrote:

> On Thu, Mar 08, 2012 at 10:08:41PM +0200, Manu wrote:
> > I find myself really wishing for proper multiple return values almost
> every
> > day, particularly when I work with maths heavy code, and especially for
> > efficiently returning error codes in functions I'd rather not throw from.
> > Many maths-y functions return some sort of pair; intersections return
> (ray,
> > t) or something of that type.
> > I'm finding HEAPS of SIMD functions want to return pairs (unpacks in
> > particular): int4 (low, hight) = unpack(someShort8);
> > Currently I have to duplicate everyting: int4 low =
> > unpackLow(someShort8); int4 high = unpackHigh(someShort8);
> > I'm getting really sick of that, it feels so... last millennium.
> >
> > The point of 'proper' multiple return values is to return each value in
> > registers, in its own register type, using exactly the same register
> > assignment pattern as when passing args TO functions.
> > I don't think this causes any side effects to the ABI, since the arg
> > registers are already volatile across function calls in the first place.
> > It just means that the returned-to function can find its return
> > values already conveniently in an appropriate register, avoiding memory
> > access.
> >
> > People argue I should return a tuple, but this isn't really the same, it
> > has hidden implications that complicate the optimisation potential. For
> > instance, tuples have an implicit structure/memory layout which can't be
> > ignored, whereas return values don't have memory allocated, ie, you can't
> > take the address of a return value without first assigning it to some
> local
> > syntactically.
> [...]
>
> Couldn't you just use ref parameters? Or does the compiler turn them
> into pointers?
>

ref parameters are pointers by definition. That's not what I'm talking
about.

Re: Multiple return values...

[Mantis]

08.03.2012 22:08, Manu пишет:
I find myself really wishing for proper multiple return values almost 
every day, particularly when I work with maths heavy code, and 
especially for efficiently returning error codes in functions I'd 
rather not throw from.
Many maths-y functions return some sort of pair; intersections return 
(ray, t) or something of that type.
I'm finding HEAPS of SIMD functions want to return pairs (unpacks in 
particular): int4 (low, hight) = unpack(someShort8);
Currently I have to duplicate everyting: int4 low = 
unpackLow(someShort8); int4 high = unpackHigh(someShort8);

I'm getting really sick of that, it feels so... last millennium.

The point of 'proper' multiple return values is to return each value 
in registers, in its own register type, using exactly the same 
register assignment pattern as when passing args TO functions.
I don't think this causes any side effects to the ABI, since the arg 
registers are already volatile across function calls in the first place.
It just means that the returned-to function can find its return 
values already conveniently in an appropriate register, avoiding 
memory access.


People argue I should return a tuple, but this isn't really the same, 
it has hidden implications that complicate the optimisation potential. 
For instance, tuples have an implicit structure/memory layout which 
can't be ignored, whereas return values don't have memory allocated, 
ie, you can't take the address of a return value without first 
assigning it to some local syntactically.
The implementation of efficient tuple return values would be much more 
complicated I would imagine too, and the rules are less clear; I can't 
intuitively presume what behaviour returning a tuple of different 
things should actually have in terms of register assignment. I also 
know precisely how multiple return values should work, because it is 
exactly the same as passing arguments to the function, but in reverse.


... just saying :)


I'd like to see this as a part of tuple improvement, since tuples may 
hold additional compile-time information. This would make it possible to 
write efficient code, and at the same time have the ability to reference 
tuple components by name rather than position. For example:

{
auto t = getSomeTuple(...); // returns Tuple!(float, "x", float, 
"y"), no actual assignment is made
someVar = t.x^^2 + t.y^^2; // t.x and t.y are just aliases for st1 
and st0

}

With a tuple unpacking syntax suggested by bearophile, this would be 
nicer than any distinct multiple return values syntax, IMO.

Re: Multiple return values...

[Manu]

On 9 March 2012 00:29, Mantis  wrote:

> 08.03.2012 22:08, Manu пишет:
>
>  I find myself really wishing for proper multiple return values almost
>> every day, particularly when I work with maths heavy code, and especially
>> for efficiently returning error codes in functions I'd rather not throw
>> from.
>> Many maths-y functions return some sort of pair; intersections return
>> (ray, t) or something of that type.
>> I'm finding HEAPS of SIMD functions want to return pairs (unpacks in
>> particular): int4 (low, hight) = unpack(someShort8);
>> Currently I have to duplicate everyting: int4 low =
>> unpackLow(someShort8); int4 high = unpackHigh(someShort8);
>> I'm getting really sick of that, it feels so... last millennium.
>>
>> The point of 'proper' multiple return values is to return each value in
>> registers, in its own register type, using exactly the same register
>> assignment pattern as when passing args TO functions.
>> I don't think this causes any side effects to the ABI, since the arg
>> registers are already volatile across function calls in the first place.
>> It just means that the returned-to function can find its return values
>> already conveniently in an appropriate register, avoiding memory access.
>>
>> People argue I should return a tuple, but this isn't really the same, it
>> has hidden implications that complicate the optimisation potential. For
>> instance, tuples have an implicit structure/memory layout which can't be
>> ignored, whereas return values don't have memory allocated, ie, you can't
>> take the address of a return value without first assigning it to some local
>> syntactically.
>> The implementation of efficient tuple return values would be much more
>> complicated I would imagine too, and the rules are less clear; I can't
>> intuitively presume what behaviour returning a tuple of different things
>> should actually have in terms of register assignment. I also know precisely
>> how multiple return values should work, because it is exactly the same as
>> passing arguments to the function, but in reverse.
>>
>> ... just saying :)
>>
>
> I'd like to see this as a part of tuple improvement, since tuples may hold
> additional compile-time information. This would make it possible to write
> efficient code, and at the same time have the ability to reference tuple
> components by name rather than position. For example:
> {
>auto t = getSomeTuple(...); // returns Tuple!(float, "x", float, "y"),
> no actual assignment is made
>someVar = t.x^^2 + t.y^^2; // t.x and t.y are just aliases for st1 and
> st0
> }
>
> With a tuple unpacking syntax suggested by bearophile, this would be nicer
> than any distinct multiple return values syntax, IMO.
>

The problem is, that approach feels like a double negative to me. A tuple
is fundamentally a structure, returned by value. Implementing hacks to
subvert the standard behaviour of returning a structure by value is
unintuitive for a start, and you also lose the ability to *actually* return
a structure by value should that be what you intend.
You're sacrificing a well defined, 'properly' implemented mechanic
to imitate something the language simply can't express. I just think that's
the wrong way to go about it.

Let me try and make my case as I see it...

These are 2 distinct concepts, returning multiple values, and returning a
struct by value.

Returning a structure by value is currently well defined, and behaves as
any same programmer would expect; it is written to the stack with memory
layout according to the STRUCTURE. This is true for a tuple, and it works
as one expects. I have no issue here. If you return a tuple, you SHOULD be
able to take the pointer of the first item, perform some arithmetic, and
address some other struct member. It is a struct, it ought to behave like
one, precisely as any programmer will expect.

By contrast, multiple return values are quite the opposite. They are
explicitly NON-STRUCTURED. These serve a totally different purpose; to
return multiple unstructured things from a function.
Imagine an inline function which returns 2 results, only one of which is
captured. It is easy and intuitive to eliminate the code path leading to
the ignored result. Not so simple if you're returning structured data,
because it could be indirectly addressed.
In this case, using an explicit syntax to perform this specific task
doesn't suffer from the implicit problems associated with subverting the
structure syntax (what to do about memory layout/pointer arithmetic?
reserve stack space and generate code to store implicitly? ick!), but it
also clearly states the programmers intent, and also clearly communicates a
presumed behaviour. The presumption in this case is that multiple return
values would follow the exact same set of rules as passing multiple args TO
a function, but in reverse.

Both operations seem useful and important in their own ways, they are also
both distinct operations, and they both warrant an expression in the

Re: Multiple return values...

[Manu]

On 9 March 2012 00:29, Mantis  wrote:

> 08.03.2012 22:08, Manu пишет:
>
>  I find myself really wishing for proper multiple return values almost
>> every day, particularly when I work with maths heavy code, and especially
>> for efficiently returning error codes in functions I'd rather not throw
>> from.
>> Many maths-y functions return some sort of pair; intersections return
>> (ray, t) or something of that type.
>> I'm finding HEAPS of SIMD functions want to return pairs (unpacks in
>> particular): int4 (low, hight) = unpack(someShort8);
>> Currently I have to duplicate everyting: int4 low =
>> unpackLow(someShort8); int4 high = unpackHigh(someShort8);
>> I'm getting really sick of that, it feels so... last millennium.
>>
>> The point of 'proper' multiple return values is to return each value in
>> registers, in its own register type, using exactly the same register
>> assignment pattern as when passing args TO functions.
>> I don't think this causes any side effects to the ABI, since the arg
>> registers are already volatile across function calls in the first place.
>> It just means that the returned-to function can find its return values
>> already conveniently in an appropriate register, avoiding memory access.
>>
>> People argue I should return a tuple, but this isn't really the same, it
>> has hidden implications that complicate the optimisation potential. For
>> instance, tuples have an implicit structure/memory layout which can't be
>> ignored, whereas return values don't have memory allocated, ie, you can't
>> take the address of a return value without first assigning it to some local
>> syntactically.
>> The implementation of efficient tuple return values would be much more
>> complicated I would imagine too, and the rules are less clear; I can't
>> intuitively presume what behaviour returning a tuple of different things
>> should actually have in terms of register assignment. I also know precisely
>> how multiple return values should work, because it is exactly the same as
>> passing arguments to the function, but in reverse.
>>
>> ... just saying :)
>>
>
> I'd like to see this as a part of tuple improvement, since tuples may hold
> additional compile-time information. This would make it possible to write
> efficient code, and at the same time have the ability to reference tuple
> components by name rather than position. For example:
> {
>auto t = getSomeTuple(...); // returns Tuple!(float, "x", float, "y"),
> no actual assignment is made
>someVar = t.x^^2 + t.y^^2; // t.x and t.y are just aliases for st1 and
> st0
> }
>
> With a tuple unpacking syntax suggested by bearophile, this would be nicer
> than any distinct multiple return values syntax, IMO.
>

One more little detail too, a Tuple is just a standard template.
Are the suggestion that the LANGUAGE definite case sensitive behavioural
tweaks, relying on the definition of a specific library defined template?

The only way I can imagine such an implementation actually working in a
well defined way, is if the concept were applied to all structures-by-value
universally, which leads straight back to the rant in my prior post...

I wonder if the details of the implementation would be considerably harder
to implement given this approach too (it sounds way loaded with issues to
me), and particularly so for GDC?

Re: Multiple return values...

[Andrej Mitrovic]

I can't talk about multiple return values, but D allows you to do some
pretty crazy things. The following might not be very efficient, but it
does show the language is quite flexible. E.g.:
http://ideone.com/qy8i3

Re: Multiple return values...

[Mantis]

09.03.2012 1:28, Manu пишет:

[...]
The problem is, that approach feels like a double negative to me. A 
tuple is fundamentally a structure, returned by value. Implementing 
hacks to subvert the standard behaviour of returning a structure by 
value is unintuitive for a start, and you also lose the ability to 
*actually* return a structure by value should that be what you intend.
You're sacrificing a well defined, 'properly' implemented mechanic 
to imitate something the language simply can't express. I just think 
that's the wrong way to go about it.


Let me try and make my case as I see it...

These are 2 distinct concepts, returning multiple values, and 
returning a struct by value.


Returning a structure by value is currently well defined, and behaves 
as any same programmer would expect; it is written to the stack with 
memory layout according to the STRUCTURE. This is true for a tuple, 
and it works as one expects. I have no issue here. If you return a 
tuple, you SHOULD be able to take the pointer of the first item, 
perform some arithmetic, and address some other struct member. It is a 
struct, it ought to behave like one, precisely as any programmer will 
expect.


By contrast, multiple return values are quite the opposite. They are 
explicitly NON-STRUCTURED. These serve a totally different purpose; to 
return multiple unstructured things from a function.
Imagine an inline function which returns 2 results, only one of which 
is captured. It is easy and intuitive to eliminate the code path 
leading to the ignored result. Not so simple if you're returning 
structured data, because it could be indirectly addressed.
In this case, using an explicit syntax to perform this specific task 
doesn't suffer from the implicit problems associated with subverting 
the structure syntax (what to do about memory layout/pointer 
arithmetic? reserve stack space and generate code to store implicitly? 
ick!), but it also clearly states the programmers intent, and also 
clearly communicates a presumed behaviour. The presumption in this 
case is that multiple return values would follow the exact same set of 
rules as passing multiple args TO a function, but in reverse.


Both operations seem useful and important in their own ways, they are 
also both distinct operations, and they both warrant an expression in 
the language. Returning a tuple if perfect how it is, it is just not 
what I want to do in cases like those I list in my OP.


How is any programmer supposed to intuitively assume that returning a 
tuple by value would behave in that way? And how are you supposed to 
trust it? It's an abuse of concept and syntax. It seems like a 
convolution that could only possibly confuse people, they are 
conceptually quite different things, and shouldn't be lumped into the 
same syntax for my money.


Is tuple required to be anonymous struct? I thought it's implementation 
details that may be done the other way if tuples implemented in language 
rather then library. There's another problem with non-named return 
values, as this:

auto (sin_a, cos_a) = sincos( a );
is not equivalent to this:
auto (cos_a, sin_a) = sincos( a );
Here it doesn't matter much, but if you are returning more variables, 
that may become confusing and bloated, and will provoke juniors to make 
errors, which is obviously not good for your money =). Note also, since 
variables should be defined /before/ function call, your IDE will not 
help you to avoid such mistakes.


Another thing is that if you may 'save' packed variables for later use, 
such code can be made possible:

auto t = getSomeTuple(...);
someVar = t.var;
foreach( v; t[1..$] ) { // static foreach, gets unrolled for every value 
in t except for the first

sum += v;
}
Not for everyday use, but sometimes may be useful.

Anyway, what's your suggestion for the syntax?

Re: Multiple return values...

[Manu]

On 9 March 2012 01:56, Mantis  wrote:

> 09.03.2012 1:28, Manu пишет:
>
>> [...]
>>
>> The problem is, that approach feels like a double negative to me. A tuple
>> is fundamentally a structure, returned by value. Implementing hacks to
>> subvert the standard behaviour of returning a structure by value is
>> unintuitive for a start, and you also lose the ability to *actually* return
>> a structure by value should that be what you intend.
>> You're sacrificing a well defined, 'properly' implemented mechanic to
>> imitate something the language simply can't express. I just think that's
>> the wrong way to go about it.
>>
>> Let me try and make my case as I see it...
>>
>> These are 2 distinct concepts, returning multiple values, and returning a
>> struct by value.
>>
>> Returning a structure by value is currently well defined, and behaves as
>> any same programmer would expect; it is written to the stack with memory
>> layout according to the STRUCTURE. This is true for a tuple, and it works
>> as one expects. I have no issue here. If you return a tuple, you SHOULD be
>> able to take the pointer of the first item, perform some arithmetic, and
>> address some other struct member. It is a struct, it ought to behave like
>> one, precisely as any programmer will expect.
>>
>> By contrast, multiple return values are quite the opposite. They are
>> explicitly NON-STRUCTURED. These serve a totally different purpose; to
>> return multiple unstructured things from a function.
>> Imagine an inline function which returns 2 results, only one of which is
>> captured. It is easy and intuitive to eliminate the code path leading to
>> the ignored result. Not so simple if you're returning structured data,
>> because it could be indirectly addressed.
>> In this case, using an explicit syntax to perform this specific task
>> doesn't suffer from the implicit problems associated with subverting the
>> structure syntax (what to do about memory layout/pointer arithmetic?
>> reserve stack space and generate code to store implicitly? ick!), but it
>> also clearly states the programmers intent, and also clearly communicates a
>> presumed behaviour. The presumption in this case is that multiple return
>> values would follow the exact same set of rules as passing multiple args TO
>> a function, but in reverse.
>>
>> Both operations seem useful and important in their own ways, they are
>> also both distinct operations, and they both warrant an expression in the
>> language. Returning a tuple if perfect how it is, it is just not what I
>> want to do in cases like those I list in my OP.
>>
>> How is any programmer supposed to intuitively assume that returning a
>> tuple by value would behave in that way? And how are you supposed to trust
>> it? It's an abuse of concept and syntax. It seems like a convolution that
>> could only possibly confuse people, they are conceptually quite different
>> things, and shouldn't be lumped into the same syntax for my money.
>>
>
> Is tuple required to be anonymous struct? I thought it's implementation
> details that may be done the other way if tuples implemented in language
> rather then library. There's another problem with non-named return values,
> as this:
> auto (sin_a, cos_a) = sincos( a );
> is not equivalent to this:
> auto (cos_a, sin_a) = sincos( a );
>

I can't imagine a syntax that's non-destructive to the existing grammar
where order is not important, but order would be clearly stated in the
auto-complete pop-up, and in the reference. Also mismatching types would
throw errors.

Here it doesn't matter much, but if you are returning more variables, that
> may become confusing and bloated, and will provoke juniors to make errors,
> which is obviously not good for your money =). Note also, since variables
> should be defined /before/ function call, your IDE will not help you to
> avoid such mistakes.
>

If you feel so strongly, state a company policy banning the feature. My
company does that with things like stl, sort the problem right out :)
I don't think that's a reasonable argument to intentionally make the
language unable to express a very useful software concept.

Another thing is that if you may 'save' packed variables for later use,
> such code can be made possible:
> auto t = getSomeTuple(...);
> someVar = t.var;
> foreach( v; t[1..$] ) { // static foreach, gets unrolled for every value
> in t except for the first
>sum += v;
> }
> Not for everyday use, but sometimes may be useful.
>

If I want to do this, I will do exactly this! :)

Anyway, what's your suggestion for the syntax?
>

I can imagine syntax using parentheses, but I don't think I'm qualified to
propose a robust syntax, I don't know enough about the finer details of the
grammar.
Perhaps if other people agree with me, they could present some creative
solutions to the syntax?

I imagine something like:
auto (x, y) = func(); // specify auto for all results?
float (x, y) = func(); // specify explicit type for all results?
(int x

Re: Multiple return values...

[bearophile]

Manu:
7
> How is any programmer supposed to intuitively assume that returning a tuple
> by value would behave in that way? And how are you supposed to trust it?
> It's an abuse of concept and syntax. It seems like a convolution that could
> only possibly confuse people, they are conceptually quite different things,
> and shouldn't be lumped into the same syntax for my money.

I am asking for multiple return values in D since some years, so I share your 
desire. But keep in mind that in D currently there are Phobos Tuples and DMD 
typetuples. Adding a third type of tuple-like things in a single language 
sounds a bit excessive. Two kinds of tuples in a language are already too much, 
in my opinion.

Bye,
bearophile

Re: Multiple return values...

[Manu]

On 9 March 2012 02:26, bearophile  wrote:

> Manu:
> 7
> > How is any programmer supposed to intuitively assume that returning a
> tuple
> > by value would behave in that way? And how are you supposed to trust it?
> > It's an abuse of concept and syntax. It seems like a convolution that
> could
> > only possibly confuse people, they are conceptually quite different
> things,
> > and shouldn't be lumped into the same syntax for my money.
>
> I am asking for multiple return values in D since some years, so I share
> your desire. But keep in mind that in D currently there are Phobos Tuples
> and DMD typetuples. Adding a third type of tuple-like things in a single
> language sounds a bit excessive. Two kinds of tuples in a language are
> already too much, in my opinion.
>

This is most certainly NOT a tuple-like thing. I don't think I can stress
that point any harder :)
It is not an object, you can't evaluate it as a whole, you can't iterate
it, these values are not connected in ANY way. These are the properties
which make the whole concept simpler, and intuitively map to the same
concept as passing multiple args TO a function, but in reverse. No tricks,
completely intuitive and predictable.

You aren't expected to construct an arg list tuple every time you want to
CALL a function, you just line up a sequence of values you get from
wherever. Multiple return value assignment should be just as random and
flexible; eg, one item stored to memory, one retained as a local (kept in a
register with no memory expression), another may be ignored (stripping
associated code path)...

Re: Multiple return values...

[Mantis]

09.03.2012 2:23, Manu пишет:
On 9 March 2012 01:56, Mantis > wrote:


[...]
Is tuple required to be anonymous struct? I thought it's
implementation details that may be done the other way if tuples
implemented in language rather then library. There's another
problem with non-named return values, as this:
auto (sin_a, cos_a) = sincos( a );
is not equivalent to this:
auto (cos_a, sin_a) = sincos( a );


I can't imagine a syntax that's non-destructive to the existing 
grammar where order is not important, but order would be clearly 
stated in the auto-complete pop-up, and in the reference. Also 
mismatching types would throw errors.


You can't mess with the parameters order in this case: 
http://dl.dropbox.com/u/36715190/Images/par_order.jpg
Is it impossible to make efficient multiple return values without the 
need to trade off the help from tools?

Re: Multiple return values...

[bearophile]

Manu:

> This is most certainly NOT a tuple-like thing. I don't think I can stress
> that point any harder :)

I understand. But in other languages tuples are used for such purpose, so there 
is some risk people will think of them as a third kind of tuples, despite they 
are a different thing.

What kind of syntax do you suggest?

Bye,
bearophile

Re: Multiple return values...

[Manu]

On 9 March 2012 02:59, Mantis  wrote:

> 09.03.2012 2:23, Manu пишет:
>
>> On 9 March 2012 01:56, Mantis > mail.mantis.88@gmail.**com >> wrote:
>>
>>[...]
>>
>>Is tuple required to be anonymous struct? I thought it's
>>implementation details that may be done the other way if tuples
>>implemented in language rather then library. There's another
>>problem with non-named return values, as this:
>>auto (sin_a, cos_a) = sincos( a );
>>is not equivalent to this:
>>auto (cos_a, sin_a) = sincos( a );
>>
>>
>> I can't imagine a syntax that's non-destructive to the existing grammar
>> where order is not important, but order would be clearly stated in the
>> auto-complete pop-up, and in the reference. Also mismatching types would
>> throw errors.
>>
>
> You can't mess with the parameters order in this case:
> http://dl.dropbox.com/u/**36715190/Images/par_order.jpg
> Is it impossible to make efficient multiple return values without the need
> to trade off the help from tools?
>

Eh? How so?

Visual studio show's the return value and function name in the same tooltip
(not just the argument list as your image shows). Tools may need to tweak
their popup if they don't present the return values already.
Also, tools can do the usual red-squiggly underline thing on return
assignments with mismatching result count or types easily.


On 9 March 2012 03:24, bearophile  wrote:

> I understand. But in other languages tuples are used for such purpose, so
> there is some risk people will think of them as a third kind of tuples,
> despite they are a different thing.
>

I really don't think so. The only language I know of that behaves like you
say is lua (oh, and python?); scripting languages, not too interested with
ABI expression, and bear virtually no relation whatsoever to any c-like
language.
I think the comparison might be drawn to something more like Go. Octave
does it with an interesting syntax, but these are not compatible with D in
a non-breaking way.

What kind of syntax do you suggest?
>

I detailed my initial idea in my post prior to yours, but perhaps other
more interesting syntax might also exist:

I imagine something like:
> auto (x, y) = func(); // specify auto for all results?
> float (x, y) = func(); // specify explicit type for all results?
> (int x, float y) = func; // explicitly type each result?
> int x; ... (x, float y) = func(); // assign to predeclared variable(/s)?
> (x, , z) = func(); // ignore the second result value, intuitive and
> visible (elimination of the second result's code path)
>
> I'm sure other more bizarre syntax could be possible too to help reduce
> bracket spam. Ideas from things like lambda syntax?
>

Re: Multiple return values...

[Timon Gehr]

On 03/09/2012 01:23 AM, Manu wrote:

I can imagine syntax using parentheses, but I don't think I'm qualified
to propose a robust syntax, I don't know enough about the finer details
of the grammar.
Perhaps if other people agree with me, they could present some creative
solutions to the syntax?

I imagine something like:
auto (x, y) = func(); // specify auto for all results?
float (x, y) = func(); // specify explicit type for all results?
(int x, float y) = func; // explicitly type each result?


This works, and Kenji Hara has already implemented appropriate parser 
extensions.



int x; ... (x, float y) = func(); // assign to predeclared variable(/s)?
(x, , z) = func(); // ignore the second result value (elimination of the
second result's code path)



Those two would work, but (x,y) = func(); conflicts with the comma 
operator. (I'd prefer (,) to be a tuple constructor though.)

Re: Multiple return values...

[H. S. Teoh]

On Fri, Mar 09, 2012 at 03:27:14PM +0100, Timon Gehr wrote:
> On 03/09/2012 01:23 AM, Manu wrote:
[...]
> >int x; ... (x, float y) = func(); // assign to predeclared variable(/s)?
> >(x, , z) = func(); // ignore the second result value (elimination of the
> >second result's code path)
> >
> 
> Those two would work, but (x,y) = func(); conflicts with the comma
> operator. (I'd prefer (,) to be a tuple constructor though.)

Just out of curiosity, *why* does D have a comma operator? It's one of
those obscure things about C that can be really, really, nasty if you're
unaware of it. And C++ makes it worse by making the comma operator
*overloadable*.


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:

Re: Multiple return values...

[Manu]

On 9 March 2012 16:27, Timon Gehr  wrote:

> On 03/09/2012 01:23 AM, Manu wrote:
>
>> I can imagine syntax using parentheses, but I don't think I'm qualified
>> to propose a robust syntax, I don't know enough about the finer details
>> of the grammar.
>> Perhaps if other people agree with me, they could present some creative
>> solutions to the syntax?
>>
>> I imagine something like:
>> auto (x, y) = func(); // specify auto for all results?
>> float (x, y) = func(); // specify explicit type for all results?
>> (int x, float y) = func; // explicitly type each result?
>>
>
> This works, and Kenji Hara has already implemented appropriate parser
> extensions.
>
>  int x; ... (x, float y) = func(); // assign to predeclared variable(/s)?
>> (x, , z) = func(); // ignore the second result value (elimination of the
>>
>> second result's code path)
>>
>>
> Those two would work, but (x,y) = func(); conflicts with the comma
> operator. (I'd prefer (,) to be a tuple constructor though.)
>

You think so? Within that context, I would think the coma could be
reinterpreted however it likes. The usual use of the coma operator makes no
sense in this context?

These last 2 examples are what I see as being the most important part, and
the precise reason that it SHOULDN'T be a tuple. The ability to directly
assign results to explicit (existing) variables, and to ignore some/all of
the return values, is a fundamental feature of the *concept* of return
values universally. I see this as basically the whole point.
Another example: (someStruct.x, y, , int err) = func();
In this example, I assign the x result to a struct, y assigns to some
existing local, we ignore z because we can (visually states our intent,
would be hidden through use of a tuple), and we declare an int to capture a
potential error in place.
If we were abusing the tuple syntax, we would need additional lines
following the call to assign the rvalues out to their appropriate places,
which is unnecessary spaghetti.

Re: Multiple return values...

[Kevin Cox]

On Mar 9, 2012 10:28 AM, "H. S. Teoh"  wrote:
>
> On Fri, Mar 09, 2012 at 03:27:14PM +0100, Timon Gehr wrote:
> > On 03/09/2012 01:23 AM, Manu wrote:
> [...]
> > >int x; ... (x, float y) = func(); // assign to predeclared
variable(/s)?
> > >(x, , z) = func(); // ignore the second result value (elimination of
the
> > >second result's code path)
> > >
> >
> > Those two would work, but (x,y) = func(); conflicts with the comma
> > operator. (I'd prefer (,) to be a tuple constructor though.)
>
> Just out of curiosity, *why* does D have a comma operator? It's one of
> those obscure things about C that can be really, really, nasty if you're
> unaware of it. And C++ makes it worse by making the comma operator
> *overloadable

The comma operator can be worked around by using braces instead.  Of course
it is very breaking and quite arguably ugly.

for ({int I; float j;} ; ) ;

But then you have to worry about what it returns.  The sequence operator is
actually useful it is just unfortunate that they used it for function
parameters also.

I think that the best work around is either the braces which can be quite
elegant because it fits the rest of the language and say it returns the
return value of the last statement.  Or pick a new character for the
sequence operator.

This is actually kinds nice because you get "tuple" in "tuple" out.  Of
course they don't need to be tuples that can be passed around they can be
values that must be unpacked right away.

Re: Multiple return values...

[Timon Gehr]

On 03/09/2012 04:38 PM, Manu wrote:

On 9 March 2012 16:27, Timon Gehr mailto:timon.g...@gmx.ch>> wrote:

On 03/09/2012 01:23 AM, Manu wrote:

I can imagine syntax using parentheses, but I don't think I'm
qualified
to propose a robust syntax, I don't know enough about the finer
details
of the grammar.
Perhaps if other people agree with me, they could present some
creative
solutions to the syntax?

I imagine something like:
auto (x, y) = func(); // specify auto for all results?
float (x, y) = func(); // specify explicit type for all results?
(int x, float y) = func; // explicitly type each result?


This works, and Kenji Hara has already implemented appropriate
parser extensions.

int x; ... (x, float y) = func(); // assign to predeclared
variable(/s)?
(x, , z) = func(); // ignore the second result value
(elimination of the

second result's code path)


Those two would work, but (x,y) = func(); conflicts with the comma
operator. (I'd prefer (,) to be a tuple constructor though.)


You think so? Within that context, I would think the coma could be
reinterpreted however it likes. The usual use of the coma operator makes
no sense in this context?


void main(){
int a,b;
(a,b)=2;
assert(a==0);
assert(b==2);
}



These last 2 examples are what I see as being the most important part,
and the precise reason that it SHOULDN'T be a tuple.


You are probably confusing the tuple concept with a Phobos Tuple.


The ability to
directly assign results to explicit (existing) variables, and to ignore
some/all of the return values, is a fundamental feature of the /concept/
of return values universally.
I see this as basically the whole point.
Another example: (someStruct.x, y, , int err) = func();
In this example, I assign the x result to a struct, y assigns to some
existing local, we ignore z because we can (visually states our intent,
would be hidden through use of a tuple), and we declare an int to
capture a potential error in place.


This is simple pattern matching.


If we were abusing the tuple syntax, we would need additional lines
following the call to assign the rvalues out to their appropriate
places, which is unnecessary spaghetti.


What you propose is tuple syntax.

Re: Multiple return values...

[Manu]

On 9 March 2012 17:57, Timon Gehr  wrote:

> On 03/09/2012 04:38 PM, Manu wrote:
>
>> On 9 March 2012 16:27, Timon Gehr > > wrote:
>>
>>On 03/09/2012 01:23 AM, Manu wrote:
>>
>>I can imagine syntax using parentheses, but I don't think I'm
>>qualified
>>to propose a robust syntax, I don't know enough about the finer
>>details
>>of the grammar.
>>Perhaps if other people agree with me, they could present some
>>creative
>>solutions to the syntax?
>>
>>I imagine something like:
>>auto (x, y) = func(); // specify auto for all results?
>>float (x, y) = func(); // specify explicit type for all results?
>>(int x, float y) = func; // explicitly type each result?
>>
>>
>>This works, and Kenji Hara has already implemented appropriate
>>parser extensions.
>>
>>int x; ... (x, float y) = func(); // assign to predeclared
>>variable(/s)?
>>(x, , z) = func(); // ignore the second result value
>>(elimination of the
>>
>>second result's code path)
>>
>>
>>Those two would work, but (x,y) = func(); conflicts with the comma
>>operator. (I'd prefer (,) to be a tuple constructor though.)
>>
>>
>> You think so? Within that context, I would think the coma could be
>> reinterpreted however it likes. The usual use of the coma operator makes
>> no sense in this context?
>>
>
> void main(){
>int a,b;
>(a,b)=2;
>assert(a==0);
>assert(b==2);
>
> }
>
>
>> These last 2 examples are what I see as being the most important part,
>> and the precise reason that it SHOULDN'T be a tuple.
>>
>
> You are probably confusing the tuple concept with a Phobos Tuple.
>
>  The ability to
>> directly assign results to explicit (existing) variables, and to ignore
>> some/all of the return values, is a fundamental feature of the /concept/
>>
>> of return values universally.
>> I see this as basically the whole point.
>> Another example: (someStruct.x, y, , int err) = func();
>> In this example, I assign the x result to a struct, y assigns to some
>> existing local, we ignore z because we can (visually states our intent,
>> would be hidden through use of a tuple), and we declare an int to
>> capture a potential error in place.
>>
>
> This is simple pattern matching.


I'm not sure what you mean by this?


> If we were abusing the tuple syntax, we would need additional lines
>> following the call to assign the rvalues out to their appropriate
>> places, which is unnecessary spaghetti.
>>
>
> What you propose is tuple syntax.
>

What I mean is this:

retTuple = func();
someStruct.x = retTuple[0];
y = retTuple[1];
// retTuple[2] is ignored, but the intent is not clear in the code as it
was in my prior example, I like how my prior example makes this intent
explicit
int err = retTuple[3];

This is pretty horrible. Surely you can see why I want to be able to
arbitrarily assign the return values directly?
That's what I mean by 'abuse of the tuple syntax', but if that's not what
you mean, then show me an example of the usage of your suggestion?

Re: Multiple return values...

[Timon Gehr]

On 03/09/2012 05:14 PM, Manu wrote:

On 9 March 2012 17:57, Timon Gehr mailto:timon.g...@gmx.ch>> wrote:

On 03/09/2012 04:38 PM, Manu wrote:

On 9 March 2012 16:27, Timon Gehr mailto:timon.g...@gmx.ch>
>> wrote:

On 03/09/2012 01:23 AM, Manu wrote:

I can imagine syntax using parentheses, but I don't
think I'm
qualified
to propose a robust syntax, I don't know enough about
the finer
details
of the grammar.
Perhaps if other people agree with me, they could
present some
creative
solutions to the syntax?

I imagine something like:
auto (x, y) = func(); // specify auto for all results?
float (x, y) = func(); // specify explicit type for all
results?
(int x, float y) = func; // explicitly type each result?


This works, and Kenji Hara has already implemented appropriate
parser extensions.

int x; ... (x, float y) = func(); // assign to predeclared
variable(/s)?
(x, , z) = func(); // ignore the second result value
(elimination of the

second result's code path)


Those two would work, but (x,y) = func(); conflicts with the
comma
operator. (I'd prefer (,) to be a tuple constructor though.)


You think so? Within that context, I would think the coma could be
reinterpreted however it likes. The usual use of the coma
operator makes
no sense in this context?


void main(){
int a,b;
(a,b)=2;
assert(a==0);
assert(b==2);

}


These last 2 examples are what I see as being the most important
part,
and the precise reason that it SHOULDN'T be a tuple.


You are probably confusing the tuple concept with a Phobos Tuple.

The ability to
directly assign results to explicit (existing) variables, and to
ignore
some/all of the return values, is a fundamental feature of the
/concept/

of return values universally.
I see this as basically the whole point.
Another example: (someStruct.x, y, , int err) = func();
In this example, I assign the x result to a struct, y assigns to
some
existing local, we ignore z because we can (visually states our
intent,
would be hidden through use of a tuple), and we declare an int to
capture a potential error in place.


This is simple pattern matching.


I'm not sure what you mean by this?

If we were abusing the tuple syntax, we would need additional lines
following the call to assign the rvalues out to their appropriate
places, which is unnecessary spaghetti.


What you propose is tuple syntax.


What I mean is this:

retTuple = func();
someStruct.x = retTuple[0];
y = retTuple[1];
// retTuple[2] is ignored, but the intent is not clear in the code as it
was in my prior example, I like how my prior example makes this intent
explicit
int err = retTuple[3];

This is pretty horrible. Surely you can see why I want to be able to
arbitrarily assign the return values directly?
That's what I mean by 'abuse of the tuple syntax', but if that's not
what you mean, then show me an example of the usage of your suggestion?


There are two parts, syntax and semantics.

Semantics:
D is already able to express those:

template Tuple(T...){alias T Tuple;} // not the same as std.typecons.Tuple!

// function with multiple return values:
Tuple!(int,double) foo(int a, double b){
Tuple!(int, double) result; // ok, _no imposed memory layout_
result[0] = a;  // ok
result[1] = a+b;  // ok
return result;
}

Multiple return values are currently *disallowed explicitly*:
DMD sez: "Error: functions cannot return a tuple"

Just specify the ABI, implement the code gen, and we're done.

Moot point: built-in tuples auto-flatten inside comma-separated lists.

std.typecons.Tuple is a hack to circumvent the arbitrary "cannot return 
tuple from function" restriction as well as the auto-flattening. The 
problem is that it is implemented as a struct with a built-in tuple 
member. The fact that it is a struct imposes a memory layout. This is 
just a side-effect of attempting to address the other two issues. It is 
not something that is desirable.



Syntax:
Currently, there is just none. Tuples are a built-in types that cannot 
be created without a template that makes them accessible.


IMHO Ideally, it would look like this:


(int, double) foo(int a, double b) => (a, a+b);//Jonathan does not like this

void main(){
(int, double) bar = (1,2.0);
auto (x,y) = (1,2.0);
bar = foo(x,y);
(x,_) = foo(bar); // ignore y,

Re: Multiple return values...

[Timon Gehr]

On 03/09/2012 05:14 PM, Manu wrote:

What I mean is this:

retTuple = func();
someStruct.x = retTuple[0];
y = retTuple[1];
// retTuple[2] is ignored, but the intent is not clear in the code as it
was in my prior example, I like how my prior example makes this intent
explicit
int err = retTuple[3];

This is pretty horrible. Surely you can see why I want to be able to
arbitrarily assign the return values directly?


Yes. I want to be able to do that too.


That's what I mean by 'abuse of the tuple syntax',


Ah ok. I misunderstood then.

Re: Multiple return values...

[H. S. Teoh]

On Fri, Mar 09, 2012 at 07:16:19PM +0100, Timon Gehr wrote:
[...]
> Another issue is that people would complain about auto-flattening all
> the time once built-in tuples get more accessible, even though it is
> not actually a problem. It would be just due to the fact that it does
> not occur in most other popular programming languages.
[...]

Perl auto-flattens lists. To prevent auto-flattening you have to
explicitly take an array reference. But yeah, most other languages
don't.


T

-- 
Real Programmers use "cat > a.out".

Re: Multiple return values...

[bearophile]

H. S. Teoh:

> Perl auto-flattens lists.

Maybe they have fixed this big design mistake in Perl6.

Bye,
bearophile

Re: Multiple return values...

[H. S. Teoh]

On Fri, Mar 09, 2012 at 02:01:34PM -0500, bearophile wrote:
> H. S. Teoh:
> 
> > Perl auto-flattens lists.
> 
> Maybe they have fixed this big design mistake in Perl6.
[...]

Whether or not it was a mistake is debatable. It does have its uses...
though forcing everyone to use references to prevent flattening *is* a
bit extreme.


T

-- 
"A man's wife has more power over him than the state has." -- Ralph Emerson

Re: Multiple return values...

[Manfred Nowak]

Manu wrote:


> By contrast, multiple return values are quite the opposite. They
> are explicitly NON-STRUCTURED.

> The presumption in this case is that multiple resturn 
> values would follow the exact same set of rules as passing
> multiple args TO a function, but in reverse.

this seams to state a contradiction, because one cannot use the 
"multiple return values" as actual parameters for any call of a 
function.

Furthermore it is not defined how the formal parameters of a function 
are to be declared, if they are assumed to receive "multiple return 
values" as actual arguments. If those actual parameters are indeed NON-
STRUCTURED how can one define them or even mix them with STRUCTURED 
actual parameters?

-manfred

Re: Multiple return values...

[Manu]

On 9 March 2012 22:39, Manfred Nowak  wrote:

> Manu wrote:
>
>
> > By contrast, multiple return values are quite the opposite. They
> > are explicitly NON-STRUCTURED.
>
> > The presumption in this case is that multiple resturn
> > values would follow the exact same set of rules as passing
> > multiple args TO a function, but in reverse.
>
> this seams to state a contradiction, because one cannot use the
> "multiple return values" as actual parameters for any call of a
> function.
>
> Furthermore it is not defined how the formal parameters of a function
> are to be declared, if they are assumed to receive "multiple return
> values" as actual arguments. If those actual parameters are indeed NON-
> STRUCTURED how can one define them or even mix them with STRUCTURED
> actual parameters?
>

I'm just talking about the ABI for returning multiple values, not chaining
a multiple return into a secondary call (I think this is what you are
saying?).
A multiple return should use precisely the same register/stack assignment
as when passing args in to a function. Just that the process is in reverse.

Re: Multiple return values...

[a]

I'm finding HEAPS of SIMD functions want to return pairs 
(unpacks in

particular): int4 (low, hight) = unpack(someShort8);
Currently I have to duplicate everyting: int4 low =
unpackLow(someShort8); int4 high = unpackHigh(someShort8);
I'm getting really sick of that, it feels so... last millennium.


It can also be realy inefficient. For example ARM NEON has vzip 
instruction that is used like this:


vzip.32 q0, q1

This will interleave elements of vectors in q0 and q1 in one 
instruction.

Re: Multiple return values...

[Manfred Nowak]

Manu wrote:

> I'm just talking about the ABI for returning multiple values, not
> chaining 

Does this mean, that you want a special type of "function"? For example  
this would be disallowed statement: `auto result= f( g( parameters));', 
if `f' and `g' are functions returning multiple values?

Under "chaining" I understand concatenation of actual parameter lists 
like `auto result= f( parameters1)(parameters2);', which are legal 
because `f( parameters1)' returns a function `g', for which 
`parameters2' is a valid actual parameterlist.

What about recursion?

-manfred

Re: Multiple return values...

[Timon Gehr]

On 03/10/2012 12:02 AM, Manfred Nowak wrote:

Manu wrote:


I'm just talking about the ABI for returning multiple values, not
chaining


Does this mean, that you want a special type of "function"? For example
this would be disallowed statement: `auto result= f( g( parameters));',
if `f' and `g' are functions returning multiple values?



This needs to work. Multiple return values should integrate with the 
existing language support for tuples.

Re: Multiple return values...

[Andrei Alexandrescu]

On 3/9/12 10:16 AM, Timon Gehr wrote:

There are two parts, syntax and semantics.

Semantics:
D is already able to express those:

template Tuple(T...){alias T Tuple;} // not the same as std.typecons.Tuple!

// function with multiple return values:
Tuple!(int,double) foo(int a, double b){
Tuple!(int, double) result; // ok, _no imposed memory layout_
result[0] = a; // ok
result[1] = a+b; // ok
return result;
}

Multiple return values are currently *disallowed explicitly*:
DMD sez: "Error: functions cannot return a tuple"

Just specify the ABI, implement the code gen, and we're done.

Moot point: built-in tuples auto-flatten inside comma-separated lists.

std.typecons.Tuple is a hack to circumvent the arbitrary "cannot return
tuple from function" restriction as well as the auto-flattening.


No, it was more like an implementation of a generic tuple.


The
problem is that it is implemented as a struct with a built-in tuple
member. The fact that it is a struct imposes a memory layout. This is
just a side-effect of attempting to address the other two issues. It is
not something that is desirable.


Why?

I don't understand what you're trying to solve, that Tuple is not good at.


Andrei

Re: Multiple return values...

[Timon Gehr]

On 03/10/2012 12:27 AM, Andrei Alexandrescu wrote:

On 3/9/12 10:16 AM, Timon Gehr wrote:

There are two parts, syntax and semantics.

Semantics:
D is already able to express those:

template Tuple(T...){alias T Tuple;} // not the same as
std.typecons.Tuple!

// function with multiple return values:
Tuple!(int,double) foo(int a, double b){
Tuple!(int, double) result; // ok, _no imposed memory layout_
result[0] = a; // ok
result[1] = a+b; // ok
return result;
}

Multiple return values are currently *disallowed explicitly*:
DMD sez: "Error: functions cannot return a tuple"

Just specify the ABI, implement the code gen, and we're done.

Moot point: built-in tuples auto-flatten inside comma-separated lists.

std.typecons.Tuple is a hack to circumvent the arbitrary "cannot return
tuple from function" restriction as well as the auto-flattening.


No, it was more like an implementation of a generic tuple.



Yes, but we wouldn't have needed it if the built-in one would have been 
considered sufficient. It is akin to implementing an 'Int' struct in the 
standard library that wraps a built-in 'int' but does not implicitly 
convert to unsigned.



The
problem is that it is implemented as a struct with a built-in tuple
member. The fact that it is a struct imposes a memory layout. This is
just a side-effect of attempting to address the other two issues. It is
not something that is desirable.


Why?

I don't understand what you're trying to solve, that Tuple is not good at.


Andrei


Off the top of my head:

- Returning it from a function is not efficient.

- It does not play very nicely with type deduction.

- It has clumsy syntax and is therefore rarely used.

Notably, there is no convenient unpacking syntax. Walter does not merge 
the patches of Kenji Hara that would fix this because presumably he 
fears it could get in the way of a more general solution.


- It is a trivial wrapper for an underpowered built-in type. This causes 
confusion.



This situation is neither particularly pragmatic nor pure enough. I'd 
call it a wart.

Re: Multiple return values...

[Manu]

On 10 March 2012 01:17, Timon Gehr  wrote:

> On 03/10/2012 12:02 AM, Manfred Nowak wrote:
>
>> Manu wrote:
>>
>>  I'm just talking about the ABI for returning multiple values, not
>>> chaining
>>>
>>
>> Does this mean, that you want a special type of "function"? For example
>> this would be disallowed statement: `auto result= f( g( parameters));',
>> if `f' and `g' are functions returning multiple values?
>>
>>
> This needs to work. Multiple return values should integrate with the
> existing language support for tuples.
>

What should that do exactly? A function that returns multiple values is
passed as an argument to another function... what exactly is fed to which
arguments of the outer function?
Describe how you see multiple return values working through tuples, while
satisfying the issues I raise in my first couple of posts?

Re: Multiple return values...

[Timon Gehr]

On 03/10/2012 01:00 AM, Manu wrote:

On 10 March 2012 01:17, Timon Gehr mailto:timon.g...@gmx.ch>> wrote:

On 03/10/2012 12:02 AM, Manfred Nowak wrote:

Manu wrote:

I'm just talking about the ABI for returning multiple
values, not
chaining


Does this mean, that you want a special type of "function"? For
example
this would be disallowed statement: `auto result= f( g(
parameters));',
if `f' and `g' are functions returning multiple values?


This needs to work. Multiple return values should integrate with the
existing language support for tuples.


What should that do exactly? A function that returns multiple values is
passed as an argument to another function... what exactly is fed to
which arguments of the outer function?


It would work like built-in tuples already do.

(int, int) foo(int a, int b){return (a,b);}

assert(foo(foo(foo(foo(1,2==(1,2));

(int, int) goo(int a, int b, int c){return (a+b, c);}

assert(goo(foo(2,3),1) == (5,1));



Describe how you see multiple return values working through tuples,
while satisfying the issues I raise in my first couple of posts?


I have done so in my other post, you may refer to that one if something 
is still unclear: http://forum.dlang.org/post/jjdhdk$16v3$1...@digitalmars.com

Re: Multiple return values...

[Manu]

On 10 March 2012 02:10, Timon Gehr  wrote:

> On 03/10/2012 01:00 AM, Manu wrote:
>
>> On 10 March 2012 01:17, Timon Gehr > > wrote:
>>
>>On 03/10/2012 12:02 AM, Manfred Nowak wrote:
>>
>>Manu wrote:
>>
>>I'm just talking about the ABI for returning multiple
>>values, not
>>chaining
>>
>>
>>Does this mean, that you want a special type of "function"? For
>>example
>>this would be disallowed statement: `auto result= f( g(
>>parameters));',
>>if `f' and `g' are functions returning multiple values?
>>
>>
>>This needs to work. Multiple return values should integrate with the
>>existing language support for tuples.
>>
>>
>> What should that do exactly? A function that returns multiple values is
>> passed as an argument to another function... what exactly is fed to
>> which arguments of the outer function?
>>
>
> It would work like built-in tuples already do.
>
> (int, int) foo(int a, int b){return (a,b);}
>
> assert(foo(foo(foo(foo(1,2**==(1,2));
>
> (int, int) goo(int a, int b, int c){return (a+b, c);}
>
> assert(goo(foo(2,3),1) == (5,1));
>
>
>
>  Describe how you see multiple return values working through tuples,
>> while satisfying the issues I raise in my first couple of posts?
>>
>
> I have done so in my other post, you may refer to that one if something is
> still unclear: 
> http://forum.dlang.org/post/**jjdhdk$16v3$1...@digitalmars.com
>

Right, that all looks good.
But I'm unclear of the language semantics, is this built-in a tuple
actually a physical struct like phobos Tuple, or is it strictly a higher
level concept?
I didn't realise such a thing as 'built-in tuple' existed, why does phobos
Tuple exist?

Re: Multiple return values...

[Andrei Alexandrescu]

On 3/9/12 3:59 PM, Timon Gehr wrote:

Yes, but we wouldn't have needed it if the built-in one would have been
considered sufficient.


The goal is considering the language sufficient for implementing a 
useful structure such as Tuple in a library. Same goes about associative 
arrays.



- Returning it from a function is not efficient.


Implementation issue, if at all. Solving that will optimize the return 
of any structure in any D code.



- It does not play very nicely with type deduction.


It plays by the rules enacted by the D programming language. If those 
aren't nice, there's a problem with them rules. If you like them rules 
but want "special" rules for templates, you're asking for magic. Magic 
smells.



- It has clumsy syntax and is therefore rarely used.


Then templates have clumsy syntax and therefore are rarely used.


Notably, there is no convenient unpacking syntax. Walter does not merge
the patches of Kenji Hara that would fix this because presumably he
fears it could get in the way of a more general solution.


This is because we need to think about that stuff to maximize its 
generality.



- It is a trivial wrapper for an underpowered built-in type. This causes
confusion.


It's not quite trivial.


This situation is neither particularly pragmatic nor pure enough. I'd
call it a wart.


I'd call it a great artifact. Love the Tuple (and the Drake).


Andrei

Re: Multiple return values...

[Andrei Alexandrescu]

On 3/9/12 4:10 PM, Timon Gehr wrote:

(int, int) foo(int a, int b){return (a,b);}

assert(foo(foo(foo(foo(1,2==(1,2));

(int, int) goo(int a, int b, int c){return (a+b, c);}

assert(goo(foo(2,3),1) == (5,1));


This is a recipe for disaster because of the implicit expansion. 
Consider forwarding the result of foo to some template. Should the 
template take the tuple type, or automatically expand the tuple and 
accept individual values?


Andrei

Re: Multiple return values...

[Robert Jacques]

On Fri, 09 Mar 2012 22:16:44 -0600, Andrei Alexandrescu 
 wrote:

On 3/9/12 3:59 PM, Timon Gehr wrote:

Notably, there is no convenient unpacking syntax. Walter does not merge
the patches of Kenji Hara that would fix this because presumably he
fears it could get in the way of a more general solution.


This is because we need to think about that stuff to maximize its
generality.


Was simply applying the unpacking rules to all types (i.e. structs, classes, 
etc) ever considered?

Re: Multiple return values...

[Andrei Alexandrescu]

On 3/9/12 9:35 PM, Robert Jacques wrote:

On Fri, 09 Mar 2012 22:16:44 -0600, Andrei Alexandrescu
 wrote:

On 3/9/12 3:59 PM, Timon Gehr wrote:

Notably, there is no convenient unpacking syntax. Walter does not merge
the patches of Kenji Hara that would fix this because presumably he
fears it could get in the way of a more general solution.


This is because we need to think about that stuff to maximize its
generality.


Was simply applying the unpacking rules to all types (i.e. structs,
classes, etc) ever considered?


How would that work?

Andrei

Re: Multiple return values...

[Manu]

On 10 March 2012 06:16, Andrei Alexandrescu
wrote:

> On 3/9/12 3:59 PM, Timon Gehr wrote:
>
>> Yes, but we wouldn't have needed it if the built-in one would have been
>> considered sufficient.
>>
>
> The goal is considering the language sufficient for implementing a useful
> structure such as Tuple in a library. Same goes about associative arrays.


Well if the library Tuple is the official Tuple, then I'm back at square
one, and all my points I raised starting this thread stand.

- Returning it from a function is not efficient.
>>
>
> Implementation issue, if at all. Solving that will optimize the return of
> any structure in any D code.


I disagree, refer to the start of the thread, I argue that returning a
struct by value and returning multiple un-associated values is a
fundamentally different concept. The 2 don't relate in my mind, and
shouldn't try to be smashed together in one concept. This just convolutes
one or the other, depending which one the implementation favours.


> - It does not play very nicely with type deduction.
>>
>
> It plays by the rules enacted by the D programming language. If those
> aren't nice, there's a problem with them rules. If you like them rules but
> want "special" rules for templates, you're asking for magic. Magic smells.


Perhaps the type deduction rule when passing the results of a
multiple-return function to a template could be to implicitly collect into
Tuple only at _THAT_ point. But it could also just as readily be a syntax
error; multiple things passed into a slot meant to receive a single thing
is obviously a syntax error, and an explicit mechanism to collect the
multiple results into a Tuple to perform the pass-through is preferable by
my measure (Intent is explicit, and clearly written before your eyes).

- It has clumsy syntax and is therefore rarely used.
>>
>
> Then templates have clumsy syntax and therefore are rarely used.


This doesn't make sense to me, template syntax has nothing to do with it.
Munging a bunch of things together in order to perform a multi-return is
self-defeating. Why am I needing to write code to structurally&logically
associate these things? That defeats the initial premise. If I *want* to
return a Tuple, I will return a Tuple, and there is no discussion.


> Notably, there is no convenient unpacking syntax. Walter does not merge
>> the patches of Kenji Hara that would fix this because presumably he
>> fears it could get in the way of a more general solution.
>>
>
> This is because we need to think about that stuff to maximize its
> generality.


Any automated 'unpacking' syntax is really just syntactical sugar though...
it's not addressing the problem in a direct way.
The fact that you should need 'unpacking' in the first place means we've
already missed the point.

 This situation is neither particularly pragmatic nor pure enough. I'd
>> call it a wart.
>>
>
> I'd call it a great artifact. Love the Tuple (and the Drake).


I still fundamentally see a clear divide between a Tuple, which is a
deliberately structured association of multiple values, and 'multiple
return values' which is an explicit non-association of multiple returned
things.
You need to address that principle before I can begin to accept the idea of
abusing a structured tuple as a substitute for this important language
feature.

My analogy is the function argument list. You don't think of the arguments
you pass to a function as a Tuple (is it implemented internally in this
way? if so, it is well hidden, and perhaps similar magic can be done...)
You pass, TO a function, multiple un-associated values. They follow a well
understood calling convention (primitives in registers, byval structs
copied to the stack)
It should be possible in the language to express receiving multiple
un-associated values back from the function, in PRECISELY the same way, but
in reverse (primitives in registers, byval structs returned on the stack)

If I pass a structure TO a function by value, I know what happens, a copy
is written to the stack which the function expects to find there.
If I return a structure (Tuple) from a function, I expect precisely the
same behaviour. That is what I have written, that is what I expect the
language to do. We shouldn't mess with this.

I see serious ABI problems and confusion with 'optimising' byval structs to
behave in the way I want from multiple return values.
If I do this: func(byValStruct, intX, floatY, intZ);
What would happen under the 'optimised' call? byValStruct could potentially
hog all the argument registers the CPU has, and the following primitive
arguments may overflow to the stack. As a programmer, I do NOT expect this,
it would be a very nasty surprise.
Those primitive arguments will certainly be used within the function, which
means I've performed a (hidden) redundant store/load straight up. Also,
it's not clear whether the function has business with many/every single
value in the struct I'm passing, so I may very well be loading items fr

Re: Multiple return values...

[Robert Jacques]

On Sat, 10 Mar 2012 00:30:05 -0600, Andrei Alexandrescu 
 wrote:

On 3/9/12 9:35 PM, Robert Jacques wrote:

On Fri, 09 Mar 2012 22:16:44 -0600, Andrei Alexandrescu
 wrote:

On 3/9/12 3:59 PM, Timon Gehr wrote:

Notably, there is no convenient unpacking syntax. Walter does not merge
the patches of Kenji Hara that would fix this because presumably he
fears it could get in the way of a more general solution.


This is because we need to think about that stuff to maximize its
generality.


Was simply applying the unpacking rules to all types (i.e. structs,
classes, etc) ever considered?


How would that work?

Andrei



The proposed unpacking syntax, IIRC, was:

(double a, int b) = fun();

which lowered to:

auto __tup = fun();
double x = __tup[0];  // __tup[0] has a type that is implicitly convertible to 
double
int y = __tup[1];  // ditto

Why not instead lower to:

auto __tup = fun();
double x = __tup.tupleof[0];
int y = __tup.tupleof[1];

That way, fun() could return any struct / class, not just a Tuple. For example,

float3 position = { 1.0f, 2.0f, 3.0f };
auto (x1) = position;
auto (x2,y2) = position;
auto (x3,y3,z3) = position;
assert(x1 == x2 && x1 == x3 && x1 == 1);
assert(y2 == y3 && y2 == 2);
assert(z3 == 3);

would be valid. Note that only unpacking a few of the components of a MRV 
function is a common practice in my experience (mostly Matlab).

P.S. For unpacking classes / structs directly, i.e.

auto (x3,y3,z3) = position;

should lower to

auto x3 = position.tupleof[0];
auto y3 = position.tupleof[1];
auto z3 = position.tupleof[2];

in order to avoid the extra temporary.

P.S.S. Unpacking should respect protection attributes. Since tupleof doesn't 
respect private, protected, etc, the actual lowering should be:

auto __tup = fun();
double x = mixin(__tup.tupleof[0].stringof);
int y = mixin(__tup.tupleof[1].stringof);

Re: Multiple return values...

[Timon Gehr]

On 03/10/2012 05:26 AM, Andrei Alexandrescu wrote:

On 3/9/12 4:10 PM, Timon Gehr wrote:

(int, int) foo(int a, int b){return (a,b);}

assert(foo(foo(foo(foo(1,2==(1,2));

(int, int) goo(int a, int b, int c){return (a+b, c);}

assert(goo(foo(2,3),1) == (5,1));


This is a recipe for disaster because of the implicit expansion.
Consider forwarding the result of foo to some template. Should the
template take the tuple type, or automatically expand the tuple and
accept individual values?

Andrei


That is how built-in tuples behave. I don't feel strongly for or against 
it, but consistency should be preserved. What would be a scenario that 
demonstrates disaster striking?

Re: Multiple return values...

[bearophile]

Robert Jacques:

> The proposed unpacking syntax, IIRC, was:
> 
> (double a, int b) = fun();
> 
> which lowered to:
> 
> auto __tup = fun();
> double x = __tup[0];  // __tup[0] has a type that is implicitly convertible 
> to double
> int y = __tup[1];  // ditto
> 
> Why not instead lower to:
> 
> auto __tup = fun();
> double x = __tup.tupleof[0];
> int y = __tup.tupleof[1];

The advange of lowering it to:
> auto __tup = fun();
> double x = __tup[0]; 

Is that it works with this too:
int[2] foo() { return [10,20; }
auto (x,y) = foo();

This case is quite common.

Bye,
bearophile

Re: Multiple return values...

[Timon Gehr]

On 03/10/2012 05:16 AM, Andrei Alexandrescu wrote:

On 3/9/12 3:59 PM, Timon Gehr wrote:

Yes, but we wouldn't have needed it if the built-in one would have been
considered sufficient.


The goal is considering the language sufficient for implementing a
useful structure such as Tuple in a library.


I am not saying it is not awesome that the language is powerful enough 
to implement a thing like std.typecons.Tuple, but Tuple is too basic to 
be implemented meaningfully in a library. That is why 
std.typecons.Tuple's implementation depends on built-in _tuples_.


Furthermore, it shows that there is a minor language insufficiency: It 
is impossible to overload the static index or slice operators.




Same goes about associative arrays.



Associative arrays are proper built-in types. The rest is an 
implementation detail.



- Returning it from a function is not efficient.


Implementation issue, if at all.


The ABI is part of the design, even if open-ended.


Solving that will optimize the return
of any structure in any D code.



That would presumably reduce the efficiency of built-in structs in some 
other cases.



- It does not play very nicely with type deduction.


It plays by the rules enacted by the D programming language. If those
aren't nice, there's a problem with them rules. If you like them rules
but want "special" rules for templates, you're asking for magic. Magic
smells.



immutable int* p;
(auto x, const(int)* y) = (p, p);

I'm not spotting any templates.


- It has clumsy syntax and is therefore rarely used.


Then templates have clumsy syntax and therefore are rarely used.



... for implementing basic language constructs.


Notably, there is no convenient unpacking syntax. Walter does not merge
the patches of Kenji Hara that would fix this because presumably he
fears it could get in the way of a more general solution.


This is because we need to think about that stuff to maximize its
generality.



... without screwing with existing syntax. That is the only thing that 
makes it hard. Making built-in tuples more powerful and accessible 
across the board would be a reasonably general solution.



- It is a trivial wrapper for an underpowered built-in type. This causes
confusion.


It's not quite trivial.



What is non-trivial about it?
I know that making implementation of trivial things mind-bending is 
appreciated in the C++ community, but in D it often really *is* trivial.

Re: Multiple return values...

[Sean Cavanaugh]

On 3/10/2012 4:37 AM, Manu wrote:


If I pass a structure TO a function by value, I know what happens, a
copy is written to the stack which the function expects to find there.


This is only true if the compiler is forced to use the ABI, when 
inlining is impossible, or the type being passed is too complex. 
Structs of pods, most compilers do magical things to provided you don't 
actively work against the code gen (virtual methods, dllexports etc), 
too many separate .obj units in C++ etc.

Re: Multiple return values...

[Manu]

On 11 March 2012 00:25, Sean Cavanaugh  wrote:

> On 3/10/2012 4:37 AM, Manu wrote:
>
>>
>> If I pass a structure TO a function by value, I know what happens, a
>> copy is written to the stack which the function expects to find there.
>>
>
> This is only true if the compiler is forced to use the ABI, when inlining
> is impossible, or the type being passed is too complex. Structs of pods,
> most compilers do magical things to provided you don't actively work
> against the code gen (virtual methods, dllexports etc), too many separate
> .obj units in C++ etc.
>

Talking about the typical case here, obviously not inlined, calling through
the ABI, struct may be simple, just 2-3 values, can you show me a case
where C++ is able to to anything particularly fancy?

I've never seen the compiler do wildly unexpected things unless whole
program optimisation is enabled, which I don't imagine D will be able to
support any time real soon?

...and even then, relying on WPO for the language to generate good code in
certain circumstances is a really really bad idea. This makes the task of
implementing an efficient compiler for the language extremely difficult.
In most cases, making concise expression of the operation you want to
perform possible in the language will generate better results anyway,
without depending on an aggressive optimiser. It will also make the code
more explicit and readable.

Re: Multiple return values...

[Robert Jacques]

On Sat, 10 Mar 2012 09:58:30 -0600, bearophile  wrote:


Robert Jacques:


The proposed unpacking syntax, IIRC, was:

(double a, int b) = fun();

which lowered to:

auto __tup = fun();
double x = __tup[0];  // __tup[0] has a type that is implicitly convertible to 
double
int y = __tup[1];  // ditto

Why not instead lower to:

auto __tup = fun();
double x = __tup.tupleof[0];
int y = __tup.tupleof[1];


The advange of lowering it to:

auto __tup = fun();
double x = __tup[0];


Is that it works with this too:
int[2] foo() { return [10,20; }
auto (x,y) = foo();

This case is quite common.

Bye,
bearophile



And should not tupleof work with int[2] as well? Ever since int[2] switched to 
being by value, as opposed to by reference it has behaved, under the hood, just 
like a struct with 2 int fields. For that manner, a lot of generic programming 
would get simpler if tupleof worked with the built-in D types.

(Of course we could just hack it via static if statements)

Re: Multiple return values...

[Robert Jacques]

On Sat, 10 Mar 2012 19:27:05 -0600, Manu  wrote:

On 11 March 2012 00:25, Sean Cavanaugh  wrote:


On 3/10/2012 4:37 AM, Manu wrote:



If I pass a structure TO a function by value, I know what happens, a
copy is written to the stack which the function expects to find there.



This is only true if the compiler is forced to use the ABI, when inlining
is impossible, or the type being passed is too complex. Structs of pods,
most compilers do magical things to provided you don't actively work
against the code gen (virtual methods, dllexports etc), too many separate
.obj units in C++ etc.



Talking about the typical case here, obviously not inlined, calling through
the ABI, struct may be simple, just 2-3 values, can you show me a case
where C++ is able to to anything particularly fancy?

I've never seen the compiler do wildly unexpected things unless whole
program optimisation is enabled, which I don't imagine D will be able to
support any time real soon?

...and even then, relying on WPO for the language to generate good code in
certain circumstances is a really really bad idea. This makes the task of
implementing an efficient compiler for the language extremely difficult.
In most cases, making concise expression of the operation you want to
perform possible in the language will generate better results anyway,
without depending on an aggressive optimiser. It will also make the code
more explicit and readable.



Manu, please go read the D ABI (http://dlang.org/abi.html). Remember, your 
example of returning two values using Tuple vs 'real' MRV? The D ABI states 
that those values will be returned via registers. Returning something larger? 
Then the NVRO kicks in which gives you a zero copy approach. On x86-64 these 
limits are different, since you have more registers to play with, but the 
concept is the same. In fact, returning arguments has always been more 
efficient than passing arguments.

Re: Multiple return values...

[Mantis]

11.03.2012 3:45, Robert Jacques пишет:

[...]
Manu, please go read the D ABI (http://dlang.org/abi.html). Remember, 
your example of returning two values using Tuple vs 'real' MRV? The D 
ABI states that those values will be returned via registers. Returning 
something larger? Then the NVRO kicks in which gives you a zero copy 
approach. On x86-64 these limits are different, since you have more 
registers to play with, but the concept is the same. In fact, 
returning arguments has always been more efficient than passing 
arguments.


It doesn't work with FPU stack however, so returning these packed into 
structure should be slower that via ref parameters. E.g:


Tuple!(float, float) callee() {
do something to achieve result in st0,st1
fst st0, st1 into stack
load stack values into EAX, EDX
ret
}

void caller() {
call callee()
push EAX, EDX into a stack
fld stack values into st0, st1
do something with st0, st1
}

As opposed to:

Tuple!(float, float) callee() {
do something to achieve result in st0,st1
ret
}

void caller() {
call callee()
do something with st0, st1
}

Is there something I miss here?

Re: Multiple return values...

[Sean Cavanaugh]

On 3/10/2012 8:08 PM, Mantis wrote:

Tuple!(float, float) callee() {
do something to achieve result in st0,st1
fst st0, st1 into stack
load stack values into EAX, EDX
ret
}

void caller() {
call callee()
push EAX, EDX into a stack
fld stack values into st0, st1
do something with st0, st1
}

As opposed to:

Tuple!(float, float) callee() {
do something to achieve result in st0,st1
ret
}

void caller() {
call callee()
do something with st0, st1
}

Is there something I miss here?


Yes, the fact the FPU stack is deprecated :)

Re: Multiple return values...

[F i L]

Lot of cool discussions going on right now with UFCS and language 
Tuples. I don't know if it's been said before, but one thing I'd 
really like with Tuples syntax is the ability to access multiple 
members the same way. Plus it could be nice to be able to define 
multiple function parameters the same way. eg:


class Actor
{
string name;
float x, y, z;

this(string name, float (x, y, z))
{
this.name = name;
this.(x, y, z) = (x, y, z);
}
}

Re: Multiple return values...

[Andrei Alexandrescu]

On 3/10/12 4:37 AM, Manu wrote:

I still fundamentally see a clear divide between a Tuple, which is a
deliberately structured association of multiple values, and 'multiple
return values' which is an explicit non-association of multiple returned
things.


There is no difference. A Tuple is a product type, exactly like 
superposing arbitrary values together.



You need to address that principle before I can begin to accept the idea
of abusing a structured tuple as a substitute for this important
language feature.

My analogy is the function argument list. You don't think of the
arguments you pass to a function as a Tuple (is it implemented
internally in this way? if so, it is well hidden, and perhaps similar
magic can be done...)
You pass, TO a function, multiple un-associated values.


This analogy is tenuous for D because functions are defined to return 
one type, e.g. typeof(fun(args)) is defined. Once we get into 
disallowing that for certain functions, we're looking at major language 
changes for little benefit.



Andrei

Re: Multiple return values...

[Robert Jacques]

On Sat, 10 Mar 2012 20:35:02 -0600, Sean Cavanaugh  
wrote:


On 3/10/2012 8:08 PM, Mantis wrote:

Tuple!(float, float) callee() {
do something to achieve result in st0,st1
fst st0, st1 into stack
load stack values into EAX, EDX
ret
}

void caller() {
call callee()
push EAX, EDX into a stack
fld stack values into st0, st1
do something with st0, st1
}

As opposed to:

Tuple!(float, float) callee() {
do something to achieve result in st0,st1
ret
}

void caller() {
call callee()
do something with st0, st1
}

Is there something I miss here?


Yes, the fact the FPU stack is deprecated :)



And that on x86-64, the plan is to allow the passing of floating point stack 
data through SSE, which would speed up _all_ structs, not just special MRV 
returns. IIRC, the exact ABI design for this isn't finished yet.

Re: Multiple return values...

[Manu]

On 11 March 2012 03:45, Robert Jacques  wrote:

> On Sat, 10 Mar 2012 19:27:05 -0600, Manu  wrote:
>
>> On 11 March 2012 00:25, Sean Cavanaugh 
>> wrote:
>>
>>  On 3/10/2012 4:37 AM, Manu wrote:
>>>
>>>
 If I pass a structure TO a function by value, I know what happens, a
 copy is written to the stack which the function expects to find there.


>>> This is only true if the compiler is forced to use the ABI, when inlining
>>> is impossible, or the type being passed is too complex. Structs of pods,
>>> most compilers do magical things to provided you don't actively work
>>> against the code gen (virtual methods, dllexports etc), too many separate
>>> .obj units in C++ etc.
>>>
>>>
>> Talking about the typical case here, obviously not inlined, calling
>> through
>> the ABI, struct may be simple, just 2-3 values, can you show me a case
>> where C++ is able to to anything particularly fancy?
>>
>> I've never seen the compiler do wildly unexpected things unless whole
>> program optimisation is enabled, which I don't imagine D will be able to
>> support any time real soon?
>>
>> ...and even then, relying on WPO for the language to generate good code in
>> certain circumstances is a really really bad idea. This makes the task of
>> implementing an efficient compiler for the language extremely difficult.
>> In most cases, making concise expression of the operation you want to
>> perform possible in the language will generate better results anyway,
>> without depending on an aggressive optimiser. It will also make the code
>> more explicit and readable.
>>
>>
> Manu, please go read the D ABI (http://dlang.org/abi.html). Remember,
> your example of returning two values using Tuple vs 'real' MRV? The D ABI
> states that those values will be returned via registers. Returning
> something larger? Then the NVRO kicks in which gives you a zero copy
> approach. On x86-64 these limits are different, since you have more
> registers to play with, but the concept is the same. In fact, returning
> arguments has always been more efficient than passing arguments.
>

Please go read my prior posts. This has absolutely no bearing on what I'm
talking about. In fact, it fuels my argument in some cases.

That said, I've read it, and it scares the hell out of me. D states that a
small struct may be returned in up to 2 registers (8 bytes/16 bytes), I
suspect this is a hack introduced specifically to make ranges efficient?
If it is not simply a struct of 2 register sized things, they get packed
into a magic 8-16 byte struct implicitly for return, and this makes my
brain explode. If I wanted to perform a bunch of shifts, or's, and and's
until it's snugly packed into a 8-16 byte block before returning, I'd do
that explicitly... I DON'T want to do that however, and I *really* don't
want the language doing that for me. If I want to return 4 byte values,
they should be returned in 4 registers, not packed together into a 32bit
word and returned in one. Also, if there are mixed ints, floats, vectors,
even other structs, none of it works; what if there is float data? Swapping
registers now? That's one of the worst penalties there is.

Re: Multiple return values...

[Manu]

On 11 March 2012 04:35, Sean Cavanaugh  wrote:

> On 3/10/2012 8:08 PM, Mantis wrote:
>
>> Tuple!(float, float) callee() {
>> do something to achieve result in st0,st1
>> fst st0, st1 into stack
>> load stack values into EAX, EDX
>> ret
>> }
>>
>> void caller() {
>> call callee()
>> push EAX, EDX into a stack
>> fld stack values into st0, st1
>> do something with st0, st1
>> }
>>
>> As opposed to:
>>
>> Tuple!(float, float) callee() {
>> do something to achieve result in st0,st1
>> ret
>> }
>>
>> void caller() {
>> call callee()
>> do something with st0, st1
>> }
>>
>> Is there something I miss here?
>>
>
> Yes, the fact the FPU stack is deprecated :)
>

Don't dismiss the point, the same still stands true with XMM regs.

Re: Multiple return values...

[Manu]

On 11 March 2012 05:04, Andrei Alexandrescu
wrote:

> On 3/10/12 4:37 AM, Manu wrote:
>
>> I still fundamentally see a clear divide between a Tuple, which is a
>> deliberately structured association of multiple values, and 'multiple
>> return values' which is an explicit non-association of multiple returned
>> things.
>>
>
> There is no difference. A Tuple is a product type, exactly like
> superposing arbitrary values together.


So you're saying that whatever I think about the implementation of Tuple is
wrong? It is not actually a structured type?


> You need to address that principle before I can begin to accept the idea
>> of abusing a structured tuple as a substitute for this important
>> language feature.
>>
>> My analogy is the function argument list. You don't think of the
>> arguments you pass to a function as a Tuple (is it implemented
>> internally in this way? if so, it is well hidden, and perhaps similar
>> magic can be done...)
>> You pass, TO a function, multiple un-associated values.
>>
>
> This analogy is tenuous for D because functions are defined to return one
> type, e.g. typeof(fun(args)) is defined. Once we get into disallowing that
> for certain functions, we're looking at major language changes for little
> benefit.


I don't know how 'major' they'd really work out to be. It's not a paradigm
buster. There are some details, but I don't even think it would need to be
a breaking change to the language (maybe some very subtle tweaks).
Can you quantify 'little' benefit? I started this thread because I have
found myself wishing for this feature every other day. It would be of huge
value, and many others seem to agree.

D has it all, there are so many features in D which make it feel like a
modern language, but this is a missed(/rejected?) opportunity. It's a
natural thing to want to ask a computer to do, but we're mentally trained
into the returns-a-single-thing model from C and whatever and apparently it
was never considered at the initial design stage, but apart from the
expressive side, more importantly in D's case as a native language, it is
an opportunity to implement an important low level feature that no other
language offers me; an efficient multi-return syntax+ABI.

In the tight loops of every program I've ever written, I can't recall a
time when I haven't had a function that needs to return multiple things,
and C/C++ has always forced me into unnecessary memory access to express
this (ref parameters). For the first time in language/compiler history, D
would finally be able to eliminate the last of the redundant memory
accesses in my hottest code in a portable way, at a language/expression
level.

Are you saying it's impossible, that you don't think it should be done, or
it's already solved?
Is it that you don't see the value in it? Is that what I need to convince
you of?
You haven't argued with any of the other points I raised, which leads me to
suspect you either see my points, or you think the entire premise of my
rant is wrong... if so, can you show how, and present a solution that's
workable within existing constructs that meets my criteria without adding
other implicit/logical baggage? Nobody has acknowledged or disputed the
majority of my points :/

D is a native language, that's it's most attractive feature, and while I
appreciate all the high-level correct-ness D offers, it still needs to get
the low level right and ideally improve on existing offerings in meaningful
ways to motivate new users to switch. This is a big un-checked checkbox for
me and my colleagues, and I'd wager any low level realtime programmer out
there who has had to struggle with the codegen in their inner loops..
doubly so if they ever work on non-x86 systems since the penalties are so
much greater.

Re: Multiple return values...

[Manu]

On 11 March 2012 05:04, Andrei Alexandrescu
wrote:
>
> This analogy is tenuous for D because functions are defined to return one
> type, e.g. typeof(fun(args)) is defined. Once we get into disallowing that
> for certain functions, we're looking at major language changes for little
> benefit.


I do appreciate this point though...
Is it currently possible to enumerate a functions argument list? How do you
express that? The problems seem very similar to me.

Re: Multiple return values...

[Timon Gehr]

On 03/11/2012 01:30 PM, Manu wrote:

On 11 March 2012 05:04, Andrei Alexandrescu
mailto:seewebsiteforem...@erdani.org>>
wrote:

This analogy is tenuous for D because functions are defined to
return one type, e.g. typeof(fun(args)) is defined. Once we get into
disallowing that for certain functions, we're looking at major
language changes for little benefit.


I do appreciate this point though...


It is not a valid concern. typeof(fun(args)) would just be a tuple of types.


Is it currently possible to enumerate a functions argument list?


Yes it is.


How do you express that? The problems seem very similar to me.


Language built-in tuples, of course.
See std.traits.ParameterTypeTuple.

It is extremely obvious how multiple return values should work.

Re: Multiple return values...

[Timon Gehr]

On 03/11/2012 12:50 PM, Manu wrote:

Nobody has acknowledged
or disputed the majority of my points :/


I agree with the majority of your points.

Re: Multiple return values...

[David Nadlinger]

On Sunday, 11 March 2012 at 03:04:38 UTC, Andrei Alexandrescu 
wrote:
This analogy is tenuous for D because functions are defined to 
return one type, e.g. typeof(fun(args)) is defined. Once we get 
into disallowing that for certain functions, we're looking at 
major language changes for little benefit.


TypeTuple!(ReturnType1, ReturnType2)?

David

Re: Multiple return values...

[Manu]

On 11 March 2012 14:56, Timon Gehr  wrote:

> On 03/11/2012 12:50 PM, Manu wrote:
>
>> Nobody has acknowledged
>> or disputed the majority of my points :/
>>
>
> I agree with the majority of your points.
>

Cool, well that's encouraging :)
I can't really argue the implementation details, all I can do is assert
criteria/requirements as I see them.

So what was the perceived issue with the pull request you mentioned in an
earlier post? I presume it only implemented the syntax, and not the ABI
bits?


On 11 March 2012 15:08, David Nadlinger  wrote:

> On Sunday, 11 March 2012 at 03:04:38 UTC, Andrei Alexandrescu wrote:
>
>> This analogy is tenuous for D because functions are defined to return one
>> type, e.g. typeof(fun(args)) is defined. Once we get into disallowing that
>> for certain functions, we're looking at major language changes for little
>> benefit.
>>
>
> TypeTuple!(ReturnType1, ReturnType2)?


Right, well I'm glad I'm not the only one :)
I figured this must have some important implication that I totally missed...

Re: Multiple return values...

[Timon Gehr]

On 03/11/2012 02:23 PM, Manu wrote:

On 11 March 2012 14:56, Timon Gehr mailto:timon.g...@gmx.ch>> wrote:

On 03/11/2012 12:50 PM, Manu wrote:

Nobody has acknowledged
or disputed the majority of my points :/


I agree with the majority of your points.


Cool, well that's encouraging :)
I can't really argue the implementation details, all I can do is assert
criteria/requirements as I see them.

So what was the perceived issue with the pull request you mentioned in
an earlier post?


This is the pull request.
https://github.com/D-Programming-Language/dmd/pull/341

I think the issue with it is that there are no obvious issues with it.


I presume it only implemented the syntax, and not the
ABI bits?



Exactly. It implements the assignment from built-in/library tuples to 
multiple newly declared variables, but it does not implement multiple 
return values.

Re: Multiple return values...

[Artur Skawina]

On 03/11/12 02:27, Manu wrote:
> I've never seen the compiler do wildly unexpected things unless whole program 
> optimisation is enabled

The compiler can only ignore the ABI when it knows it sees the whole picture,
ie whole program, or whole unit in case of private/local functions.

> which I don't imagine D will be able to support any time real soon?

Umm, GDC supports WPO, both as LTO and "std" WPO, though i don't remember if
i ever tried the latter.
In fact, until GDC learns cross-module inlining, using these modes is the only
way to make the compiler generate sane code, w/o reverting to turning all
D functions into templates or compiling all sources together (at which point
you could just as well turn on WPO...).
Also, unit-at-a-time, which is on by default, can help.

It (gdc) still has a few LTO bugs which sometimes cause trouble, but often can
be worked around. And you may need to use the *.d runtime files (instead of 
*.di)
etc. 

> ...and even then, relying on WPO for the language to generate good code in 
> certain circumstances is a really really bad idea. This makes the task of 
> implementing an efficient compiler for the language extremely difficult.

There is a reason why i never even considered trying out a non-gcc-based D 
compiler... [1]

For a language like D, WPO in some form is almost required; w/o it you'd need to
always think about how the compiler will treat your code; which, while possible,
would often result in more "unnatural" and tricky solutions. (In C/C++ there's
a *.h/*.c split, which helps mitigate the problem)


artur

[1] LLVM is possibly an option too, but i haven't yet found the time to 
investigate.

Re: Multiple return values...

[Manu]

On 11 March 2012 15:35, Timon Gehr  wrote:

> On 03/11/2012 02:23 PM, Manu wrote:
>
>> On 11 March 2012 14:56, Timon Gehr >
>> > wrote:
>>
>>On 03/11/2012 12:50 PM, Manu wrote:
>>
>>Nobody has acknowledged
>>or disputed the majority of my points :/
>>
>>
>>I agree with the majority of your points.
>>
>>
>> Cool, well that's encouraging :)
>> I can't really argue the implementation details, all I can do is assert
>> criteria/requirements as I see them.
>>
>> So what was the perceived issue with the pull request you mentioned in
>> an earlier post?
>>
>
> This is the pull request.
> https://github.com/D-**Programming-Language/dmd/pull/**341
>
> I think the issue with it is that there are no obvious issues with it.
>
>
>  I presume it only implemented the syntax, and not the
>> ABI bits?
>>
>>
> Exactly. It implements the assignment from built-in/library tuples to
> multiple newly declared variables, but it does not implement multiple
> return values.
>

How does the syntax work precisely?
Is it possible to do each of those things I mentioned earlier; assign to
existing locals, ignore individual return values, etc?

How would the language distinguish from this implementation of implicitly
building/returning/unpacking a tuple, and from explicit intent to do so
structurally? Does the syntax give the code generation enough information
to distinguish multiple-return from struct byval?

Re: Multiple return values...

[Manu]

On 11 March 2012 16:01, Artur Skawina  wrote:

> > which I don't imagine D will be able to support any time real soon?
>
> In fact, until GDC learns cross-module inlining, using these modes is the
> only
> way to make the compiler generate sane code


Yeah I'm very concerned by this currently. But it seems to we a well known
issue with a plan/intent to fix it right?
(Side note: I'd still really like an explicit @forceinline attribute)


> > ...and even then, relying on WPO for the language to generate good code
> in certain circumstances is a really really bad idea. This makes the task
> of implementing an efficient compiler for the language extremely difficult.
>
> There is a reason why i never even considered trying out a non-gcc-based D
> compiler... [1]
>
> For a language like D, WPO in some form is almost required; w/o it you'd
> need to
> always think about how the compiler will treat your code; which, while
> possible,
> would often result in more "unnatural" and tricky solutions. (In C/C++
> there's
> a *.h/*.c split, which helps mitigate the problem)
>

"think about how the compiler will treat your code" ... "'unnatural' and
tricky solutions" ... Outside of cross module inlining, what are some other
common problem cases?

C/C++ .h files only really simplify 2 things, inlining, and templates, but
I can't see how the situation is any different in D?
My understanding is that templates depends on .d/.di files being present
during compilation? So why doesn't D do inlining the same way? If I declare
some inline function in a .d/.di file, surely it should be capable of
inlining?
C/C++ can't inline something from a foreign object either without a
definition in a header...

Re: Multiple return values...

[Artur Skawina]

On 03/11/12 15:59, Manu wrote:
> On 11 March 2012 16:01, Artur Skawina  > wrote:
> 
> > which I don't imagine D will be able to support any time real soon?
> 
> In fact, until GDC learns cross-module inlining, using these modes is the 
> only
> way to make the compiler generate sane code
> 
> 
> Yeah I'm very concerned by this currently. But it seems to we a well known 
> issue with a plan/intent to fix it right?
> (Side note: I'd still really like an explicit @forceinline attribute)

Just '@forceinline' is IMHO too specific and not enough  -- because you also 
want
@noinline, @noclone, @flatten, @cold and more of theses kind of /generic/ 
attributes,
not to mention the target-specific ones. So a way to specify function (and data)
attributes is needed, together with a way to bundle them. Maybe something like
"alias @attr(noinline, noclone, cold, nothrow) @errorpath;".


> > ...and even then, relying on WPO for the language to generate good code 
> in certain circumstances is a really really bad idea. This makes the task of 
> implementing an efficient compiler for the language extremely difficult.
> 
> There is a reason why i never even considered trying out a non-gcc-based 
> D compiler... [1]
> 
> For a language like D, WPO in some form is almost required; w/o it you'd 
> need to
> always think about how the compiler will treat your code; which, while 
> possible,
> would often result in more "unnatural" and tricky solutions. (In C/C++ 
> there's
> a *.h/*.c split, which helps mitigate the problem)
> 
> 
> "think about how the compiler will treat your code" ... "'unnatural' and 
> tricky solutions" ... Outside of cross module inlining, what are some other 
> common problem cases?
> 
> C/C++ .h files only really simplify 2 things, inlining, and templates, but I 
> can't see how the situation is any different in D?

Traditionally, the pre-WPO way, the coder would put the things expected to 
become
part of the caller in header files. In D, header files are unnecessary, other 
than
for the binary-lib-interface case. And currently even if you duplicate part of 
the
code in a *.di file, it doesn't really change anything, other than preventing 
some
optimizations like inlining or cloning (for the omitted parts).

If you have a complex data type which exports eg a range, you may want just 
'.front'
etc to be inlined, and creating a *.di file just for this purpose shouldn't be 
required. Template definitions need to be visible to the caller/user too - 
having
to place them all in a *.di file would not be a good solution - in many cases
most of the code would move there...

Other than that, there is devirtualization - D does not have 'virtual'; the 
compiler
can still do it and even inline the virtual methods, but it needs to know that
nothing overrides the functions - impossible w/o WPO for public non-final 
classes. [1]

> My understanding is that templates depends on .d/.di files being present 
> during compilation? So why doesn't D do inlining the same way? If I declare 
> some inline function in a .d/.di file, surely it should be capable of 
> inlining?
> C/C++ can't inline something from a foreign object either without a 
> definition in a header...

It's the same for D, I just don't think *.di files should be necessary for 
_internal_
cross-module interfaces, or when the full source is available anyway.
Currently, cross-module inlining is a problem for GDC, but even when that's 
fixed,
WPO will help things like devirtualization or changing calling conventions (gcc 
does
it already, but i don't know if it clones non-local functions just for this, 
when not
in WPO mode).

artur

[1] BTW, this means that a "@virtual" attribute might be needed - to prevent
devirtualizing methods for classes used by plugins or other dynamically 
loaded code.

Re: Multiple return values...

[Robert Jacques]

On Sun, 11 Mar 2012 05:57:05 -0500, Manu  wrote:


On 11 March 2012 04:35, Sean Cavanaugh  wrote:


On 3/10/2012 8:08 PM, Mantis wrote:


Tuple!(float, float) callee() {
do something to achieve result in st0,st1
fst st0, st1 into stack
load stack values into EAX, EDX
ret
}

void caller() {
call callee()
push EAX, EDX into a stack
fld stack values into st0, st1
do something with st0, st1
}

As opposed to:

Tuple!(float, float) callee() {
do something to achieve result in st0,st1
ret
}

void caller() {
call callee()
do something with st0, st1
}

Is there something I miss here?



Yes, the fact the FPU stack is deprecated :)



Don't dismiss the point, the same still stands true with XMM regs.



And Walter has talked about using the XMM registers to return floating point 
data for exactly this reason. But those optimization apply to all structs and 
all data types. There's nothing special about MRV. It has to return a set of 
data in a structured manner; this is identical to the case of returning a 
struct.

Re: Multiple return values...

[Manu]

On 11 March 2012 18:50, Robert Jacques  wrote:

> On Sun, 11 Mar 2012 05:57:05 -0500, Manu  wrote:
>
>  On 11 March 2012 04:35, Sean Cavanaugh 
>> wrote:
>>
>>  On 3/10/2012 8:08 PM, Mantis wrote:
>>>
>>>  Tuple!(float, float) callee() {
 do something to achieve result in st0,st1
 fst st0, st1 into stack
 load stack values into EAX, EDX
 ret
 }

 void caller() {
 call callee()
 push EAX, EDX into a stack
 fld stack values into st0, st1
 do something with st0, st1
 }

 As opposed to:

 Tuple!(float, float) callee() {
 do something to achieve result in st0,st1
 ret
 }

 void caller() {
 call callee()
 do something with st0, st1
 }

 Is there something I miss here?


>>> Yes, the fact the FPU stack is deprecated :)
>>>
>>>
>> Don't dismiss the point, the same still stands true with XMM regs.
>>
>>
> And Walter has talked about using the XMM registers to return floating
> point data for exactly this reason. But those optimization apply to all
> structs and all data types. There's nothing special about MRV. It has to
> return a set of data in a structured manner; this is identical to the case
> of returning a struct.
>

Well you can't build these sorts of tightly packed structs in XMM
registers... and even for the GPR's, it's very inefficient to do so. I
haven't done tests, but I suspect this feature is probably a
de-optimisation in all cases other than an int-pair struct (ranges,
delegates? I suspect that's why this was invented). Structure
packing/unpacking code will likely be slower than a store/load.

We just need proper multiple return values, then this can go away. And the
earlier any ABI breaking changes are implemented, the better, while there
are still few(/none?) closed source, binary distributed D libraries.

Re: Multiple return values...

[Manu]

On 11 March 2012 18:45, Artur Skawina  wrote:
>
> Other than that, there is devirtualization - D does not have 'virtual';
> the compiler
> can still do it and even inline the virtual methods, but it needs to know
> that
> nothing overrides the functions - impossible w/o WPO for public non-final
> classes. [1]
>

Oh don't get me started! (...oops, too late!)

This is my single greatest complaint about D, and if I were to adopt D
professionally, I would almost certainly fork the language and fix this.
virtual-by-default seems like the single biggest and most harmful
*mistake*in the whole language.
Defaulting a feature that introduces very costly damage that ONLY a WPO
pass can possibly un-do can only be described as insanity.
Choosing to do this intentionally... does DMD have a powerful WPO? >_<

I genuinely fear for this decision. Junior programmers will write really
slow code unknowingly, and it's too easy for anyone to simply forget to
declare 'final'. Seniors will spend god only knows how much time (late
nights?) trawling through the codebase verifying non-virtuals, and marking
them final, time you probably can't afford when crunching to ship a product.
Add to that the fact that *validating* a method is not overridden anywhere
is NOT a trivial task. Being able to do this with confidence will be
tedious and waste unimaginable amounts of time and frustration... and why?
What was the harm in explicit virtual? I expect explicit 'final' will
surely overwhelm virtuals in number too when considering keyword clutter
(trivial accessors, properties, etc make up the majority of methods) :/

If I were evaluating D to use commercially, this would be the biggest red
flag to consider. The time/efficiency costs could potentially be very high.

> My understanding is that templates depends on .d/.di files being present
> during compilation? So why doesn't D do inlining the same way? If I declare
> some inline function in a .d/.di file, surely it should be capable of
> inlining?
> > C/C++ can't inline something from a foreign object either without a
> definition in a header...
>
> It's the same for D, I just don't think *.di files should be necessary for
> _internal_
> cross-module interfaces, or when the full source is available anyway.
> Currently, cross-module inlining is a problem for GDC, but even when
> that's fixed,
> WPO will help things like devirtualization or changing calling conventions
> (gcc does
> it already, but i don't know if it clones non-local functions just for
> this, when not
> in WPO mode).
>

So assuming that most libraries will be closed source, what's the
implication here for inlines and tempaltes long term? It sounds no
different than .h files.
Is there a plan to be able to store that metadata inside lib/object files
somehow? (I assume not, if there was, why need .di files at all)

Re: Multiple return values...

[Andrei Alexandrescu]

On 3/11/12 7:52 AM, Timon Gehr wrote:

It is extremely obvious how multiple return values should work.


(int, int) fun();

writeln(fun());
auto a = fun();
writeln(a);

What should happen?


Andrei

Re: Multiple return values...

[Andrei Alexandrescu]

On 3/11/12 8:08 AM, David Nadlinger wrote:

On Sunday, 11 March 2012 at 03:04:38 UTC, Andrei Alexandrescu wrote:

This analogy is tenuous for D because functions are defined to return
one type, e.g. typeof(fun(args)) is defined. Once we get into
disallowing that for certain functions, we're looking at major
language changes for little benefit.


TypeTuple!(ReturnType1, ReturnType2)?


So how is the proposed feature any different from Tuple?

Andrei

Re: Multiple return values...

[Andrei Alexandrescu]

On 3/11/12 6:50 AM, Manu wrote:

On 11 March 2012 05:04, Andrei Alexandrescu
mailto:seewebsiteforem...@erdani.org>>
There is no difference. A Tuple is a product type, exactly like
superposing arbitrary values together.

So you're saying that whatever I think about the implementation of Tuple
is wrong? It is not actually a structured type?


I don't know what you mean by "structured type". What I mean by "product 
type" is this: http://en.wikipedia.org/wiki/Product_type



This analogy is tenuous for D because functions are defined to
return one type, e.g. typeof(fun(args)) is defined. Once we get into
disallowing that for certain functions, we're looking at major
language changes for little benefit.

I don't know how 'major' they'd really work out to be. It's not a
paradigm buster. There are some details, but I don't even think it would
need to be a breaking change to the language (maybe some very subtle
tweaks).


I am convinced you are underestimating. The notion that a function 
returns one typed value is strongly embedded in the language and the 
standard library.



Can you quantify 'little' benefit? I started this thread because I have
found myself wishing for this feature every other day. It would be of
huge value, and many others seem to agree.


I think you and others are concerned with different aspects 
(incrementally pleasant syntax vs. efficiency).



D has it all, there are so many features in D which make it feel like a
modern language, but this is a missed(/rejected?) opportunity. It's a
natural thing to want to ask a computer to do, but we're mentally
trained into the returns-a-single-thing model from C and whatever and
apparently it was never considered at the initial design stage, but
apart from the expressive side, more importantly in D's case as a native
language, it is an opportunity to implement an important low level
feature that no other language offers me; an efficient multi-return
syntax+ABI.


D can return multiple values from a function by putting them in a tuple. 
This approach has many advantages, mostly related to avoiding awkward 
ambiguities (e.g. forwarding the result of a multi-return to a variadic 
or overloaded function). Regarding syntax, I am not convinced by 
arguments predicated on removing "Tuple!" from "Tuple!(int, int)". It's 
not progress, and pouring more magic in tuples that is not available 
anywhere else does not strike me as a good path to go.


Regarding low-level efficiency, the main issue is that structs and 
function argument lists have distinct layouts. Consider:


import std.stdio, std.typecons;
int a(int b, int c) {
return b + c;
}
auto foo() {
return tuple(1, 1);
}
void main() {
writeln(a(foo().expand));
}

An adjustment may be needed from the output of a() to the arguments of 
foo(). (Probably not in this case.) I understand that someone very 
concerned with low-level performance would scrutinize such code 
carefully. But I also believe that the same person is willing to forgo a 
whole category of language and library features that don't fulfill a 
demanding performance profile.



In the tight loops of every program I've ever written, I can't recall a
time when I haven't had a function that needs to return multiple things,
and C/C++ has always forced me into unnecessary memory access to express
this (ref parameters). For the first time in language/compiler history,
D would finally be able to eliminate the last of the redundant memory
accesses in my hottest code in a portable way, at a language/expression
level.

Are you saying it's impossible, that you don't think it should be done,
or it's already solved?


I understand your pledge, and all I'm saying is that it is very 
difficult to implement (Walter and I discussed the matter many times 
over the years) and the feature will benefit only very, very few.



Is it that you don't see the value in it? Is that what I need to
convince you of?
You haven't argued with any of the other points I raised, which leads me
to suspect you either see my points, or you think the entire premise of
my rant is wrong... if so, can you show how, and present a solution
that's workable within existing constructs that meets my criteria
without adding other implicit/logical baggage? Nobody has acknowledged
or disputed the majority of my points :/


Your points are understood, but please also understand what exceptional 
circumstances you are invoking. You mention how transferring function 
results costs extra operations. If that cost is significant, that means 
the cost of the function proper is extremely low. So you want to call 
extremely small functions in extremely core loops. The first advice 
would be "avoid calling extremely small functions in extremely core loops"!



D is a native language, that's it's most attractive feature, and while I
appreciate all the high-level correct-ness D offers, it still needs to
get the low level right and ideally improve on existing offerings in

Re: Multiple return values...

[Artur Skawina]

On 03/11/12 18:45, Manu wrote:
> On 11 March 2012 18:45, Artur Skawina  > wrote:
> 
> Other than that, there is devirtualization - D does not have 'virtual'; 
> the compiler
> can still do it and even inline the virtual methods, but it needs to know 
> that
> nothing overrides the functions - impossible w/o WPO for public non-final 
> classes. [1]
[...]
> virtual-by-default seems like the single biggest and most harmful _mistake_ 
> in the whole language.
> Defaulting a feature that introduces very costly damage that ONLY a WPO pass 
> can possibly un-do can only be described as insanity.

Well, there is 'final' and limiting the visibility can help too ('private' etc).

> /validating/ a method is not overridden anywhere is NOT a trivial task. Being 
> able to do this with confidence will be tedious and waste unimaginable 
> amounts of time and frustration...

CPUs won't complain and their time is relatively cheap. :)
IOW, if it can be done by the compiler then it should.

> If I were evaluating D to use commercially, this would be the biggest red 
> flag to consider. The time/efficiency costs could potentially be very high.

If it can be done automatically in C++, by looking just at the generated RTTI 
in object
files, then a similar approach should work for D. I don't think this is such a 
big
problem, especially compared to some things which currently are inexpressible 
in D. 

eg: http://drdobbs.com/184401938

> > My understanding is that templates depends on .d/.di files being 
> present during compilation? So why doesn't D do inlining the same way? If I 
> declare some inline function in a .d/.di file, surely it should be capable of 
> inlining?
> > C/C++ can't inline something from a foreign object either without a 
> definition in a header...
> 
> It's the same for D, I just don't think *.di files should be necessary 
> for _internal_
> cross-module interfaces, or when the full source is available anyway.
> Currently, cross-module inlining is a problem for GDC, but even when 
> that's fixed,
> WPO will help things like devirtualization or changing calling 
> conventions (gcc does
> it already, but i don't know if it clones non-local functions just for 
> this, when not
> in WPO mode).
> 
> 
> So assuming that most libraries will be closed source, what's the implication 
> here for inlines and tempaltes long term? It sounds no different than .h 
> files.
> Is there a plan to be able to store that metadata inside lib/object files 
> somehow? (I assume not, if there was, why need .di files at all)

Well, if you distribute your library as closed source, you can't expect compile 
time
optimizations to work across the app/lib boundary - you can't both have the 
cake and
eat it. Note you *can* add enough information to the object files so that the 
compiler
can do a better job -- simply by compiling your library with LTO enabled. While 
this
effectively turns closed-source into obfuscated-source, it's possible to do this
selectively on a by unit basis - which can be acceptable in some situations.
And if your closed-source library exports non-final classes you may not really 
want
devirtualization; marking certain members as final would probably be good idea 
anyway.
So *.di files work the same as *.h -- you only have to expose the API and can 
add as
much extras as you can/want.

artur

Re: Multiple return values...

[Andrei Alexandrescu]

On 3/11/12 8:23 AM, Manu wrote:

On 11 March 2012 14:56, Timon Gehr mailto:timon.g...@gmx.ch>> wrote:

On 03/11/2012 12:50 PM, Manu wrote:

Nobody has acknowledged
or disputed the majority of my points :/


I agree with the majority of your points.


Cool, well that's encouraging :)
I can't really argue the implementation details, all I can do is assert
criteria/requirements as I see them.

So what was the perceived issue with the pull request you mentioned in
an earlier post? I presume it only implemented the syntax, and not the
ABI bits?


Yes. Just to clarify, Kenji's pull request only improves the syntax.


On 11 March 2012 15:08, David Nadlinger mailto:s...@klickverbot.at>> wrote:

On Sunday, 11 March 2012 at 03:04:38 UTC, Andrei Alexandrescu wrote:

This analogy is tenuous for D because functions are defined to
return one type, e.g. typeof(fun(args)) is defined. Once we get
into disallowing that for certain functions, we're looking at
major language changes for little benefit.


TypeTuple!(ReturnType1, ReturnType2)?


Right, well I'm glad I'm not the only one :)
I figured this must have some important implication that I totally missed...


Of course it does. If you create a variable of type 
TypeTuple!(ReturnType1, ReturnType2), what's its layout?



Andrei