On 13 December 2014 at 08:14, Walter Bright via Digitalmars-d
<digitalmars-d@puremagic.com> wrote:
> On 12/12/2014 4:19 AM, Manu via Digitalmars-d wrote:
>>>
>>> I simply do not understand why distinguishing beteen ref and not-ref is a
>>> cornerstone of everything you do.
>>
>>
>> I've said so many times, it's the single greatest regular frustration
>> I encounter, by far.
>> That is of course a relative measure. It's not the 'cornerstone of
>> everything I do'; I don't start discussions about things that are not
>> broken and otherwise painless.
>> It is the thing that is *the most broken*, and leads to the most edge
>> cases, general bloat, and text mixins.
>> It comes up the most frequently, and by that metric alone, I consider
>> it highest priority on my list.
>>
>> I've also said many times before, it possibly stems from the fact that
>> one of the key cornerstones of almost everything I do in D is interact
>> with other languages.
>> This is a practical reality, I can't write all my code in D, and have
>> mountains of existing code to interact with.
>> Boilerplate is the result. D has powerful systems to automate
>> boilerplate, like a carrot dangling right in front of my face, but
>> it's almost always thwarted by ref, and almost exclusively so.
>>
>> My recent work updating LuaD to support all the features I required
>> wasted about 80% of my time dealing with ref issues.
>> In my past C++ bindings solutions, much code, which was otherwise
>> simple, readable, and elegant, turned into a mess of text mixins,
>> almost exclusively because ref is broken.
>
>
> You've said this before, many times, but what is lacking is an explanation
> of WHY. What is the pattern, and why do you need that pattern? You say you
> don't like auto ref because it doesn't give you exact control, but what are
> the cases where auto ref is wrong?
I did just give some examples, I'll repeat; auto ref fails when the
function is extern.
It is also wrong that when I pass an int or float, it is passed by ref
instead of by value... that is never what I want!
What do you get when you take a pointer of a function with an auto-ref
arg? ...an error, because it's not actually a function!
So in a context where I'm dealing with functions and function pointers
(very, very frequent), I suddenly have something that's not a function
find it's way into my meta and I have to special-case the hell out of
it.
The solution in this case is to wrap it in a non-auto-ref shim with
the ref-ness explicitly stated in the way I expect... which is in
itself another problem, because 'ref' is not part of the type! So the
annoying de-auto-ref-ing shim must be a text mixin with some very
complex, practically unreadable, definitely unmaintainable logic
surrounding it. It's insanity on top of insanity!
I also run the invisible risk of generating 2^num_args instances of
the code for functions containing auto-ref args.
When I'm writing a function that's not a template, I intend, and
expect, to write a function that's _not a template_.
Templates and functions are different things. I think it's a massive
mistake to have created a way to write a template that looks nothing
like a template.
auto-ref is not, and never has been a tool I have wanted. I don't have
any use for auto-ref, and it has only proven to make an already severe
problem worse. I've tried to use it before in some instances, but it
made ref's out of int's and floats, so even in the rare potentially
useful cases, I had to reject it.
At the time, you introduced auto-ref as a 'solution' to an issue that
I was the primary complainant (although the solution was mainly pushed
by Andrei iirc?). I absolutely rejected it then, and said it would
have the disastrous effect of setting ref in stone, which I was
already very critical about. I reject it even more now that I have had
some experience with it.
Who was the customer that you designed it for? (It wasn't me!)
What issue was it designed to address?
>>> Consider a ref counted type, RC!T. If scope were transitive, then you
>>> could
>>> not have, say, a tree where the edges were RC!T. I.e., the payload of an
>>> RC
>>> type should not be forced to be scope.
>>
>>
>> I'm not sure I quite visualise this correctly...
>
>
> struct Tree {
> RefCount!(Tree*) left;
> RefCount!(Tree*) right;
> ...
> }
... I don't think I'd ever have a use for this code.
I've been using trees for a long time, and I can't imagine a situation
where each node in a tree would want to be ref counted.
It sounds like a disaster for performance, and I can't imagine any
advantage it would offer?
Perhaps in some complex graph structure... but I expect that sort of
thing would be far more specialised.
I can see common cases where nodes may contain a refcounted object as
node data, but that's different than the situation you demonstrate
here.
I'll need to think about how that situation would be affected in this case.
>> So you're saying that a pointer itself shouldn't be able to escape a
>> call tree, but the thing it points to should be able to escape just
>> fine?
>
>
> Yes.
>
>
>> It feels like that kinda defeats the purpose...
>
>
> You're arguing that a data structure with only one access point is the only
> kind of data structure in use. With transitive scope, such a data structure
> would be the only one possible!
Surely the possibility remains for such a data structure that is NOT
scope... but I think I can see your point.
There is a common case, ie, refcounting, where we want to be able to
elide rc fiddling, and that may be the sole call for scope in those
cases.
It seems like that situation calls for something like head-scope. I
think transitive scope has a different application, useful in
different situations.
It seems a problem that a struct that contains a single pointer, which
is passed by value, can't have scope applied under your proposal(?).
I use this pattern in almost every interaction with other languages
that I encounter. Some external API which operates via opaque pointer,
wants to be wrapped in such a way that it maintains it's behaviour as
a reference type, but also gains convenient member-style access. It's
kinda making a pseudo-class out of something that's not a class, and
scope needs to support this.
>> I guess you're seeing a situation where 'scope' is almost exclusively
>> useful as a mechanism to tell the RC that it doesn't need to worry
>> about ref-fiddling, and the thing we're passing isn't interested in
>> scope restrictions at any level other than that RC optimisation?
>> I guess I can see your angle... but my reaction is if that's all you
>> are concerned about, maybe scope is the wrong tool for eliding ref
>> fiddling in that case :/
>
>
> 'scope' is a way to say that this use of a pointer does not escape this
> scope. That is incredibly useful.
>
> Recall my statements that pervasive refcounting is a terrible performance
> problem because of all the inc/dec? Knowing that references cannot escape
> means an inc/dec pair can be elided.
I know this. I'm not arguing against any proposition that we need
scope to make RC practical. I've been arguing in favour of that for at
least as long as I've been batting for the ARC team, and even then
some more, because it would also address rvalue->ref, which is another
massive pain point!
But there are more things than pointers which need to be protected
against escaping their scope, in particular, things that contain
pointers...
Maybe there's a compromise. If we say scope isn't 'transitive', but it
is transitive when it comes to aggregates?
Ie, you can apply scope to a by-val struct, and all aggregate members
have scope applied.
It is not transitive in the sense that it does not follow through
pointer members, but any such pointer members may not escape as if it
were a pointer argument alone.
This is unlike any other behaviour in D, but without that, I think
this proposal fails to suit my most common use cases.
