On 13 December 2014 at 08:14, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote: > On 12/12/2014 4:19 AM, Manu via Digitalmars-d wrote: >>> >>> I simply do not understand why distinguishing beteen ref and not-ref is a >>> cornerstone of everything you do. >> >> >> I've said so many times, it's the single greatest regular frustration >> I encounter, by far. >> That is of course a relative measure. It's not the 'cornerstone of >> everything I do'; I don't start discussions about things that are not >> broken and otherwise painless. >> It is the thing that is *the most broken*, and leads to the most edge >> cases, general bloat, and text mixins. >> It comes up the most frequently, and by that metric alone, I consider >> it highest priority on my list. >> >> I've also said many times before, it possibly stems from the fact that >> one of the key cornerstones of almost everything I do in D is interact >> with other languages. >> This is a practical reality, I can't write all my code in D, and have >> mountains of existing code to interact with. >> Boilerplate is the result. D has powerful systems to automate >> boilerplate, like a carrot dangling right in front of my face, but >> it's almost always thwarted by ref, and almost exclusively so. >> >> My recent work updating LuaD to support all the features I required >> wasted about 80% of my time dealing with ref issues. >> In my past C++ bindings solutions, much code, which was otherwise >> simple, readable, and elegant, turned into a mess of text mixins, >> almost exclusively because ref is broken. > > > You've said this before, many times, but what is lacking is an explanation > of WHY. What is the pattern, and why do you need that pattern? You say you > don't like auto ref because it doesn't give you exact control, but what are > the cases where auto ref is wrong?
I did just give some examples, I'll repeat; auto ref fails when the function is extern. It is also wrong that when I pass an int or float, it is passed by ref instead of by value... that is never what I want! What do you get when you take a pointer of a function with an auto-ref arg? ...an error, because it's not actually a function! So in a context where I'm dealing with functions and function pointers (very, very frequent), I suddenly have something that's not a function find it's way into my meta and I have to special-case the hell out of it. The solution in this case is to wrap it in a non-auto-ref shim with the ref-ness explicitly stated in the way I expect... which is in itself another problem, because 'ref' is not part of the type! So the annoying de-auto-ref-ing shim must be a text mixin with some very complex, practically unreadable, definitely unmaintainable logic surrounding it. It's insanity on top of insanity! I also run the invisible risk of generating 2^num_args instances of the code for functions containing auto-ref args. When I'm writing a function that's not a template, I intend, and expect, to write a function that's _not a template_. Templates and functions are different things. I think it's a massive mistake to have created a way to write a template that looks nothing like a template. auto-ref is not, and never has been a tool I have wanted. I don't have any use for auto-ref, and it has only proven to make an already severe problem worse. I've tried to use it before in some instances, but it made ref's out of int's and floats, so even in the rare potentially useful cases, I had to reject it. At the time, you introduced auto-ref as a 'solution' to an issue that I was the primary complainant (although the solution was mainly pushed by Andrei iirc?). I absolutely rejected it then, and said it would have the disastrous effect of setting ref in stone, which I was already very critical about. I reject it even more now that I have had some experience with it. Who was the customer that you designed it for? (It wasn't me!) What issue was it designed to address? >>> Consider a ref counted type, RC!T. If scope were transitive, then you >>> could >>> not have, say, a tree where the edges were RC!T. I.e., the payload of an >>> RC >>> type should not be forced to be scope. >> >> >> I'm not sure I quite visualise this correctly... > > > struct Tree { > RefCount!(Tree*) left; > RefCount!(Tree*) right; > ... > } ... I don't think I'd ever have a use for this code. I've been using trees for a long time, and I can't imagine a situation where each node in a tree would want to be ref counted. It sounds like a disaster for performance, and I can't imagine any advantage it would offer? Perhaps in some complex graph structure... but I expect that sort of thing would be far more specialised. I can see common cases where nodes may contain a refcounted object as node data, but that's different than the situation you demonstrate here. I'll need to think about how that situation would be affected in this case. >> So you're saying that a pointer itself shouldn't be able to escape a >> call tree, but the thing it points to should be able to escape just >> fine? > > > Yes. > > >> It feels like that kinda defeats the purpose... > > > You're arguing that a data structure with only one access point is the only > kind of data structure in use. With transitive scope, such a data structure > would be the only one possible! Surely the possibility remains for such a data structure that is NOT scope... but I think I can see your point. There is a common case, ie, refcounting, where we want to be able to elide rc fiddling, and that may be the sole call for scope in those cases. It seems like that situation calls for something like head-scope. I think transitive scope has a different application, useful in different situations. It seems a problem that a struct that contains a single pointer, which is passed by value, can't have scope applied under your proposal(?). I use this pattern in almost every interaction with other languages that I encounter. Some external API which operates via opaque pointer, wants to be wrapped in such a way that it maintains it's behaviour as a reference type, but also gains convenient member-style access. It's kinda making a pseudo-class out of something that's not a class, and scope needs to support this. >> I guess you're seeing a situation where 'scope' is almost exclusively >> useful as a mechanism to tell the RC that it doesn't need to worry >> about ref-fiddling, and the thing we're passing isn't interested in >> scope restrictions at any level other than that RC optimisation? >> I guess I can see your angle... but my reaction is if that's all you >> are concerned about, maybe scope is the wrong tool for eliding ref >> fiddling in that case :/ > > > 'scope' is a way to say that this use of a pointer does not escape this > scope. That is incredibly useful. > > Recall my statements that pervasive refcounting is a terrible performance > problem because of all the inc/dec? Knowing that references cannot escape > means an inc/dec pair can be elided. I know this. I'm not arguing against any proposition that we need scope to make RC practical. I've been arguing in favour of that for at least as long as I've been batting for the ARC team, and even then some more, because it would also address rvalue->ref, which is another massive pain point! But there are more things than pointers which need to be protected against escaping their scope, in particular, things that contain pointers... Maybe there's a compromise. If we say scope isn't 'transitive', but it is transitive when it comes to aggregates? Ie, you can apply scope to a by-val struct, and all aggregate members have scope applied. It is not transitive in the sense that it does not follow through pointer members, but any such pointer members may not escape as if it were a pointer argument alone. This is unlike any other behaviour in D, but without that, I think this proposal fails to suit my most common use cases.