Re: DIP 1014
On Thursday, 4 October 2018 at 12:08:38 UTC, Shachar Shemesh wrote: Two distinct things. Kinke was talking about how to pass a struct through the ABI. You are talking about special-casing a specific name. Not just name, but argument passing as well. Not to mention, your special case is to transform it to something you can *already* specify in the language. Why? Because that syntax pertains specifically to construction, which is what a compiler move is; is not currently used by the language (the fact that the compiler doesn't error on it is an oversight); enforces calling convention. Which is, however, not a reason to formalize it and make it a requirement for an isolated specific case, such as this one, utilizing a syntax that is currently not used by the language. There is positively nothing in DIP 1014 that is "syntax not used by the language". Quite the contrary. Which is what I said in the very next sentence, so I'm not sure what your point is here. It's like we're having a discussion but we aren't at the same time. As opposed to trying to fit existing language semantics to something that the language didn't seem to want to allow in the first place. Formalize it as a suggestion, and we can discuss the "as opposed to". Alright, let's get back to it after the weekend then. Like I said, I think there's a lot you're glossing over here (such as backwards compatibility). Backwards compatibility? With what, exactly? Non-existing explicit moves?
Re: DIP 1014
On 04/10/18 13:43, Stanislav Blinov wrote: * move the data as part of the call hook rather than before * Use a different name and signature on the hook function Yes, exactly. It would have to be special if you don't want to leave room for the compiler implementors. That's not how standards work. If you don't want compiler implementors to have a choice in the matter, you put MUST in the specs. Doing anything else is, by and large, considered harmful. The calling convention for particular types (i.e. those that do have a move hook defined) would have to be enforced in some way. See the neighbor thread wrt move semantics by kinke. Two distinct things. Kinke was talking about how to pass a struct through the ABI. You are talking about special-casing a specific name. Not to mention, your special case is to transform it to something you can *already* specify in the language. Why? Which is, however, not a reason to formalize it and make it a requirement for an isolated specific case, such as this one, utilizing a syntax that is currently not used by the language. There is positively nothing in DIP 1014 that is "syntax not used by the language". Quite the contrary. As opposed to trying to fit existing language semantics to something that the language didn't seem to want to allow in the first place. Formalize it as a suggestion, and we can discuss the "as opposed to". Like I said, I think there's a lot you're glossing over here (such as backwards compatibility). Shachar
Re: DIP 1014
On Thursday, 4 October 2018 at 08:32:44 UTC, Shachar Shemesh wrote: On 04/10/18 11:16, Paolo Invernizzi wrote: While I want to thank you both, about the quality of this thread, what kind of "consequences that go beyond what I think you understand" are you thinking of? Can you give an example? Assuming I understand Stanislav's proposal correctly (an Yes it seems you do. assumption I'm reluctant to make, hence my request for something more formal), it boils down to two points: * move the data as part of the call hook rather than before * Use a different name and signature on the hook function Yes, exactly. The first one we can argue for or against. My original proposal was phrased the way it was precisely because that's the way copying works in D (copy first, patch the data later). About a week after I submitted it, Andrei came forward with requesting to move to copy constructors. The second, to me, is a non-starter. The only way you'd get a function who's signature is: void someName(Type rhs); But which actually maintains rhs's address from before the call is if the compiler treats "someName" as a special case, and emits code which would normally be emitted for the function: void someName(ref Type rhs); It would have to be special if you don't want to leave room for the compiler implementors. The calling convention for particular types (i.e. those that do have a move hook defined) would have to be enforced in some way. See the neighbor thread wrt move semantics by kinke. That's why it was important for me to clear up whether there is *ever* a case in the current language where that happens (answer: only by accident, which is the same as saying "no"). Which is, however, not a reason to formalize it and make it a requirement for an isolated specific case, such as this one, utilizing a syntax that is currently not used by the language. As opposed to trying to fit existing language semantics to something that the language didn't seem to want to allow in the first place. So to get that to work, you'd need to insert a special case into the ABI of the language: if the function's name is someName, treat it differently. Yes, that's what I'm talking about. At this point, you might as well call a spade a spade, and just give the function that signature explicitly. s/someName/opPostMove/, and you get DIP 1014 (as far as point #2 is concerned).
Re: DIP 1014
On 04/10/18 11:16, Paolo Invernizzi wrote: While I want to thank you both, about the quality of this thread, what kind of "consequences that go beyond what I think you understand" are you thinking of? Can you give an example? Assuming I understand Stanislav's proposal correctly (an assumption I'm reluctant to make, hence my request for something more formal), it boils down to two points: * move the data as part of the call hook rather than before * Use a different name and signature on the hook function The first one we can argue for or against. My original proposal was phrased the way it was precisely because that's the way copying works in D (copy first, patch the data later). About a week after I submitted it, Andrei came forward with requesting to move to copy constructors. The second, to me, is a non-starter. The only way you'd get a function who's signature is: void someName(Type rhs); But which actually maintains rhs's address from before the call is if the compiler treats "someName" as a special case, and emits code which would normally be emitted for the function: void someName(ref Type rhs); That's why it was important for me to clear up whether there is *ever* a case in the current language where that happens (answer: only by accident, which is the same as saying "no"). So to get that to work, you'd need to insert a special case into the ABI of the language: if the function's name is someName, treat it differently. At this point, you might as well call a spade a spade, and just give the function that signature explicitly. s/someName/opPostMove/, and you get DIP 1014 (as far as point #2 is concerned). Shachar
Re: DIP 1014
On Thursday, 4 October 2018 at 08:10:31 UTC, Shachar Shemesh wrote: On 04/10/18 11:05, Stanislav Blinov wrote: On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh wrote: [...] For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much. The example isn't brittle. It is simply not an example. If you want to leave it out, however, then I think you should submit an orderly proposal. The changes you seem to be suggesting have consequences that go beyond what I think you understand, and there can be no serious discussion of it while it is not clear from your posts which part of what you say is the relevant one. Shachar While I want to thank you both, about the quality of this thread, what kind of "consequences that go beyond what I think you understand" are you thinking of? Can you give an example? Thanks, Paolo
Re: DIP 1014
On 04/10/18 11:05, Stanislav Blinov wrote: On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh wrote: If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working. You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong. For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much. The example isn't brittle. It is simply not an example. If you want to leave it out, however, then I think you should submit an orderly proposal. The changes you seem to be suggesting have consequences that go beyond what I think you understand, and there can be no serious discussion of it while it is not clear from your posts which part of what you say is the relevant one. Shachar
Re: DIP 1014
On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh wrote: If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working. You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong. For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much.
Re: DIP 1014
On Wed, Oct 3, 2018 at 11:00 PM Timothee Cour via Digitalmars-d wrote: > > @Manu, @Jonathan M Davis > > > GNU's std::string implementation stores an interior pointer! >_< > > it's not just GNU's std::string ; it can crop up in other places, see > https://github.com/Syniurge/Calypso/issues/70 in opencv (cv:: MatStep) Sure. Certainly, it shows up in C++ fairly often... but I'm working on the STL containers, and I didn't think there were any implementations that did that (because inefficient use of space), but turns out the GNU implementation bent me over, at least as far as I've encountered yet. It's kinda got me stuck.
Re: DIP 1014
@Manu, @Jonathan M Davis > GNU's std::string implementation stores an interior pointer! >_< it's not just GNU's std::string ; it can crop up in other places, see https://github.com/Syniurge/Calypso/issues/70 in opencv (cv:: MatStep) On Wed, Oct 3, 2018 at 8:10 PM Shachar Shemesh via Digitalmars-d wrote: > > On 03/10/18 23:25, Stanislav Blinov wrote: > > It *is* true when the type doesn't have a destructor. Extending that to > > a move hook, it will also be true because destruction will be elided. > > I know what you're talking about, that happens for types that have > > destructors. > > No, destructors have nothing to do with it, as well they shouldn't. The > whole point of D moving structs around is that no destruction is needed. > It took me a while to figure out why your program does appear to work. > At first I thought it was because of inlining, but that was wrong. > > The reason your test case works (sometimes, if you don't breath on it > too heavily) is because the object is actually moved twice. Once when > returning from the function into the variable, and another when copied > into opAssign's argument. This results in it returning to its original > address. > > If you do *anything* to that program, and that includes even changing > its compilation flags (try enabling inlining), it will stop working. > > You should have known that when you found out it doesn't work on ldc: > ldc and dmd use the same front-end. If you think something works > fundamentally different between the two, you are probably wrong. > > To verify my guess is right, I tried the following change: add to > createCounter and createCounterNoNRV in your original program (no > destructors) the following two lines: >int a; >write(a); > > You have added another local variable to the functions, but otherwise > changed absolutely nothing. You will notice your program now has an offset. > > Shachar
Re: DIP 1014
On 03/10/18 23:25, Stanislav Blinov wrote: It *is* true when the type doesn't have a destructor. Extending that to a move hook, it will also be true because destruction will be elided. I know what you're talking about, that happens for types that have destructors. No, destructors have nothing to do with it, as well they shouldn't. The whole point of D moving structs around is that no destruction is needed. It took me a while to figure out why your program does appear to work. At first I thought it was because of inlining, but that was wrong. The reason your test case works (sometimes, if you don't breath on it too heavily) is because the object is actually moved twice. Once when returning from the function into the variable, and another when copied into opAssign's argument. This results in it returning to its original address. If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working. You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong. To verify my guess is right, I tried the following change: add to createCounter and createCounterNoNRV in your original program (no destructors) the following two lines: int a; write(a); You have added another local variable to the functions, but otherwise changed absolutely nothing. You will notice your program now has an offset. Shachar
Re: DIP 1014
On Wednesday, 3 October 2018 at 18:58:37 UTC, Shachar Shemesh wrote: On 03/10/18 20:43, Stanislav Blinov wrote: On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh wrote: I.e. - I am asserting if a move was not caught. The program fails to run on either ldc or dmd. To me, this makes perfect sense as for the way D is built. In essence, opAssign isn't guaranteed to run. Feel free to build a struct where that assert passes to convince me. That's a slightly different issue here. Well, I view this issue as a deal breaker. If you need to move the object *in order* to pass it to your move hook, then anything that requires knowing the address of the old instance will, by definition, not work. I feel like we're still not on the same page here. this(typeof(this)) doesn't work like a move hook in D right now. I'm *suggesting* making that be a move ctor, instead of opPostMove from your DIP (see below). Look at the output. The operator is being run, it can't *not* run, Sure it can. Just look at the example I posted on the other thread (https://forum.dlang.org/post/pp2v16$1014$1...@digitalmars.com). The hook you mention is downright @disabled there. In fact, had that not been the case, this DIP would never have happened. Yup, we're definitely not on the same page :) That's not what I'm talking about at all. Here is the flaw in your logic: void opAssign(Tracker rhs) rhs is passed by value. This means that already at the point opAssign is called, rhs *already* has a different address than the one it was passed in with. Currently that is only true if you define a destructor. No, that's not true. Try printing the instance's address in the constructor and again in your operator. It *is* true when the type doesn't have a destructor. Extending that to a move hook, it will also be true because destruction will be elided. I know what you're talking about, that happens for types that have destructors. In the presence of a move hook, 'rhs' would first have to pass through that hook, which will not take destructors into account at all. I'm sorry, I'm not following. What is the difference between what you're proposing and opPostMove as defined? 1. with this(typeof(this)) the type of argument would never change. With opPostMove, it may. Remember that 'is(Tracker == const(Tracker))' is false. 2. you won't have to always move all the bits unconditionally. 3. symmetry with this(this) This is the run I got: $ ./movetest2 Address of temporary is 'b382e390', counter points to 'b382e390' ... which is '0' bytes from the address of temporary. ...I'm not sure what I should have seen, or what I should have concluded from it. This is your original program, unmodified. This illustrates the intended behavior of a move hook if it existed in the language. The 'rhs' that was passed to the call was constructed at that same address (&rhs.counter == &rhs.localCounter). I.e. this is how I'm suggesting a this(typeof(this)) *could* work. this(typeof(this)), of course, would need to be special in the ABI, but again, that's one special function instead of two. No. My proposal requires one amendment to argument passing in the ABI, but no special cases at all. Changes to the ABI are not the same as changes to the run time library. How so? Or, more to the point, what's argument passing OR runtime have to do with this? Let's take a step back for a moment and look at what should actually be happening for this hook to work (which you briefly mention in the DIP): 1. The compiler constructs the value. In your case, it constructs two: It does not. It copies the bits from one to the other. Poor choice of words on my part. It *creates* two. Whereas with this(typeof(this)) no implicit copying of bits is required, a-la a C++ move constructor. The programmer is free to choose the bits they need, or do a blit-and-patch if so desired. Except that unlike a C++ move constructor, no state bookkeeping would be necessary (same is true with your DIP as is). the original and the new one. In my case, it constructs the original and then passes it over to the move ctor (one blit potentially avoided). 2. It calls the hook (move ctor). I'm not sure I follow you on that one. What did you mean? In your case, that would be when it calls __move_post_blt. 3. In your case, it calls the opPostMove. In all cases, you need to call the hook, whatever it is, for those structs that have it, and do some default handling for those that don't. Correct, which would just be whatever the compilers already do. 4. In any case, it *doesn't* destruct the original. Ever. The alternative would be to force the programmer to put the original back into valid state, and suddenly we're back to C++ with all it's pleasantries. That last part is quite different from the current model, in which the compiler always destructs function arguments. That's why my example fails when a de
Re: DIP 1014
On Wednesday, 3 October 2018 at 18:38:50 UTC, Manu wrote: On Wed, Oct 3, 2018 at 7:00 AM Stanislav Blinov via Digitalmars-d wrote: Any function in D that has a signature of the form ReturnType foo(Type x); in C++ would have an equivalent signature of ReturnType foo(Type&& x); // NOT ReturnType foo(Type x); What are you talking about? Equivalent C++ is: ReturnType foo(Type x); C++ has rvalue references, move semantics are explicit. D doesn't have any of that. Perhaps I wasn't quite clear in that above statement though. Given some type Bar, compare these two calls in C++ in D, and tell me, which signature in C++ should correspond to D? I'm not talking about ABI, I'm talking about semantics. foo(std::move(bar)); // C++ foo(move(bar)); // D remembering that D's move() doesn't call postblit. void foo(Bar bar) wouldn't satisfy that last bit, would it? Yes, the semantics are different, as in D the move occurs before the call. But in D right now you *must* assume that the argument may have been moved, with all the consequences the language currently entails (and some of which the DIP attempts to resolve), whereas in C++ you can be explicit about it via overloading for rvalue references. It's impossible to perform copy elision when passing an lvalue by-value *to* a function, but that's a case where you depend on move semantics. Of course it's impossible. I'm not sure I understand your point here. Also, within the function that receives an argument by value, you depend on move to construct something with that argument. Type&& passes a reference, which means you can perform the move direct from source to destination, without a stop at the middle man. Yup, no argument there.
Re: DIP 1014
On 03/10/18 20:43, Stanislav Blinov wrote: On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh wrote: I.e. - I am asserting if a move was not caught. The program fails to run on either ldc or dmd. To me, this makes perfect sense as for the way D is built. In essence, opAssign isn't guaranteed to run. Feel free to build a struct where that assert passes to convince me. That's a slightly different issue here. Well, I view this issue as a deal breaker. If you need to move the object *in order* to pass it to your move hook, then anything that requires knowing the address of the old instance will, by definition, not work. Look at the output. The operator is being run, it can't *not* run, Sure it can. Just look at the example I posted on the other thread (https://forum.dlang.org/post/pp2v16$1014$1...@digitalmars.com). The hook you mention is downright @disabled there. In fact, had that not been the case, this DIP would never have happened. Here is the flaw in your logic: void opAssign(Tracker rhs) rhs is passed by value. This means that already at the point opAssign is called, rhs *already* has a different address than the one it was passed in with. Currently that is only true if you define a destructor. No, that's not true. Try printing the instance's address in the constructor and again in your operator. In the presence of a move hook, 'rhs' would first have to pass through that hook, which will not take destructors into account at all. I'm sorry, I'm not following. What is the difference between what you're proposing and opPostMove as defined? Consider your own DIP: what you're suggesting is the ability to take the address of the original when a move is taking place. My example shows that in the simplest case even today, address of the original is already the address of the argument. This is the run I got: $ ./movetest2 Address of temporary is 'b382e390', counter points to 'b382e390' ... which is '0' bytes from the address of temporary. Address of temporary is 'b382e390', counter points to '8eb82b60' ... which is '-966984906800' bytes from the address of temporary. Address of temporary is 'b382e390', counter points to 'b382e390' ... which is '0' bytes from the address of temporary. Address of temporary is 'b382e390', counter points to '8eb82b60' ... which is '-966984906800' bytes from the address of temporary. I'm not sure what I should have seen, or what I should have concluded from it. This is your original program, unmodified. The changes are literally the same as the ones you're proposing: "When moving a struct's instance, the compiler MUST call __move_post_blt giving it both new and old instances' addresses." That is the same that would have to happen with this(typeof(this) rhs), where &this is the address of new instance, and &rhs is the address of old instance, but there's no need for opPostMove then. I guess what I should've said from the start is that the semantics you're proposing fit nicely within one special function, instead of two. Except, like I said, it's not working for me, and I find it hard to understand how it *can* work (inlining notwithstanding), which is why I did not propose it. this(typeof(this)), of course, would need to be special in the ABI, but again, that's one special function instead of two. No. My proposal requires one amendment to argument passing in the ABI, but no special cases at all. Changes to the ABI are not the same as changes to the run time library. Let's take a step back for a moment and look at what should actually be happening for this hook to work (which you briefly mention in the DIP): 1. The compiler constructs the value. In your case, it constructs two: It does not. It copies the bits from one to the other. This also means that a struct where some of its members have a hook is copied wholesale and patched, which is typically faster than copying in parts. the original and the new one. In my case, it constructs the original and then passes it over to the move ctor (one blit potentially avoided). 2. It calls the hook (move ctor). I'm not sure I follow you on that one. What did you mean? 3. In your case, it calls the opPostMove. In all cases, you need to call the hook, whatever it is, for those structs that have it, and do some default handling for those that don't. 4. In any case, it *doesn't* destruct the original. Ever. The alternative would be to force the programmer to put the original back into valid state, and suddenly we're back to C++ with all it's pleasantries. That last part is quite different from the current model, in which the compiler always destructs function arguments. That's why my example fails when a destructor is present. Like I said above, I don't think that's correct. The other thing to note (again something that you mention but don't expand on), and that's nodding back to my comment about making move() and empl
Re: DIP 1014
On Wed, Oct 3, 2018 at 7:00 AM Stanislav Blinov via Digitalmars-d
wrote:
>
> Shachar, as I don't see a better place of discussing that DIP at
> the moment, I'll pour some observations and thoughts in here if
> you don't mind, will add some comments on GitHub later.
> As I see it right now, it's a case of over-engineering of a quite
> simple concept.
>
> 1. A new function, called __move_post_blt, will be added to
> DRuntime.
>
> That's unnecessary, if not downright harmful for the language. We
> should strive to remove things from DRuntime, not add to it. The
> core language should deal with type memory, not a .so or dll. And
> it's extraneous, because...
>
> 2 and 3. onPostMove and __move_post_blt:
>
> They're unnecessary as well. All that's required is to allow a
> by-value constructor, e.g:
>
> struct S {
> this(S rhs);
> }
>
> Any function in D that has a signature of the form
>
> ReturnType foo(Type x);
>
> in C++ would have an equivalent signature of
>
> ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
What are you talking about? Equivalent C++ is:
ReturnType foo(Type x);
It's impossible to perform copy elision when passing an lvalue
by-value *to* a function, but that's a case where you depend on move
semantics.
Also, within the function that receives an argument by value, you
depend on move to construct something with that argument.
Type&& passes a reference, which means you can perform the move direct
from source to destination, without a stop at the middle man.
This all has nothing to do with Walter's surprising claim that "as the
DMD compiler doesn't actually move structs"... I'm still trying to
understand this statement.
Re: DIP 1014
On Wednesday, 3 October 2018 at 08:21:38 UTC, Manu wrote: Okay, so copy elision is working... but moves otherwise are not? That's still not what we've been peddling all these years. A whole lot of design surface area is dedicated to implicit move semantics... and they don't work? What does it do? postblit unnecessarily? No. The problem is that the language is under-specified. It is built on the *assumption* that no one ever should create self-referencing data. But it does not enforce that. Which eventually leads to someone trying to make such data and then run into a wall, or worse, a heisenbug. Thing is, there isn't anything wrong with self-referencing data per se. It's that the language plumbing should either disallow it wholesale (i.e. Rust) or allow a graceful way of handling it. Neither is present in D. The latter could be added though, that's what the DIP is about.
Re: DIP 1014
On Wednesday, 3 October 2018 at 17:43:08 UTC, Stanislav Blinov wrote: But IMHO, it's something that should be fixed by not making these facilities built into the language. s/not//
Re: DIP 1014
On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh
wrote:
On 03/10/18 17:29, Stanislav Blinov wrote:
OMG, that's so simple!!! Why didn't I think of it?
Oh wait, I did.
Now I see why sometimes your posts are greeted with hostility.
Yes. I am actually sorry about that. I was responding to your
assumption that I'm wrong. Had your post been phrased as "why
didn't you", instead of "you're wrong wrong wrong" I wouldn't
have responded that way.
Like I said, I am sorry.
I am sorry as well since I wasn't clear in my initial post.
> Allow me to further illustrate with something that can be
written in D > today:
I am not sure what you were trying to demonstrate, so instead I
wanted to see if you succeeded. I added the following to your
Tracker struct:
~this() {
writefln("%s destructed", &this);
assert(counter is null || counter is &localCounter);
}
I.e. - I am asserting if a move was not caught. The program
fails to run on either ldc or dmd. To me, this makes perfect
sense as for the way D is built. In essence, opAssign isn't
guaranteed to run. Feel free to build a struct where that
assert passes to convince me.
That's a slightly different issue here.
Look at the output. The operator is being run, it can't *not*
run, unlike postblit (ironically, right now it doesn't run on
fixed-size arrays though). In fact, as soon as you define a
destructor, the compiler will generate a by-value opAssign if you
haven't defined one.
That's a separate problem. Currently, presence of a destructor
makes the compilers generate different code, because it cannot
elide destruction of arguments, because explicit move semantics
do not exist in the language. That's why I haven't included a
destructor in the example to begin with.
Here is the flaw in your logic:
void opAssign(Tracker rhs)
rhs is passed by value. This means that already at the point
opAssign is called, rhs *already* has a different address than
the one it was passed in with.
Currently that is only true if you define a destructor. That
would not be true, however, if a move hook in any form existed in
the language. That was my point.
I only used opAssign as something resembling the supposed new
behavior, not as a "look, it already works". In the presence of a
move hook, 'rhs' would first have to pass through that hook,
which will not take destructors into account at all.
Consider your own DIP: what you're suggesting is the ability to
take the address of the original when a move is taking place. My
example shows that in the simplest case even today, address of
the original is already the address of the argument. Except it
cannot be enforced in any way right now. A move hook will have to
enforce that, as it will have to be called for every move.
I did not follow your logic on why this isn't so, but I don't
see how you can make it not so without changing the ABI quite
drastically.
The changes are literally the same as the ones you're proposing:
"When moving a struct's instance, the compiler MUST call
__move_post_blt giving it both new and old instances' addresses."
That is the same that would have to happen with this(typeof(this)
rhs), where &this is the address of new instance, and &rhs is the
address of old instance, but there's no need for opPostMove then.
I guess what I should've said from the start is that the
semantics you're proposing fit nicely within one special
function, instead of two.
this(typeof(this)), of course, would need to be special in the
ABI, but again, that's one special function instead of two.
Let's take a step back for a moment and look at what should
actually be happening for this hook to work (which you briefly
mention in the DIP):
1. The compiler constructs the value. In your case, it constructs
two: the original and the new one. In my case, it constructs the
original and then passes it over to the move ctor (one blit
potentially avoided).
2. It calls the hook (move ctor).
3. In your case, it calls the opPostMove.
4. In any case, it *doesn't* destruct the original. Ever. The
alternative would be to force the programmer to put the original
back into valid state, and suddenly we're back to C++ with all
it's pleasantries.
That last part is quite different from the current model, in
which the compiler always destructs function arguments. That's
why my example fails when a destructor is present.
The other thing to note (again something that you mention but
don't expand on), and that's nodding back to my comment about
making move() and emplace() intrinsics, is that creating such a
hook *will* invalidate current behavior of move(). Which is
perhaps more easily fixed with your implementation, actually,
*except* for the part about eliding destruction. Unions are
unreliable for that unless we also change the spec that talks
about them.
But IMHO, it's something that should be fixed by not making these
facilities built into the
Re: DIP 1014
On 03/10/18 12:48, Corel wrote:
The fact that in D the structures to date are not moved, is known for
years ... take advantage of this fact, and move on.
I have no idea where you got this fact:
import std.stdio;
struct MoveTest {
static uint counter=1;
uint id;
@disable this(this);
@disable this(MoveTest);
this(uint dummy) {
id = counter++;
writefln("Constructed %s id %s", &this, id);
}
~this() {
writefln("Id %s destroyed at %s", id, &this);
}
}
MoveTest func1() {
return MoveTest(3);
}
void func2(MoveTest m) {
}
int main() {
func2(func1());
return 0;
}
$ rdmd movetest.d
Constructed 7FFDC7A663E0 id 1
Id 1 destroyed at 7FFDC7A66400
Our instance was constructed at one address, but destroyed at another.
In other words, it was moved.
Can we, please, put that myth to rest?
Shachar
Re: DIP 1014
On Wed, Oct 3, 2018 at 2:50 AM Corel via Digitalmars-d wrote: > > On Wednesday, 3 October 2018 at 08:21:38 UTC, Manu wrote: > > On Tue, Oct 2, 2018 at 6:15 PM Walter Bright via Digitalmars-d > > wrote: > >> > >> On 10/2/2018 4:30 PM, Adam D. Ruppe wrote: > >> > On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis > >> > wrote: > >> >> Yeah. IIRC, it was supposed to be _guaranteed_ that the > >> >> compiler moved structs in a number of situations - e.g. > >> >> when the return value was an rvalue. Something like > >> > > >> > Eh, I don't think that moves it, but rather just constructs > >> > it in-place for the next call. > >> > >> The technical term for that is "copy elision". > > > > Okay, so copy elision is working... but moves otherwise are > > not? That's still not what we've been peddling all these years. > > A whole lot of design surface area is dedicated to implicit > > move semantics... and they don't work? What does it do? > > postblit unnecessarily? > > The impression is that you are complaining about the continuous > lack of "things" based on an incomplete knowledge of how D works > in detail ... tragically you invoke low-level features, and you > do not know the question. > > The fact that in D the structures to date are not moved, is known > for years ... take advantage of this fact, and move on. > > Work on an implementation that works, AFTER profile it, and > possibly complain about performance. O_o .. this is one of the stranger replies I've ever gotten here.
Re: DIP 1014
On 03/10/18 18:33, Shachar Shemesh wrote:
~this() {
writefln("%s destructed", &this);
assert(counter is null || counter is &localCounter);
}
You might also want to add @disable this(this); and remove the dead code
(i.e. - the case where the pointer is global) to reduce noise. I
verified that neither one changes anything in the outcome.
Shachar
Re: DIP 1014
On 03/10/18 17:29, Stanislav Blinov wrote:
OMG, that's so simple!!! Why didn't I think of it?
Oh wait, I did.
Now I see why sometimes your posts are greeted with hostility.
Yes. I am actually sorry about that. I was responding to your assumption
that I'm wrong. Had your post been phrased as "why didn't you", instead
of "you're wrong wrong wrong" I wouldn't have responded that way.
Like I said, I am sorry.
> Allow me to further illustrate with something that can be written in
D > today:
I am not sure what you were trying to demonstrate, so instead I wanted
to see if you succeeded. I added the following to your Tracker struct:
~this() {
writefln("%s destructed", &this);
assert(counter is null || counter is &localCounter);
}
I.e. - I am asserting if a move was not caught. The program fails to run
on either ldc or dmd. To me, this makes perfect sense as for the way D
is built. In essence, opAssign isn't guaranteed to run. Feel free to
build a struct where that assert passes to convince me.
Here is the flaw in your logic:
void opAssign(Tracker rhs)
rhs is passed by value. This means that already at the point opAssign is
called, rhs *already* has a different address than the one it was passed
in with. I did not follow your logic on why this isn't so, but I don't
see how you can make it not so without changing the ABI quite drastically.
Shachar
Re: DIP 1014
On Wednesday, 3 October 2018 at 14:07:58 UTC, Shachar Shemesh
wrote:
If you read the DIP, you will notice that the *address* in
which the old instance resides is quite important...
Allow me to further illustrate with something that can be written
in D today:
import std.stdio;
struct Tracker {
static int globalCounter;
int localCounter;
int* counter;
this (bool local) {
if (local) counter = &localCounter;
else counter = &globalCounter;
}
// this should be this(Tracker rhs)
void opAssign(Tracker rhs) {
// note: taking address of local parameter
// note: LDC will already 'optimize' the move which in
the absence
// of any move hooks will mess up the address; try with
DMD
printf("Address of temporary is '%x', counter points to
'%x'\n", &rhs, rhs.counter);
auto d = cast(void*) rhs.counter - cast(void*) &rhs;
printf("... which is '%ld' bytes from the address of
temporary.\n", d);
localCounter = rhs.localCounter;
counter = rhs.counter;
if (counter is &rhs.localCounter)
counter = &localCounter;
}
}
auto createCounter(bool local = true) {
Tracker result = Tracker(local);
return result;
}
auto createCounterNoNRV(bool local = true) {
return Tracker(local);
}
void main() {
Tracker stale1, stale2;
stale1 = createCounter();
stale2 = createCounter(false);
Tracker stale3, stale4;
stale3 = createCounterNoNRV();
stale4 = createCounterNoNRV(false);
}
If you run the above with DMD, you'll see what I mean about
obviating the address. If we get this(typeof(this)) (that is
*always* called on move) into the language, the behavior would be
set in stone regardless of compiler.
Re: DIP 1014
On Wednesday, 3 October 2018 at 14:07:58 UTC, Shachar Shemesh
wrote:
On 03/10/18 16:56, Stanislav Blinov wrote:
struct S {
this(S rhs);
OMG, that's so simple!!! Why didn't I think of it?
Oh wait, I did.
Now I see why sometimes your posts are greeted with hostility.
And this simply and utterly doesn't work.
If you read the DIP, you will notice that the *address* in
which the old instance resides is quite important for
performing the actual move. This is not available with the
interface you're suggesting, mainly because by the time you
have rhs, it has already moved.
In other words, for the interface above to work, the type must
already be movable, which kinda contradict what we're trying to
achieve here.
In the presence of such a constructor, the compiler will have to
call it every time it moves the value, same as what you're
proposing for __move_post_blt. This obviates the need of an
address: address of the argument will always already be
sufficient, even though it's not ref, as the chain of calls for
this(S) will inevitably start with the address of something
constructed in place.
in C++ would have an equivalent signature of
ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
No, it is not. You see, in C++, x is an "rvalue *reference*". x
has not moved by this point in the run, it has simply had its
address passed to foo.
You've misunderstood me. Yes, in C++ there's an obvious
difference between pass-by-value and pass-by-rvalue-reference,
and it is always user's responsibility to write a move ctor. Not
so in D. In D, you can always assume that anything passed by
value *is* an rvalue reference, precisely because of D's take on
move semantics. I.e. any argument passed by value can assumed to
be moved or constructed in place (that's the main difference from
C++, where it must be explicitly specified).
Re: DIP 1014
On 03/10/18 16:56, Stanislav Blinov wrote:
struct S {
this(S rhs);
OMG, that's so simple!!! Why didn't I think of it?
Oh wait, I did.
And this simply and utterly doesn't work.
If you read the DIP, you will notice that the *address* in which the old
instance resides is quite important for performing the actual move. This
is not available with the interface you're suggesting, mainly because by
the time you have rhs, it has already moved.
In other words, for the interface above to work, the type must already
be movable, which kinda contradict what we're trying to achieve here.
in C++ would have an equivalent signature of
ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
No, it is not. You see, in C++, x is an "rvalue *reference*". x has not
moved by this point in the run, it has simply had its address passed to
foo. Please see
https://stackoverflow.com/questions/28483250/rvalue-reference-is-treated-as-an-lvalue
Shachar
Re: DIP 1014
On Wednesday, 3 October 2018 at 13:56:29 UTC, Stanislav Blinov
wrote:
Aendment, this should of course be:
this(Tracker oldLocation) {
localCounter = oldLocation.locaclCounter;
counter = oldLocation.counter;
if( counter is &oldLocation.localCounter )
counter = &localCounter;
}
Re: DIP 1014
Shachar, as I don't see a better place of discussing that DIP at
the moment, I'll pour some observations and thoughts in here if
you don't mind, will add some comments on GitHub later.
As I see it right now, it's a case of over-engineering of a quite
simple concept.
1. A new function, called __move_post_blt, will be added to
DRuntime.
That's unnecessary, if not downright harmful for the language. We
should strive to remove things from DRuntime, not add to it. The
core language should deal with type memory, not a .so or dll. And
it's extraneous, because...
2 and 3. onPostMove and __move_post_blt:
They're unnecessary as well. All that's required is to allow a
by-value constructor, e.g:
struct S {
this(S rhs);
}
Any function in D that has a signature of the form
ReturnType foo(Type x);
in C++ would have an equivalent signature of
ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
because passing by value in D always implies a possible move. The
'x' in such functions can be safely cannibalized without any
repercussions, as it is either a temporary on the call site, or,
which is especially pertaining to the original bugzilla
discussion, constructed in place via copy elision.
Thus in effect this(S) would be an equivalent of C++'s move
constructor. We already have a de-facto move-assignment in the
form of opAssign(S), this(S) would be a natural extension to that.
Note, per above, that it is NOT a copy constructor, although user
code may want to create a copy *before* calling it, to create the
temporary.
Such approach reduces added complexity. The only potential
problem with it would be a need to "special-case" initialization
from .init, although at the moment, I think even that may be
unnecessary: this is a hook after all.
Your example from the DIP would become:
struct Tracker {
static uint globalCounter;
uint localCounter;
uint* counter;
@disable this(this);
this(bool local) {
localCounter = 0;
if( local )
counter = &localCounter;
else
counter = &globalCounter;
}
this(Tracker oldLocation) {
if( counter is &oldLocation.localCounter )
counter = &localCounter;
}
void increment() {
(*counter)++;
}
}
Usage:
auto old = Tracker(true);
// ...
auto new = move(old); // calls Tracker.this(Tracker);
...this avoids any need to inject special postblits into user
code.
As I see it, in addition to the above, what would be really
desirable is for move() and emplace() family of calls to become
compiler intrinsics instead of library constructs. Those at the
moment are complete poison: being templates they infect user code
with dependencies on libc (moveEmplace calls memset and memcpy of
all things) and unnecessary calls to DRuntime (typeid), and they
of course blow up the amount of generated code in the form of
template instantiations. That's despite the fact that the
compiler possesses ALL the necessary knowledge at the time of
those calls.
Re: DIP 1014
On 03/10/18 04:10, Walter Bright wrote:
On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote:
Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler
moved structs in a number of situations - e.g. when the return value
was an rvalue. Something like
Eh, I don't think that moves it, but rather just constructs it
in-place for the next call.
The technical term for that is "copy elision".
I'm not sure I follow.
First of all, you cannot elide the copy if there is more than one
potential local variable you are returning, ala:
A someFunc() {
A a, b;
manipulate(a); manipulate(b);
if( someRandomCondition )
return a;
return b;
}
What happens then?
What happens if A has @disable this(this)?
What happens if we explicitly call std.algorithm.move?
Shachar
Re: DIP 1014
On Wednesday, 3 October 2018 at 08:21:38 UTC, Manu wrote: On Tue, Oct 2, 2018 at 6:15 PM Walter Bright via Digitalmars-d wrote: On 10/2/2018 4:30 PM, Adam D. Ruppe wrote: > On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis > wrote: >> Yeah. IIRC, it was supposed to be _guaranteed_ that the >> compiler moved structs in a number of situations - e.g. >> when the return value was an rvalue. Something like > > Eh, I don't think that moves it, but rather just constructs > it in-place for the next call. The technical term for that is "copy elision". Okay, so copy elision is working... but moves otherwise are not? That's still not what we've been peddling all these years. A whole lot of design surface area is dedicated to implicit move semantics... and they don't work? What does it do? postblit unnecessarily? The impression is that you are complaining about the continuous lack of "things" based on an incomplete knowledge of how D works in detail ... tragically you invoke low-level features, and you do not know the question. The fact that in D the structures to date are not moved, is known for years ... take advantage of this fact, and move on. Work on an implementation that works, AFTER profile it, and possibly complain about performance.
Re: DIP 1014
On Tue, Oct 2, 2018 at 6:15 PM Walter Bright via Digitalmars-d wrote: > > On 10/2/2018 4:30 PM, Adam D. Ruppe wrote: > > On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote: > >> Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved > >> structs > >> in a number of situations - e.g. when the return value was an rvalue. > >> Something like > > > > Eh, I don't think that moves it, but rather just constructs it in-place for > > the > > next call. > > The technical term for that is "copy elision". Okay, so copy elision is working... but moves otherwise are not? That's still not what we've been peddling all these years. A whole lot of design surface area is dedicated to implicit move semantics... and they don't work? What does it do? postblit unnecessarily?
Re: DIP 1014
On 10/2/2018 4:30 PM, Adam D. Ruppe wrote: On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote: Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved structs in a number of situations - e.g. when the return value was an rvalue. Something like Eh, I don't think that moves it, but rather just constructs it in-place for the next call. The technical term for that is "copy elision".
Re: DIP 1014
On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote: Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved structs in a number of situations - e.g. when the return value was an rvalue. Something like Eh, I don't think that moves it, but rather just constructs it in-place for the next call.
Re: DIP 1014
On Tuesday, October 2, 2018 11:54:57 AM MDT Manu via Digitalmars-d wrote: > On Tue, Oct 2, 2018 at 2:40 AM Walter Bright via Digitalmars-d > > wrote: > > On 10/2/2018 2:17 AM, Walter Bright wrote: > > > 1. Don't allow moving of C++ structs > > > 2. Add a struct attribute that means "not moveable" > > > 3. DIP 1014, which is add a __move_post_blit() function (most complex > > > solution) 4. Use copy/destruct for C++ structs that have copy > > > constructors (this is the old C++ solution, and is less efficient > > > than the move constructor)> > > The postblit solution can also work today, > > > > https://issues.dlang.org/show_bug.cgi?id=17448#c37 > > > > as the DMD compiler doesn't actually move structs. > > Sorry... what?! DMD doesn't move? O_O > We've been talking endlessly about move semantics for like, 10 years... > Do the other compilers move? > > I don't understand... I've missed something. Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved structs in a number of situations - e.g. when the return value was an rvalue. Something like A foo(); void bar(A); bar(foo()); is supposed to be guaranteed to move and not copy the return value. Maybe it manages to place the return value in a way that doesn't require actually moving it or copying it, but structs being moved was supposed to be a key thing that D could do. Are we currently getting a bunch of copies that we shouldn't be getting because the compiler isn't yet doing moves when it should be? A number of us have been answering questions for years based on what the spec and TDPL say indicating that the compiler moves structs. And while I'm all for having objects be moveable by default, I confess that I don't understand what the problem is with the opPostMove idea that the DIP is presenting. I don't think that it should be the norm by any means, but it sure seems like it's cleanly solving a major problem with interacting with C++, and it makes it possible to do stuff like have pointers and dynamic arrays refer to a structs internals in a way that we can't @safely do right now (even though occasionally, it really would be useful to be able to do it). And it sure seems like opPostMove fits in cleanly with the current design, though maybe I'm missing something. Either way, without something like it, we're clearly missing some key functionality for a systems language - particularly one that wants to interoperate with C++. - Jonathan M Davis
Re: DIP 1014
On 10/2/18 1:51 PM, Manu wrote: But dangling pointer is an instant crash/memory corruption... it's a pretty bad 'bug'. Yeah doesn't sound very brilliant. I think such a workaround wouldn't fare well. To keep momentum while we mull over a solution to this I suggest you look at porting other data structures.
Re: DIP 1014
On Tue, Oct 2, 2018 at 2:40 AM Walter Bright via Digitalmars-d wrote: > > On 10/2/2018 2:17 AM, Walter Bright wrote: > > 1. Don't allow moving of C++ structs > > 2. Add a struct attribute that means "not moveable" > > 3. DIP 1014, which is add a __move_post_blit() function (most complex > > solution) > > 4. Use copy/destruct for C++ structs that have copy constructors (this is > > the > > old C++ solution, and is less efficient than the move constructor) > > The postblit solution can also work today, > > https://issues.dlang.org/show_bug.cgi?id=17448#c37 > > as the DMD compiler doesn't actually move structs. Sorry... what?! DMD doesn't move? O_O We've been talking endlessly about move semantics for like, 10 years... Do the other compilers move? I don't understand... I've missed something. > So you're OK for the time > being until DMD does, or a copying garbage collector is implemented. Ummm...
Re: DIP 1014
On Tue, Oct 2, 2018 at 2:20 AM Walter Bright via Digitalmars-d wrote: > > On 9/29/2018 9:34 PM, Manu wrote: > > Who knows about DIP 1014? (struct move hook) > > Is it well received? Is it likely to be accepted soon? > > > > I'm working on the std::string binding, it's almost finished... but > > then I hit a brick wall. > > GNU's std::string implementation stores an interior pointer! >_< > > > The rationale behind D allowing structs to be moveable is to enable a copying > garbage collector. Do we do that? What's the use of that? Does the opPostMove() concept not work here? If the GC wanted to move something, it can call it like any other code? > Some solutions to this problem: > > 1. Don't allow moving of C++ structs std::string, std::vector, etc are quite impotent without move semantics. I'm pretty sure the first time I ever started using those containers was around 2015 when we became secure we could use C++11 in our code (supported by all compiler vendors). Before that, they were banned and we had alternative solutions. > 2. Add a struct attribute that means "not moveable" They must be movable though. > 3. DIP 1014, which is add a __move_post_blit() function (most complex > solution) It's alleged you're working through the DIP... what's the story there? Are you unhappy with it? > 4. Use copy/destruct for C++ structs that have copy constructors (this is the > old C++ solution, and is less efficient than the move constructor) If that's where we land on this, I'll struggle to find value in my work. Containers become as impotent as back before 2011 :/ > Pragmatically, I suggest for the moment just ignore the problem, file a bug > report for std::string, and move on. But dangling pointer is an instant crash/memory corruption... it's a pretty bad 'bug'.
Re: DIP 1014
On 10/2/2018 2:17 AM, Walter Bright wrote: 1. Don't allow moving of C++ structs 2. Add a struct attribute that means "not moveable" 3. DIP 1014, which is add a __move_post_blit() function (most complex solution) 4. Use copy/destruct for C++ structs that have copy constructors (this is the old C++ solution, and is less efficient than the move constructor) The postblit solution can also work today, https://issues.dlang.org/show_bug.cgi?id=17448#c37 as the DMD compiler doesn't actually move structs. So you're OK for the time being until DMD does, or a copying garbage collector is implemented.
Re: DIP 1014
On 9/29/2018 9:34 PM, Manu wrote: Who knows about DIP 1014? (struct move hook) Is it well received? Is it likely to be accepted soon? I'm working on the std::string binding, it's almost finished... but then I hit a brick wall. GNU's std::string implementation stores an interior pointer! >_< The rationale behind D allowing structs to be moveable is to enable a copying garbage collector. Some solutions to this problem: 1. Don't allow moving of C++ structs 2. Add a struct attribute that means "not moveable" 3. DIP 1014, which is add a __move_post_blit() function (most complex solution) 4. Use copy/destruct for C++ structs that have copy constructors (this is the old C++ solution, and is less efficient than the move constructor) A discussion of the rationale for the C++ move constructor is: https://akrzemi1.wordpress.com/2011/08/11/move-constructor/ > Anyway, I'm blocked until this DIP is accepted; so, is it looking promising? Pragmatically, I suggest for the moment just ignore the problem, file a bug report for std::string, and move on.
Re: DIP 1014
On 9/29/2018 9:34 PM, Manu wrote: Who knows about DIP 1014? (struct move hook) When discussing DIP 1014, a link is helpful: https://github.com/dlang/DIPs/blob/38cec74a7471735559e3b8a7553f55102d289d28/DIPs/DIP1014.md
Re: DIP 1014
On Sunday, September 30, 2018 1:35:28 AM MDT Shachar Shemesh via Digitalmars-d wrote: > On 30/09/18 10:26, Manu wrote: > > Other implementations make much better use of that built-in space by > > not wasting 8 bytes on an interior pointer for small-strings. > > I will point out that a pointer that *sometimes* points to an internal > member was one of the use cases I documented when I submitted the DIP. > > Starting a long discussion about the merits of the design is a bit > off-topic. I will point out that branch prediction considerations > *might* make this a wise choice, despite the loss of 8 bytes of > potential storage. > > Either way, this is a design that is highly sensitive to precise use > pattern, which, admitably, GNU's std::string probably can't know. I think that the key thing here is that if GNU's std::string is using a design like this, it's that much more critical that we have a way to hook into moves to do stuff like adjust pointers. It's one more example of a real world use case where the DIP (or something like it) is needed, or there are things that we simply won't be able to do in D - and given the push to interface with C++, it's that much more important. And while a discussion could certainly be had as to whether GNU's design decision was a good one or not, it's secondary to what really matters here, which is what the state of the DIP is how we're going to deal with interfacing with this C++ code. We need to worry about how to interface with it whether it's the best design ever or whether it's the worst design ever - and we have to take into account the fact the its design could actually change in future versions if they decide that a different way is better (e.g. GNU could change to match other implementations, or other implementations could change to match GNU, depending on what actually turned out to be best in practice when all of the various factors were taken into account - including developers making future decisions that aren't necessarily good ones; we have to interface with the code whether it's good or bad). All in all though, if anything, I have to think that this issue increases the chances of the DIP being accepted given the importance that Walter and Andrei have been placing on interfacing with C++. And having it come up while they're in the middle of discussing it probably doesn't hurt - though maybe they were already going to accept it. I don't know. Personally, while I tend to think that it's generally better to avoid designs where opPostMove is necessary if possible, I think that the case was well made that we need a solution like it in certain cases, and if we want to interface with C++, which can do more or less arbitrary stuff in its move constructors, I don't see how we can avoid having an analogue unless we want to give up on interfacing with that code without an extra compatibility layer. - Jonathan M Davis
Re: DIP 1014
On 30/09/18 10:26, Manu wrote: Other implementations make much better use of that built-in space by not wasting 8 bytes on an interior pointer for small-strings. I will point out that a pointer that *sometimes* points to an internal member was one of the use cases I documented when I submitted the DIP. Starting a long discussion about the merits of the design is a bit off-topic. I will point out that branch prediction considerations *might* make this a wise choice, despite the loss of 8 bytes of potential storage. Either way, this is a design that is highly sensitive to precise use pattern, which, admitably, GNU's std::string probably can't know. Shachar
Re: DIP 1014
On Sat, Sep 29, 2018 at 11:50 PM Walter Bright via Digitalmars-d
wrote:
>
> On 9/29/2018 9:34 PM, Manu wrote:
> > GNU's std::string implementation stores an interior pointer! >_<
> >
> > No other implementation does this. It's a really bad implementation
> > actually, quite inefficient. It could make better use of its space for
> > small-strings if it wasn't wasting 8-bytes for an interior pointer to
> > a small string buffer...
>
> Could you post a synopsis of the layout of std::string?
The code's all in the PR if you wanna dig into it.
The synopsis is:
struct string
{
char* ptr;
size_t len;
union
{
char[16] localBuffer;
size_type allocatedCapacity;
}
bool isAllocated() { return ptr != &localBuffer[0]; }
bool capacity() { return isAllocated() ? allocatedCapacity :
localBuffer.length; }
this(DefaultCtor) { ptr = &localBuffer[0]; } // <- and here it is.
interior pointer that breaks move semantics
}
Other implementations make much better use of that built-in space by
not wasting 8 bytes on an interior pointer for small-strings.
Re: DIP 1014
On 9/29/2018 9:34 PM, Manu wrote: GNU's std::string implementation stores an interior pointer! >_< No other implementation does this. It's a really bad implementation actually, quite inefficient. It could make better use of its space for small-strings if it wasn't wasting 8-bytes for an interior pointer to a small string buffer... Could you post a synopsis of the layout of std::string?
Re: DIP 1014
On Saturday, September 29, 2018 10:34:20 PM MDT Manu via Digitalmars-d wrote: > Who knows about DIP 1014? (struct move hook) > Is it well received? Is it likely to be accepted soon? > > I'm working on the std::string binding, it's almost finished... but > then I hit a brick wall. > GNU's std::string implementation stores an interior pointer! >_< > > No other implementation does this. It's a really bad implementation > actually, quite inefficient. It could make better use of its space for > small-strings if it wasn't wasting 8-bytes for an interior pointer to > a small string buffer... > > Anyway, I'm blocked until this DIP is accepted; so, is it looking > promising? Per https://github.com/dlang/DIPs/tree/master/DIPs, it's at the "Formal Assessement" stage, whatever that means (I'm not well enough versed in all of the stages of the current DIP process to say what that means). However, I would think that if Walter and Andrei were aware of this particular problem with std::string, given the importance they place on interfacing with C++, it would greatly increase the odds of that DIP being accepted. - Jonathan M Davis
Re: DIP 1014
On Sunday, 30 September 2018 at 04:34:20 UTC, Manu wrote: Who knows about DIP 1014? (struct move hook) Is it well received? Is it likely to be accepted soon? I'm working on the std::string binding, it's almost finished... but then I hit a brick wall. GNU's std::string implementation stores an interior pointer! >_< No other implementation does this. It's a really bad implementation actually, quite inefficient. It could make better use of its space for small-strings if it wasn't wasting 8-bytes for an interior pointer to a small string buffer... Anyway, I'm blocked until this DIP is accepted; so, is it looking promising? It's pending a decision from Walter & Andrei, which I hope to hear soon.
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 17/07/18 16:29, aliak00 wrote: A postblit on a class issues a compiler error. And an identity opAssign on a class also issues a compiler error. So I'm not sure how this would be different. And the page In https://dlang.org/spec/operatoroverloading.html also explicitly mentions differences between ops on classes or structs. That's actually a good argument. It should definitely be handled the same way the corresponding opAssign is handled. Thanks, Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On Thursday, 12 July 2018 at 10:24:40 UTC, Shachar Shemesh wrote: On 29/06/18 15:35, aliak wrote: On Wednesday, 27 June 2018 at 07:24:05 UTC, Mike Parker wrote: On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote: Thanks in advance for your participation. For those of you using the NNTP or mailing list interfaces, this is the thread to respond in. Thanks! Alo! This is great! Just a clarification about the last paragraph phrasing The last line: "We can further reduce this problem by calling the function opPostMove." seemed to imply that an alternate name to opPostMove would be mentioned, but am I correct in understanding that it is just saying that "naming this second function as op* will keep code breakage to a minimum" ? This is a left over from a previous draft, where the operator was called "opMove". It should be removed. Also, what should happen if someone defines an opPostMove for a class. Compile error or? Should something about that be mentioned? I think nothing should happen. The function would be ignored, just like it is today. I am open to hear other ideas, however. I'm not sure whether it should be explicitly mentioned or not. Shachar A postblit on a class issues a compiler error. And an identity opAssign on a class also issues a compiler error. So I'm not sure how this would be different. And the page In https://dlang.org/spec/operatoroverloading.html also explicitly mentions differences between ops on classes or structs. Cheers, - Ali
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 14/07/18 15:56, Johan Engelen wrote: First off: I am trying to wear a strict language lawyer hat. D spec is already very much ill specced which is _very_ problematic for language and compiler development. I am not attacking the proposal in order to kill it. I am merely commenting on points that I feel should be improved. Wouldn't these comments be better suited for the language spec's PR, then? I'm asking seriously. There is nothing in this DIP (or just about any other one) that can go, unmodified, into the specs. If that's the DIP's purpose, then so be it. If not, then I don't see the value. OK, so make _that_ explicit. I think there is value in prescribing the precise order of moves (like for construction of members), such that reasoning about code becomes easier. I disagree. The member variables at the point of the move are all: 1. Fully initialized. 2. Either move agnostic or know how to handle their own move 3. Oblivious to one another Obviously, you might wish to spite me by creating a case where any one of the above is not true, but the compiler has every right to assume all three points above are true. In your hypothetical spite case, your best course of action is to leave the members with no opPostMove at all, and handle their move in the containing struct's opPostMove, in which case you are fully defined. Assuming all three are correct, the order in which they are moved makes no difference. If you want the same semantic effect (as I wrote above), then the text should say that the ordering is relaxed. I have no objection to explicitly stating that exact move order of members is undefined. Why "SHOULD" and not "MUST"? Precisely for the reason you stated above. So that if you want to do something else, you may. Why is that freedom needed? Because compiler implementers might have a good reason to do something besides this. For example, a compiler writer might choose to place the actual moving code inside __move_post_blt, and I don't think we should stop her. The freedom is already provided by user-defined opPostMove? Different audiences. opPostMove serves the D programmer, __move_post_blt serves the compiler. I think the language spec doesn't say when a "move" is performed? I think Walter sees that as an advantage, but I'm not sure. Either way, the current language spec says structs must have semantics that remain correct even if the struct suddenly changes the memory address it resides in. The specs + DIP 1014 say that the above is true, or the struct must supply an opPostMove that fixes the semantics post-move. In both cases, *when* the move takes place is irrelevant. Or is it enough to define what a "move" is ? (didn't check but I guess the DIP already explains that) Only implicitly. (D's "move" is different from C++'s right? Yes. D's move after exiting a struct's constructor doesn't lead to a destructor call, but D's std.algorithm.mutation.move _does_ call the destructor of the moved source object.) Depends on which version of move you're referring to. For example, moveEmplace does not. I think the correct way to phrase this is to say that D's move *never* calls a destructor, but if the move's destination had a valid object in it, that one gets destructed. In a way, C++'s move is the same, except the actual moving of the data from the source location to the destination one is up to the programmer, and accordingly, so is destructing. Since, logically, a C++ move operator always copies, it also has to destruct the source. Technically, however, it doesn't always. A move assignment operator typically just swaps the content of the structs (i.e. - moves the source to the destination and the destination to the source), and lets the usual rvalue elimination code destruct it. Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On Thursday, 12 July 2018 at 10:22:33 UTC, Shachar Shemesh wrote:
On 11/07/18 20:04, Johan Engelen wrote:
On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote:
DIP 1014, "Hooking D's struct move semantics", is now ready
for final review.
after quick read:
(would be much easier to do inline commenting, but it appears
that's not supported)
### Section "Code emitted by the compiler on move"
Dangerous to talk about what "code is emitted" by the
compiler. I think this DIP doesn't need that, and semantics is
enough. "the compiler MUST call " should be reworded, because
an _actual_ call to the function should not be mandatory,
because it would limit the optimizer in eliding it or inlining
it (note that it will be hard to _prevent_ the optimizer from
eliding/inlining the call as currently specified by the DIP).
I don't draw the same conclusions from the text as you.
I'm perfectly fine with specifying that nothing here is
mandatory if the compiler ensures that *the end effect* is as
if it happened.
AFAIK, C++ has a standing order to that effect, and it greatly
simplifies documenting what you want to do.
First off: I am trying to wear a strict language lawyer hat. D
spec is already very much ill specced which is _very_ problematic
for language and compiler development. I am not attacking the
proposal in order to kill it. I am merely commenting on points
that I feel should be improved.
My point was to remove things like "compiler" and "emitted code"
from the DIP / spec. In this case, the simple rewording can be:
"When moving a struct's instance, an implicit call to
__move_post_blt is inserted with both new and old instances'
addresses as arguments."
### "__move_post_blt SHOULD be defined in a manner that is
compatible"
What does "compatible" mean?
"Has the same effect as".
It's a little as if you're complaining about something not
being explicit in one section, and again about that same thing
being explicit in the next. Precisely why such standing order
would be a good idea.
Being explicit about generated machine is not good in a language
spec. Being explicit about the semantic meaning of something
_is_. ;-)
"compatible" is very vague. It may mean that just the function
signature should match.
"has the same semantic effect" would be a much better description
of what you want.
Some things should be made more explicit, such as the order of
calls for compound structs.
I don't think it makes sense to specify the order, except to
say that member's opPostMove must be called before the
instance's opPostMove (which the code already says).
OK, so make _that_ explicit. I think there is value in
prescribing the precise order of moves (like for construction of
members), such that reasoning about code becomes easier.
If you want the same semantic effect (as I wrote above), then the
text should say that the ordering is relaxed.
Why "SHOULD" and not "MUST"?
Precisely for the reason you stated above. So that if you want
to do something else, you may.
Why is that freedom needed? The freedom is already provided by
user-defined opPostMove? I think the implicit call to
__move_post_blt is a MUST, like calls to dtors.
### "This MUST return true iff a struct or any of its members
have an opPostMove defined."
Doesn't cover struct A containing struct B containing struct C
with opPostMove. Reword e.g. into: "hasElaborateMove!S MUST
return true iff `S` has an `opPostMove` defined or if
hasElaborateMove!X is true for any member of S of type X.
Yes, I'm sorry. I worded the spec for humans.
Please don't ;-)
I can suggest a recursive definition:
hasElaborateMove for a struct MUST return true iff at least one
of the following is true:
* The struct has opPostMove defined
* hasElaborateMove returns true for at least one of the
struct's members.
Great.
### What is lacking is a clear list of exactly in which cases
`opPostMove` will be called (needed for user-facing
documentation of the function).
I don't think I understand this point. Can you suggest what
that list might contain?
I think the language spec doesn't say when a "move" is performed?
So I don't know when exactly the opPostMove is called. Things
that come to mind:
* exiting from struct ctor
* std.algorithm.mutation.move
Or is it enough to define what a "move" is ? (didn't check but I
guess the DIP already explains that)
(D's "move" is different from C++'s right? D's move after exiting
a struct's constructor doesn't lead to a destructor call, but D's
std.algorithm.mutation.move _does_ call the destructor of the
moved source object.)
I now realize that the DIP is a mix between language semantic
changes (opPostMove) and implementation suggestions/details
("__move_post_blt"). I think it would have been clearer to split
the two in the DIP (it's valuable to have implementation
suggestions in addition to spec changes), but by now that's too
late. Part of my comments s
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On Thursday, 12 July 2018 at 10:38:52 UTC, Shachar Shemesh wrote: On 12/07/18 04:17, Jonathan M Davis wrote:> I'm also> not sure if going to copy constructors means that we should do something> different with this. It don't think that it's affected by it, but I could be> missing something. I actually had that very same concern myself. Andrei does not seem to share it (I talked to him about it during DConf, and he even mentioned it in his talk). He seems to think the two are completely independent. That's already thought through? Great, no need to "lockstep" the DIPs then and one curve less to worry about.
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 12/07/18 04:17, Jonathan M Davis wrote:> I'm also> not sure if going to copy constructors means that we should do something> different with this. It don't think that it's affected by it, but I could be> missing something. I actually had that very same concern myself. Andrei does not seem to share it (I talked to him about it during DConf, and he even mentioned it in his talk). He seems to think the two are completely independent. Like I said, I'm not sure I completely agree. Some of the problems postblit have will also affect us here (specifically, what happens if you want to override a member's behavior, specifically when @disabled. What tipped the scale for me was this: This DIP is relatively simple. It requires a few changes to all three (compiler, run time and phobos), but small changes to all three. Best of all, this change does not introduce what Andrei called "magic". The changes are easily lowered to existing D code. Switching to a move-constructor type implementation will make all of those advantages disappear in a puff of bits. However, I don't agree that opPostMove needs to be nothrow on the basis that it might be confusing if it's not. The problem here is that D moves *everywhere*, and it is often quite impossible to make it not move (believe me, I've tried). With that said, downgrading this to a very hearty recommendation instead of a requirement is fine by me. Also, as soon as we discuss overriding what happens when an object is moved, that opens up the question of whether it should be allowed to be @disabled. What happens when someone does @disable opPostMove(); What happens today is that if you actually try to move the struct, you will get a compilation error. IMHO, this is the behavior we actually want to keep. I can actually see some uses for such a step - if you simply cannot have that struct move, a compilation error is preferable to things breaking. With that said, the moves are so integral to D's code generation that a struct with opPostMove @disabled would not be very useful. I am of the opinion that the programmer can make up their own mind on what to do about it. However, I would argue that if we're going to take the step of allowing the programmer to hook into the move process to do custom stuff that we should also allow it to be outright disabled - the use case that comes to mind at the moment being stuff like mutexes where the object can't safely be moved, because it's not only not thread-safe, but it's probably not possible with the mutex's C API, since you handed a pointer off to it and have no control over what it does with it. Like I said above, I completely agree. Of course, that also opens the question of what should happen with a shared opPostMove, since presumably a mutex would be in a shared object, and that would then require that we finish working out shared - though I'd argue that we would have to disallow moves on a shared object in order to be thread-safe anyway. That one is actually handled by the DIP. opPostMove is called by the compiler when it has just moved the object. This means that, at least as far as the compiler is concerned, the object has no external references to it (or it couldn't move it). If a struct's pointer is shared, and there are external pointers to update, it is up to the programmer to decide how to handle it (and, yes, @disable might be the only safe option). By no means do I think that this DIP should be contingent on anything to do with disallowing moving, but I do think that it if we're going to allow mucking around with moves like this rather than simply leaving it up to the compiler that we should seriously consider allowing it to be disabled - I disagree. I think we should allow it to be disabled whether this DIP goes in or not. On the contrary: this DIP would allow cases to be handled where today they are simply not safe, no matter what you do. In other words, the cases where you'd want to disable move are only reduced by adopting this DIP. Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 29/06/18 15:35, aliak wrote: On Wednesday, 27 June 2018 at 07:24:05 UTC, Mike Parker wrote: On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote: Thanks in advance for your participation. For those of you using the NNTP or mailing list interfaces, this is the thread to respond in. Thanks! Alo! This is great! Just a clarification about the last paragraph phrasing The last line: "We can further reduce this problem by calling the function opPostMove." seemed to imply that an alternate name to opPostMove would be mentioned, but am I correct in understanding that it is just saying that "naming this second function as op* will keep code breakage to a minimum" ? This is a left over from a previous draft, where the operator was called "opMove". It should be removed. Also, what should happen if someone defines an opPostMove for a class. Compile error or? Should something about that be mentioned? I think nothing should happen. The function would be ignored, just like it is today. I am open to hear other ideas, however. I'm not sure whether it should be explicitly mentioned or not. Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On 11/07/18 20:04, Johan Engelen wrote: On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote: DIP 1014, "Hooking D's struct move semantics", is now ready for final review. after quick read: (would be much easier to do inline commenting, but it appears that's not supported) ### Section "Code emitted by the compiler on move" Dangerous to talk about what "code is emitted" by the compiler. I think this DIP doesn't need that, and semantics is enough. "the compiler MUST call " should be reworded, because an _actual_ call to the function should not be mandatory, because it would limit the optimizer in eliding it or inlining it (note that it will be hard to _prevent_ the optimizer from eliding/inlining the call as currently specified by the DIP). I don't draw the same conclusions from the text as you. I'm perfectly fine with specifying that nothing here is mandatory if the compiler ensures that *the end effect* is as if it happened. AFAIK, C++ has a standing order to that effect, and it greatly simplifies documenting what you want to do. ### "__move_post_blt SHOULD be defined in a manner that is compatible" What does "compatible" mean? "Has the same effect as". It's a little as if you're complaining about something not being explicit in one section, and again about that same thing being explicit in the next. Precisely why such standing order would be a good idea. Some things should be made more explicit, such as the order of calls for compound structs. I don't think it makes sense to specify the order, except to say that member's opPostMove must be called before the instance's opPostMove (which the code already says). Why "SHOULD" and not "MUST"? Precisely for the reason you stated above. So that if you want to do something else, you may. ### "This MUST return true iff a struct or any of its members have an opPostMove defined." Doesn't cover struct A containing struct B containing struct C with opPostMove. Reword e.g. into: "hasElaborateMove!S MUST return true iff `S` has an `opPostMove` defined or if hasElaborateMove!X is true for any member of S of type X. Yes, I'm sorry. I worded the spec for humans. I can suggest a recursive definition: hasElaborateMove for a struct MUST return true iff at least one of the following is true: * The struct has opPostMove defined * hasElaborateMove returns true for at least one of the struct's members. I'm fine with this definition, and it resolves your concern, but I think it is less readable. You are free to suggest a better way of phrasing this. ### What is lacking is a clear list of exactly in which cases `opPostMove` will be called (needed for user-facing documentation of the function). I don't think I understand this point. Can you suggest what that list might contain? Thank you, Shachar
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On Wednesday, 27 June 2018 01:13:14 MDT Mike Parker via Digitalmars-d wrote: > DIP 1014, "Hooking D's struct move semantics", is now ready for > final review. This is a last chance for community feedback before > the DIP is handed off to Walter and Andrei for the Formal > Assessment. Please read the procedures document for details on > what is expected in this review stage: > > https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#final-review > > The current revision of the DIP for this review is located here: > > https://github.com/dlang/DIPs/blob/01af4bb1e81993066bd1e7f77263002d3883fab > f/DIPs/DIP1014.md > > In it you'll find a link to and summary of the previous review > round. This round of review will continue until 11:59 pm ET on > July 11th unless I call it off before then. > > Thanks in advance for your participation. Looks pretty good overall, though the name of the __move_post_blt arguably should be different if we're going to move to copy constructors. I'm also not sure if going to copy constructors means that we should do something different with this. It don't think that it's affected by it, but I could be missing something. However, I don't agree that opPostMove needs to be nothrow on the basis that it might be confusing if it's not. Based on that argument, postblit constructors should be required to be nothrow, and they don't have to be nothrow. Copying and moving frequently look pretty much the same to the programmer and may even depend on how the compiler is able to optimize the code (e.g. it figures out that it can use a move instead of a copy). So, while it probably is generally better for opPostMove to be nothrow, it doesn't seem to me like it really should be required that it be nothrow. For better or worse, we don't even require that destructors be nothrow. So, it doesn't really fit for opPostMove to have to be nothrow. The motivation for it is not at all technical in nature and the social aspect of it is already contradicted by similar features which are allowed to throw. Also, as soon as we discuss overriding what happens when an object is moved, that opens up the question of whether it should be allowed to be @disabled. What happens when someone does @disable opPostMove(); For better or worse, @disable works on any function in D (even if it's kind of pointless for functions that aren't constructors or operators). So, it's going to need to be handled even if it's just making it an error to @disable it. However, I would argue that if we're going to take the step of allowing the programmer to hook into the move process to do custom stuff that we should also allow it to be outright disabled - the use case that comes to mind at the moment being stuff like mutexes where the object can't safely be moved, because it's not only not thread-safe, but it's probably not possible with the mutex's C API, since you handed a pointer off to it and have no control over what it does with it. Of course, that also opens the question of what should happen with a shared opPostMove, since presumably a mutex would be in a shared object, and that would then require that we finish working out shared - though I'd argue that we would have to disallow moves on a shared object in order to be thread-safe anyway. So, maybe the mutex use case doesn't ultimately matter, since we're likely going to have to disable moves for shared anyway, but shared and mutexes aside, any use case where you hand the pointer off to something outside the object won't work with opPostMove, whereas it could be made to work if we had opPostMove and were then allowed to @disable it. By no means do I think that this DIP should be contingent on anything to do with disallowing moving, but I do think that it if we're going to allow mucking around with moves like this rather than simply leaving it up to the compiler that we should seriously consider allowing it to be disabled - especially when opPostMove gives us a really obvious syntax for it, and while some of the motivations for this DIP are solved by it, others would really require disallowing moves. - Jonathan M Davis
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote: DIP 1014, "Hooking D's struct move semantics", is now ready for final review. after quick read: (would be much easier to do inline commenting, but it appears that's not supported) ### Section "Code emitted by the compiler on move" Dangerous to talk about what "code is emitted" by the compiler. I think this DIP doesn't need that, and semantics is enough. "the compiler MUST call " should be reworded, because an _actual_ call to the function should not be mandatory, because it would limit the optimizer in eliding it or inlining it (note that it will be hard to _prevent_ the optimizer from eliding/inlining the call as currently specified by the DIP). So this should be reworded such that the semantic effect is as if the function is called. Also unspecified _when_ the function needs to be called. ### "__move_post_blt SHOULD be defined in a manner that is compatible" What does "compatible" mean? Some things should be made more explicit, such as the order of calls for compound structs. Why "SHOULD" and not "MUST"? ### "This MUST return true iff a struct or any of its members have an opPostMove defined." Doesn't cover struct A containing struct B containing struct C with opPostMove. Reword e.g. into: "hasElaborateMove!S MUST return true iff `S` has an `opPostMove` defined or if hasElaborateMove!X is true for any member of S of type X. ### What is lacking is a clear list of exactly in which cases `opPostMove` will be called (needed for user-facing documentation of the function). -Johan
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On Thursday, 5 July 2018 at 10:27:52 UTC, Dukc wrote: The DIP looks well written. I'm in favour of it. However, we need to consider how well this interacts with Razvan's DIP [1]. We should consider this together with it, so the implementations do not end up messing each other. 1: https://forum.dlang.org/thread/hhdtliynfwckcgafl...@forum.dlang.org
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote: DIP 1014, "Hooking D's struct move semantics", is now ready for final review. Structs are a low level feature that should be able to be used in any way programmer sees fit. This is just what is wrong with C# structs: In principle they're the same as D structs but disallow many things for no obvious reason, thus limiting their usability. See my question and it's answers at Stack overflow to see what I mean: https://stackoverflow.com/questions/51098690/assigning-value-to-member-of-nullable-struct-in-c-sharp This is a feature that is likely to be useful for low level programming, but is zero cost for those who don't need it. The DIP looks well written. I'm in favour of it.
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On Wednesday, 27 June 2018 at 07:24:05 UTC, Mike Parker wrote: On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote: Thanks in advance for your participation. For those of you using the NNTP or mailing list interfaces, this is the thread to respond in. Thanks! Alo! This is great! Just a clarification about the last paragraph phrasing The last line: "We can further reduce this problem by calling the function opPostMove." seemed to imply that an alternate name to opPostMove would be mentioned, but am I correct in understanding that it is just saying that "naming this second function as op* will keep code breakage to a minimum" ? Also, what should happen if someone defines an opPostMove for a class. Compile error or? Should something about that be mentioned? Cheers - Ali
Re: DIP 1014--Hooking D's struct move semantics--Final Review
On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote: Thanks in advance for your participation. For those of you using the NNTP or mailing list interfaces, this is the thread to respond in. Thanks!
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 18/05/18 22:57, kinke wrote: I checked, and the reason is that D and C++ use a different ABI wrt. by-value passing of non-POD arguments. C++ indeed passes a reference to a caller-allocated rvalue, not just on Win64; that makes it trivial, as there are no moves across call boundaries. But your proposal may imply changing the D ABI accordingly. That seems to be the case. Assuming https://dlang.org/spec/abi.html is the ABI you refer to, there is very little in way of argument calling there: https://dlang.org/spec/abi.html#function_calling_conventions " The extern (C) and extern (D) calling convention matches the C calling convention used by the supported C compiler on the host system. Except that the extern (D) calling convention for Windows x86 is described here. " So the current state is, in essence, that in C everything is PoD, and that's why that's also the case in D. Yes, something will need to change there. Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On Thursday, 17 May 2018 at 19:11:27 UTC, Shachar Shemesh wrote: On 17/05/18 18:47, kinke wrote: Since clang is able to compile this struct and do everything with it, and since the existence of the move constructor requires the precise same type of hooking as is needed in this case, I tend to believe that an IR representation of DIP 1014 is possible. I checked, and the reason is that D and C++ use a different ABI wrt. by-value passing of non-POD arguments. C++ indeed passes a reference to a caller-allocated rvalue, not just on Win64; that makes it trivial, as there are no moves across call boundaries. But your proposal may imply changing the D ABI accordingly.
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On Thursday, 17 May 2018 at 20:25:26 UTC, Shachar Shemesh wrote: Obviously, Something can be an enum or a boolean. If it is, however, then we have to perform a condition to select the correct value. The problem with conditionals is that if the CPU misses a guess about what they are (and in our case, the CPU is going to miss about 50% of the time), they are extremely expensive to evaluate. Performance wise, a much saner approach is: alias Something = int*; Of course, this means our struct now has a self referencing pointer. What I'm getting at is that even if there are alternatives to structs pointing at themselves, they may not be performance wise comparable to pointers. It's possible to do a branchless condition that chooses between two pointers. I think if the hardware (and compiler) support it it'll just optimize down to a "cmov".
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On Thursday, 17 May 2018 at 19:13:48 UTC, Shachar Shemesh wrote: The only inherent non @safe thing we advocate here is if you want to be able to move const/immutable structs, in which case DIP 1014 advocates casting the constness away. That will, of course, have to be either @system or @trusted. There's an idea to give const postblit ability to write, but it's difficult to make such exception for operator method, a possible syntax can be `this(this, const ref S prev)`
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17 May 2018 at 13:25, Shachar Shemesh via Digitalmars-d
wrote:
> On 17/05/18 22:29, Manu wrote:
>>
>>
>> This is great!
>> I've wanted this on numerous occasions when interacting with C++ code.
>> This will make interaction more complete.
>>
>> Within self-contained D code, I have avoided self-pointers by using
>> self-offsets instead in the past (a bit hack-ey). But this nicely
>> tidies up one more little rough-edge in the language.
>>
>> :+1:
>>
>
> Following Andrei's advice, I've actually started writing a couple of
> examples to illustrate why this is needed.
>
> The first was this:
>
> struct S {
> static int global;
> int local;
>
> Something selector; // Decides whether we want local or global.
> }
>
> Let's further assume that we have an array of S instances with random
> uniform distribution between local and global.
>
> Obviously, Something can be an enum or a boolean. If it is, however, then we
> have to perform a condition to select the correct value. The problem with
> conditionals is that if the CPU misses a guess about what they are (and in
> our case, the CPU is going to miss about 50% of the time), they are
> extremely expensive to evaluate.
>
> Performance wise, a much saner approach is:
> alias Something = int*;
>
> Of course, this means our struct now has a self referencing pointer.
>
> What I'm getting at is that even if there are alternatives to structs
> pointing at themselves, they may not be performance wise comparable to
> pointers.
>
> Of course, the second example was a struct that has no internal pointers,
> but rather maintains global pointers pointing to it. This problem is quite a
> bit harder to solve.
>
> Shachar
Oh yeah, I totally recognise that there are instances where a local
offset is not a solution, I was just saying how I've managed to
avoided dealing with this issue before, and not that I loved doing so
that way ;)
This would be a very welcome fix, particularly in code where I
interact with C++!
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 22:29, Manu wrote:
This is great!
I've wanted this on numerous occasions when interacting with C++ code.
This will make interaction more complete.
Within self-contained D code, I have avoided self-pointers by using
self-offsets instead in the past (a bit hack-ey). But this nicely
tidies up one more little rough-edge in the language.
:+1:
Following Andrei's advice, I've actually started writing a couple of
examples to illustrate why this is needed.
The first was this:
struct S {
static int global;
int local;
Something selector; // Decides whether we want local or global.
}
Let's further assume that we have an array of S instances with random
uniform distribution between local and global.
Obviously, Something can be an enum or a boolean. If it is, however,
then we have to perform a condition to select the correct value. The
problem with conditionals is that if the CPU misses a guess about what
they are (and in our case, the CPU is going to miss about 50% of the
time), they are extremely expensive to evaluate.
Performance wise, a much saner approach is:
alias Something = int*;
Of course, this means our struct now has a self referencing pointer.
What I'm getting at is that even if there are alternatives to structs
pointing at themselves, they may not be performance wise comparable to
pointers.
Of course, the second example was a struct that has no internal
pointers, but rather maintains global pointers pointing to it. This
problem is quite a bit harder to solve.
Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17 May 2018 at 01:12, Mike Parker via Digitalmars-d wrote: > This is the review thread for the first Community Review round for DIP 1014, > "Hooking D's struct move semantics". > > All review-related feedback on and discussion of the DIP should occur in > this thread. The review period will end at 11:59 PM ET on May 31, or when I > make a post declaring it complete. > > At the end of Round 1, if further review is deemed necessary, the DIP will > be scheduled for another round. Otherwise, it will be queued for the formal > review and evaluation by the language maintainers. > > Please familiarize yourself with the documentation for the Community Review > before participating. > > https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review > > Thanks in advance to all who participate. > > Destroy! This is great! I've wanted this on numerous occasions when interacting with C++ code. This will make interaction more complete. Within self-contained D code, I have avoided self-pointers by using self-offsets instead in the past (a bit hack-ey). But this nicely tidies up one more little rough-edge in the language. :+1:
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 19:08, Kagamin wrote: On Thursday, 17 May 2018 at 13:50:26 UTC, Shachar Shemesh wrote: There is no such use case. Please remember that at the time opPostMove is called, both new and old memory are still allocated. That's an interesting moment too. A struct that was supposed to be moved is copied instead and exists in two places simultaneously. Can't tell it yet, but it can have a hole in type system and opPostMove can only be trusted, not safe. It is a hole (of sorts) in the type system, in that "old" is not going to have a destructor run on its code. With that said, just because the code is not safe, does not mean it is not @safe. The only inherent non @safe thing we advocate here is if you want to be able to move const/immutable structs, in which case DIP 1014 advocates casting the constness away. That will, of course, have to be either @system or @trusted. Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 18:47, kinke wrote:
On Thursday, 17 May 2018 at 15:23:50 UTC, kinke wrote:
See IR for https://run.dlang.io/is/1JIsk7.
I should probably emphasize that the LLVM `byval` attribute is strange
at first sight. Pseudo-IR `void foo(S* byval param); ... foo(S* byarg
arg);` doesn't mean that the IR callee gets the S* pointer from the IR
callsite; it means 'memcpy(param, arg, S.sizeof)', with `param` being an
*implicit* address in foo's parameters stack (calculated by LLVM and so
exposed to the callee only). That's the difficulty for LDC I mentioned
earlier.
I understand there might be difficulty, but I strongly protest the idea
that it is not possible, for one very simple reason: C++.
class Movable {
int member;
public:
Movable();
Movable( const Movable &rhs ); // Copy constructor
Movable( Movable &&rhs ); // Move constructor
}
Since clang is able to compile this struct and do everything with it,
and since the existence of the move constructor requires the precise
same type of hooking as is needed in this case, I tend to believe that
an IR representation of DIP 1014 is possible.
Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On Thursday, 17 May 2018 at 13:50:26 UTC, Shachar Shemesh wrote: There is no such use case. Please remember that at the time opPostMove is called, both new and old memory are still allocated. That's an interesting moment too. A struct that was supposed to be moved is copied instead and exists in two places simultaneously. Can't tell it yet, but it can have a hole in type system and opPostMove can only be trusted, not safe.
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On Thursday, 17 May 2018 at 15:23:50 UTC, kinke wrote: See IR for https://run.dlang.io/is/1JIsk7. I should probably emphasize that the LLVM `byval` attribute is strange at first sight. Pseudo-IR `void foo(S* byval param); ... foo(S* byarg arg);` doesn't mean that the IR callee gets the S* pointer from the IR callsite; it means 'memcpy(param, arg, S.sizeof)', with `param` being an *implicit* address in foo's parameters stack (calculated by LLVM and so exposed to the callee only). That's the difficulty for LDC I mentioned earlier.
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On Thursday, 17 May 2018 at 12:36:29 UTC, Shachar Shemesh wrote:
Again, as far as I know, structs are not copied when passed as
arguments. They are allocated on the caller's stack and a
reference is passed to the callee. If that's the case, no move
(of any kind) is done.
That's the exception to the rule (LDC's `ExplicitByvalRewrite`),
and true for structs > 64 bit on Win64 (and some more structs)
and something similar for AArch64. No other ABIs supported by LDC
pass a low-level pointer to a caller-allocated copy for
high-level pass-argument-by-value semantics; the argument is
normally moved to the function parameter (in the callEE
parameters stack).
```
struct S
{
size_t a, b;
this(this) {} // no POD anymore
}
void foo(S param);
void bar()
{
// allocate a temporary on the caller's stack and move it to
the callee
foo(S(1, 2));
S lvalue;
// copy lvalue to a temporary on the caller's stack (incl.
postblit call)
// and then move that temporary to the callee
foo(lvalue);
import std.algorithm.mutation : move;
// move move()-rvalue-result to the callee
foo(move(lvalue));
}
```
'Move to callee' for most ABIs meaning a bitcopy/blit to the
callee's memory parameters stack, for LDC via LLVM `byval`
attribute.
See IR for https://run.dlang.io/is/1JIsk7.
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 16:42, Kagamin wrote: Looks like requirement for @nogc @safe has no consequence as the DIP suggests to infer them anyway. On ideological side safety can't be a requirement because it would contradict its purpose of providing guarantee. I think you are confusing __move_post_blt's implementation (by druntime) with opPostMove's implementation by the user. For the former, these attributes are deducted by the compiler. For the later, the user may choose to include them for all of the usual reasons for including them, not least of which is that if she does not include @nogc, then trying to move a struct's instance from @nogc code will cause compilation failure. > Especially if the suggested use case is handling of dangling > pointers. There is no such use case. Please remember that at the time opPostMove is called, both new and old memory are still allocated. Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
Looks like requirement for @nogc @safe has no consequence as the DIP suggests to infer them anyway. On ideological side safety can't be a requirement because it would contradict its purpose of providing guarantee. Especially if the suggested use case is handling of dangling pointers.
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
I'm not sure I follow all of your comments.
For the rest my comments, let's assume that the compiler may assume that
__move_post_blt is a no-op if hasElaborateMove returns false.
On 17/05/18 14:33, kinke wrote:
3. When deciding to move a struct instance, the compiler MUST emit a
call to the struct's __move_post_blt after blitting the instance and
before releasing the memory containing the old instance.
__move_post_blt MUST receive references to both the pre- and post-move
instances.
This implies that such structs must not be considered PODs, i.e., cannot
be passed in registers and must be passed on the stack.
I'm not familiar with passing structs in registers. I am familiar with
passing pointer to the structs in registers, which should not be
affected by this.
If actual (I'm assuming short) structs can be passed in registers, then,
yes, structs with elaborate move are not PoDs.
It also means
that the compiler will have to insert a __move_post_blt call right
before the call (as the callee has no idea about the old address),
Again, as far as I know, structs are not copied when passed as
arguments. They are allocated on the caller's stack and a reference is
passed to the callee. If that's the case, no move (of any kind) is done.
I might be missing something. Can you write a code snippet to which you
are referring?
after
blitting the arg to the callee params stack; this may be tricky to
implement for LDC, as that last blit is implicit in LLVM IR (LLVM byval
attribute).
And yet, C++ clang managed to do it in classes with && constructors.
As a side note, when passing a postblit-struct lvalue arg by value,
Just to be clear, are we talking about passing by reference an instance
of a struct that has postblit?
the
compiler first copies the lvalue
to a temporary on the caller's stack,
incl. postblit call, and then moves that copy to the callee. So this
requires either a postblit+postmove combo on the caller side before the
actual call, or a single postblit call for the final address (callee's
param).
Again, that does not align with my familiarity of the ABI. If I do:
struct S {
...
this(this) {
// some code
}
void opPostMove(ref S new, const ref S old) {
// some code
}
}
void func1(ref S arg) {
}
void func2(S arg) {
}
As far as I know the ABI, in func1, a pointer to S is passed. In func2,
a pointer to caller stack allocate instance is passed, and the original
is copied in. It sounds to me like you are talking about the case of:
S s;
func2(s);
in which case you need to copy s to a temporary, and then pass a pointer
to that temporary to func2. I don't see where a move enters here.
However, if a move does enter here (and one is necessary if, for
example, func2's frame needs to be dynamically allocated because an
escaping delegate references it), then, yes, an opPostMove will also
need to be called.
Again, if hasElaborateMove returns false, then no change from current
behavior is needed.
Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
3. When deciding to move a struct instance, the compiler MUST emit a call to the struct's __move_post_blt after blitting the instance and before releasing the memory containing the old instance. __move_post_blt MUST receive references to both the pre- and post-move instances. This implies that such structs must not be considered PODs, i.e., cannot be passed in registers and must be passed on the stack. It also means that the compiler will have to insert a __move_post_blt call right before the call (as the callee has no idea about the old address), after blitting the arg to the callee params stack; this may be tricky to implement for LDC, as that last blit is implicit in LLVM IR (LLVM byval attribute). As a side note, when passing a postblit-struct lvalue arg by value, the compiler first copies the lvalue to a temporary on the caller's stack, incl. postblit call, and then moves that copy to the callee. So this requires either a postblit+postmove combo on the caller side before the actual call, or a single postblit call for the final address (callee's param).
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/18 11:22, rikki cattermole wrote: What is the benefit of opPostMove over copy constructors (not postblit)? The two are unrelated. A copy is a very different operation from a move. With a copy, you have to figure out how to duplicate the resources used by the object. With a move, no such duplication is needed. People are somewhat conditioned to treat a move as a "copy+destroy". One certainly may implement it that way. There is a lot of power in not having to do it, though. Think a struct with "@disable this(this)". The D compiler does moves of structs, whether they are copyable or not. This DIP is about being able to hook this move and fix external and internal references. Shachar
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On 17/05/2018 8:12 PM, Mike Parker wrote: This is the review thread for the first Community Review round for DIP 1014, "Hooking D's struct move semantics". All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on May 31, or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language maintainers. Please familiarize yourself with the documentation for the Community Review before participating. https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review Thanks in advance to all who participate. Destroy! What is the benefit of opPostMove over copy constructors (not postblit)?
Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1
On Thursday, 17 May 2018 at 08:12:50 UTC, Mike Parker wrote: This is the review thread for the first Community Review round for DIP 1014, "Hooking D's struct move semantics". And the link to the DIP: https://github.com/dlang/DIPs/blob/38cec74a7471735559e3b8a7553f55102d289d28/DIPs/DIP1014.md
