Re: Deprecating this(this)
On Wednesday, 4 April 2018 at 10:37:57 UTC, Steven Schveighoffer wrote: Any place where an immutable can be observed to change between two reads is breaking immutable. That happens in constructor too.
Re: Deprecating this(this)
On Wednesday, 4 April 2018 at 10:37:57 UTC, Steven Schveighoffer
wrote:
I would like to see more flexibility for copying.
It's a tradeoff between control and ergonomics.
import std.stdio;
immutable(int)* p;
struct S
{
int x;
this(this) immutable
{
x = 42; /* First write. */
.p = &this.x;
writeln(p, " ", *p); /* Prints some address and 42. */
}
}
struct T
{
S s;
this(this) immutable
{
s = S(13); /* Second write. Breaking immutable? */
Of course this doesn't compile, because s is considered
immutable by now. What I was saying is that we can't allow
postblit to modify data that has already been postblitted,
because of the reason this example is showing.
Assignment ends lifetime of the previous value, it will be just a
dangling pointer.
AFAIK this is how reassignment works wrt destructors:
{
immutable tmp=S(13);
s.__dtor(); //if any
s=tmp; //move into immutable storage
}
I don't think you should be able to write to the same field
multiple times in an immutable/const constructor. If so, that's
a bug.
Stores to const fields are special, so it should be possible to
track them in postblits too.
Re: Deprecating this(this)
On Thursday, 5 April 2018 at 18:46:25 UTC, H. S. Teoh wrote:
I like this idea. Except the syntax could be improved:
this(this orig) // <-- N.B.
{
It's already valid syntax, equivalent to `this(typeof(this)
orig)`. It can be called explicitly, but is ignored for implicit
construction. Perhaps we will just enable `this(typeof(this)
orig)` for implicit construction too, if there's no postblit
defined.
Re: Deprecating this(this)
On Thursday, April 05, 2018 11:46:25 H. S. Teoh via Digitalmars-d wrote:
> On Wed, Apr 04, 2018 at 05:08:26PM -0600, Jonathan M Davis via
> Digitalmars-d wrote: [...]
> > Hmmm. And actually, thinking about that, I almost wonder if we could
> > just change this(this) to be a copy constructor. Assume for a moment
> > that inside this(this), we provide a new symbol that is the this
> > pointer/reference for the original. We then have make it so that any
> > member variable in the newly constructed object which is read before
> > it is assigned is default-copied. As such,
> >
> > this(this)
> > {
> > }
> >
> > would just default-copy everything basically as it does now.
>
> Why even bother with declaring an empty this(this)? Just leave it out,
> and the compiler assumes default-copy.
You wouldn't declare it. I was just talking about it to discuss the
semantics.
> > However, if we provide the object being copied via an invisible ref
> > (like this) called orig, doing
> >
> > this(this)
> > {
> >
> > _foo = orig._foo;
> >
> > }
> >
> > then that would directly initialize _foo without it being
> > default-copied. As such, existing postblit constructors should
> > continue to work, but it would be straightforward to turn them into
> > copy constructors, and most of the changes would be underneath the
> > hood.
>
> [...]
>
> I like this idea. Except the syntax could be improved:
>
> this(this orig) // <-- N.B.
> {
> _foo = orig._foo;
> }
>
> Let the user specify the symbol, instead of introducing yet another
> implicit magic identifier. :-)
Well, that might be better in that it allows the user to choose the name,
but it then opens up the question of whether this(this) should be
deprecated, and part of the point was to try and just do it all in-place and
simply turn postblit constructors into copy constructors - though it could
be argued that this(this) was just a copy constructor where you didn't
bother to use the original, and anyone actually wanting to take advantage of
copy constructors is going to need to make changes anyway. But if this(this)
simply becomes a copy constructor (whether a default name is used or
something like this(this orig) is used to define the name), then we
hopefully would be able to avoid actually deprecating anything and just make
more code work and making it so that only minor tweaks are necessary to
actually take full advantage of it being a copy constructor instead of
mutating the copy in the copy constructor.
The other issue that occurred to me after posting about this was the
question of what to do about const. Ideally, this(this) or this(this orig)
would work just fine with const (assuming that it didn't do anything that
would mutate the object), but someone might want to actually require const
or overload the copy constructor for the const case, in which case, I was
thinking making this(const this), but with your suggestion, I guess that it
would become either this(const this orig) or this(this const orig). I don't
know. As usual, adding const into the mix makes things messier, but part of
the point of all of this is to make it so that we can copy arbitrarily
complex const objects.
In any case, I think that the idea of trying to simply turn postblit
constructors into copy constructors has merit. I certainly don't think that
we need to be doing anything like
struct S
{
this(ref S orig)
{
...
}
}
where everyone will have to convert their postblits over time in order to
avoid the deprecation and where it actually risks conflicting with existing
constructors.
- Jonathan M Davis
Re: Deprecating this(this)
On Wed, Apr 04, 2018 at 05:08:26PM -0600, Jonathan M Davis via Digitalmars-d
wrote:
[...]
> AFAIK, the only real advantage to a postblit constructor is that you
> don't have to explicitly copy everything like you do when declaring a
> copy constructor in C++ - you just change the parts that need to be
> changed after the copy is made in order to make it a deep copy or do
> whatever you need to do to finish making the copy what it should be.
> As such, if we define copy constructors in a way that only requires
> listing the members that are actually going to be different from the
> default copy, then I really don't see an advantage to postblit
> constructors.
That's a very interesting concept. It could be a nice way of bridging
current semantics over to the new semantics. Some of the existing
postblit code might even be usable as-is, when reinterpreted in this
way!
> Hmmm. And actually, thinking about that, I almost wonder if we could
> just change this(this) to be a copy constructor. Assume for a moment
> that inside this(this), we provide a new symbol that is the this
> pointer/reference for the original. We then have make it so that any
> member variable in the newly constructed object which is read before
> it is assigned is default-copied. As such,
>
> this(this)
> {
> }
>
> would just default-copy everything basically as it does now.
Why even bother with declaring an empty this(this)? Just leave it out,
and the compiler assumes default-copy.
> And if you currently had
>
> this(this)
> {
> _foo = _foo.dup;
> }
>
> then after the change, it would still have the same semantics, because
> the _foo is used before it is assigned, so it must be default-copied.
That's a pretty neat way of transitioning from current semantics to new
semantics. I like it.
> However, if we provide the object being copied via an invisible ref
> (like this) called orig, doing
>
> this(this)
> {
> _foo = orig._foo;
> }
>
> then that would directly initialize _foo without it being
> default-copied. As such, existing postblit constructors should
> continue to work, but it would be straightforward to turn them into
> copy constructors, and most of the changes would be underneath the
> hood.
[...]
I like this idea. Except the syntax could be improved:
this(this orig) // <-- N.B.
{
_foo = orig._foo;
}
Let the user specify the symbol, instead of introducing yet another
implicit magic identifier. :-)
T
--
"I suspect the best way to deal with procrastination is to put off the
procrastination itself until later. I've been meaning to try this, but
haven't gotten around to it yet. " -- swr
Re: Deprecating this(this)
On Tuesday, 3 April 2018 at 11:36:27 UTC, ag0aep6g wrote: Maybe. But we can't explain the special assignment semantics with it being initialization. Or can we? Postblit is a part of initialization of a copy. Until postblit completes the object is in invalid state, same as in constructor.
Re: Deprecating this(this)
On Wednesday, April 04, 2018 15:51:25 H. S. Teoh via Digitalmars-d wrote:
> On Wed, Apr 04, 2018 at 03:30:39PM -0700, Walter Bright via Digitalmars-d
wrote:
> > On 4/1/2018 3:49 AM, bachmeier wrote:
> > > What I was wondering too. I mean, breaking changes just don't happen
> > > to this language. Now there will be, without even an indication of
> > > how existing code would have to be rewritten, or how this
> > > large-scale breakage is different than the breakages that just can't
> > > happen because reasons. I guess that's why there's always the
> > > disclaimer, "We'll only break code if there's a really good reason."
> > > That reason is "in case we want to".
> >
> > The idea is not to introduce a breaking change. Postblits will remain.
> > The copy constructor will be an additional construct, and if one is
> > defined, it will be preferred.
>
> This may lead to trouble, unless we explicitly make it illegal for
> something to have both a postblit and a copy ctor. Having the two
> interact with each other seems likely to lead to a lot of pathological,
> hard-to-understand results.
+1
> > Eventually (years later) postblits will be slowly deprecated and
> > eventually removed.
>
> Actually, if it's possible to have them coexist and there are no
> disadvantages to doing so, why not just leave both of them in? Then
> cases where postblits give the best performance could just be left
> as-is, and cases that require a copy ctor can use a copy ctor.
The big disadvantage is that you would have to deal with two distinct ways
to copy objects and would have to explain them both to everyone learnig D.
AFAIK, the only real advantage to a postblit constructor is that you don't
have to explicitly copy everything like you do when declaring a copy
constructor in C++ - you just change the parts that need to be changed after
the copy is made in order to make it a deep copy or do whatever you need to
do to finish making the copy what it should be. As such, if we define copy
constructors in a way that only requires listing the members that are
actually going to be different from the default copy, then I really don't
see an advantage to postblit constructors.
Hmmm. And actually, thinking about that, I almost wonder if we could just
change this(this) to be a copy constructor. Assume for a moment that inside
this(this), we provide a new symbol that is the this pointer/reference for
the original. We then have make it so that any member variable in the newly
constructed object which is read before it is assigned is default-copied. As
such,
this(this)
{
}
would just default-copy everything basically as it does now. And if you
currently had
this(this)
{
_foo = _foo.dup;
}
then after the change, it would still have the same semantics, because the
_foo is used before it is assigned, so it must be default-copied. However,
if we provide the object being copied via an invisible ref (like this)
called orig, doing
this(this)
{
_foo = orig._foo;
}
then that would directly initialize _foo without it being default-copied. As
such, existing postblit constructors should continue to work, but it would
be straightforward to turn them into copy constructors, and most of the
changes would be underneath the hood.
I don't know if orig is the best name (and making it a keyword would be
really annoying, since it's a name that I use all the time), but on the
surface at least, the idea seems sound to me. And assuming that there isn't
a big implementation issue that makes it a serious problem, it should
actually make transitioning from postblit constructors to copy constructors
very clean and negate the need for any kind of deprecation process.
- Jonathan M Davis
Re: Deprecating this(this)
On Wednesday, April 04, 2018 22:41:39 Stefan Koch via Digitalmars-d wrote: > On Wednesday, 4 April 2018 at 22:30:39 UTC, Walter Bright wrote: > > On 4/1/2018 3:49 AM, bachmeier wrote: > >> What I was wondering too. I mean, breaking changes just don't > >> happen to this language. Now there will be, without even an > >> indication of how existing code would have to be rewritten, or > >> how this large-scale breakage is different than the breakages > >> that just can't happen because reasons. I guess that's why > >> there's always the disclaimer, "We'll only break code if > >> there's a really good reason." That reason is "in case we want > >> to". > > > > The idea is not to introduce a breaking change. Postblits will > > remain. The copy constructor will be an additional construct, > > and if one is defined, it will be preferred. > > > > Eventually (years later) postblits will be slowly deprecated > > and eventually removed. > > Could you describe how a copy constructor differs from postblit ? With a postblit, all of the members are copied _before_ the postblit constructor is called, meaning that the struct was blitted, and then the postblit constructors were called for each member variable that has a postblit constructor, before the struct's postblit constructor is called. So, inside the postblit constructor, you're mutating an already copied object. It's doing work post the copy - hence postblit. With a copy constructor, the copy is being directly constructed. Each member will be initialized exactly once rather than being initialized and then mutated. Ideally, each member that was not explicitly initialized by the copy constructor would be automatically copied from the original so that you wouldn't have to explicitly copy each member, so syntactically, it may look very much like a postblit constructor, but the key difference is that with a copy constructor, the copy is directly constructed with whatever changes are done by the copy constructor as opposed to having it copied and then mutated. - Jonathan M Davis
Re: Deprecating this(this)
On Wed, Apr 04, 2018 at 03:30:39PM -0700, Walter Bright via Digitalmars-d wrote: > On 4/1/2018 3:49 AM, bachmeier wrote: > > What I was wondering too. I mean, breaking changes just don't happen > > to this language. Now there will be, without even an indication of > > how existing code would have to be rewritten, or how this > > large-scale breakage is different than the breakages that just can't > > happen because reasons. I guess that's why there's always the > > disclaimer, "We'll only break code if there's a really good reason." > > That reason is "in case we want to". > > The idea is not to introduce a breaking change. Postblits will remain. > The copy constructor will be an additional construct, and if one is > defined, it will be preferred. This may lead to trouble, unless we explicitly make it illegal for something to have both a postblit and a copy ctor. Having the two interact with each other seems likely to lead to a lot of pathological, hard-to-understand results. > Eventually (years later) postblits will be slowly deprecated and > eventually removed. Actually, if it's possible to have them coexist and there are no disadvantages to doing so, why not just leave both of them in? Then cases where postblits give the best performance could just be left as-is, and cases that require a copy ctor can use a copy ctor. T -- Klein bottle for rent ... inquire within. -- Stephen Mulraney
Re: Deprecating this(this)
On Wednesday, 4 April 2018 at 22:30:39 UTC, Walter Bright wrote: On 4/1/2018 3:49 AM, bachmeier wrote: What I was wondering too. I mean, breaking changes just don't happen to this language. Now there will be, without even an indication of how existing code would have to be rewritten, or how this large-scale breakage is different than the breakages that just can't happen because reasons. I guess that's why there's always the disclaimer, "We'll only break code if there's a really good reason." That reason is "in case we want to". The idea is not to introduce a breaking change. Postblits will remain. The copy constructor will be an additional construct, and if one is defined, it will be preferred. Eventually (years later) postblits will be slowly deprecated and eventually removed. Could you describe how a copy constructor differs from postblit ?
Re: Deprecating this(this)
On 4/1/2018 3:49 AM, bachmeier wrote: What I was wondering too. I mean, breaking changes just don't happen to this language. Now there will be, without even an indication of how existing code would have to be rewritten, or how this large-scale breakage is different than the breakages that just can't happen because reasons. I guess that's why there's always the disclaimer, "We'll only break code if there's a really good reason." That reason is "in case we want to". The idea is not to introduce a breaking change. Postblits will remain. The copy constructor will be an additional construct, and if one is defined, it will be preferred. Eventually (years later) postblits will be slowly deprecated and eventually removed.
Re: Deprecating this(this)
On 04/04/2018 12:37 PM, Steven Schveighoffer wrote:
With structs, we have the possibility of initialization via different
mechanisms: constructor, postblit, .init. All of these are supported by
the struct member, but currently you can only invoke postblit if you are
in a postblit. And only at the beginning. I would like to see more
flexibility for copying.
[...]
For structs, using .init is a valid initialization, so it's completely
different from classes, where a constructor MUST be invoked. Indeed,
there is no mechanism to require calling struct member constructors in
the owner's ctor.
[...]
To paraphrase your point: We should be able to initialize a member with
.init or by calling its constructor (over .init). We should not be
forced to initialize it with its postblit.
Makes sense. But I see one tricky case: Can I also choose to use the
blitted value of the member without calling its postblit? I'd say that
can't be allowed.
So the compiler would have to identify that case and reject it. And now
we're in uncharted territory. As far as I see, this is not something
that constructors do (struct or class). They happily accept any .init
value. But postblits can't accept any blitted value. So we can't say:
"Just do what constructors do." We have to come up with new rules.
But we don't want to come up with new rules, we want to say: "Just do
what constructors do." So we require the member postblit, and we say
that the outer postblit operates on the resulting value like a
constructor operates on .init.
And then we see that it breaks the type system, and that any analog
behavior of constructors just means that constructors/.init are broken, too.
[...]
struct S
{
int x;
this(this) immutable
{
x = 42; /* First write. */
[...]
}
}
struct T
{
S s;
this(this) immutable
{
s = S(13); /* Second write. Breaking immutable? */
Of course this doesn't compile, because s is considered immutable by
now.
It doesn't compile, because this(this) is a broken mess currently.
The first write doesn't compile either. Obviously, with a properly
implemented immutable this(this), you would at least be allowed to write
once.
With mutable(!) `this(this)`s, both writes are accepted.
What I was saying is that we can't allow postblit to modify data
that has already been postblitted, because of the reason this example is
showing.
Still, I'd like to have a more explosive example. The breakage here is
very localized, and could possibly be defined away somehow. Like:
"Unique immutable data is considered 'raw' in constructors and postblit
functions. That means, the data can be mutated. Only when the
constructor/postblit returns does it become 'cooked' and truly immutable."
[...]
[...]
I don't think you should be able to write to the same field multiple
times in an immutable/const constructor. If so, that's a bug.
Are you counting .init as a write, or is a constructor allowed to build
on that?
import std.stdio;
struct S
{
int x = 1;
int y = 3;
this(int dummy) immutable
{
writeln(x); /* 1 */
++x; /* Breaking immutable? */
writeln(x); /* 2 */
++y; /* Breaking immutable even though y hasn't been printed
yet? */
writeln(y); /* 4 */
}
}
void main()
{
auto s = immutable S(0);
}
I think those increments could be considered breaking immutable. Maybe
they must be.
If this were to be outlawed, I'd agree a constructors and postblits are
fundamentally different.
Re: Deprecating this(this)
On 4/3/18 5:44 PM, ag0aep6g wrote:
On 04/03/2018 10:51 PM, Steven Schveighoffer wrote:
On 4/3/18 4:26 PM, ag0aep6g wrote:
[...]
If there's a problem with running two postblits on the same field,
then I think constructors probably have similar issue. I'm having a
hard time finding a good example, though. One where we could break
immutable in an obvious way or some such.
You may NOT want to run a postblit on the member. If all you are going
to do, for example, is reassign a variable, then running the postblit,
and then the destructor, just so you can overwrite it is pointless.
Same with class constructors: You may not want to run `super` when
you're just going to overwrite what it did. But the language doesn't
give you a choice. It'll be called one way or another.
At least you can invoke the one you want, with postblit there is only
one "choice".
But this is a red herring -- we already have struct constructors, and
that requirement of invoking constructors on members is not present.
With structs, we have the possibility of initialization via different
mechanisms: constructor, postblit, .init. All of these are supported by
the struct member, but currently you can only invoke postblit if you are
in a postblit. And only at the beginning. I would like to see more
flexibility for copying.
I'm not saying that imitating how constructors work will make the best
possible copying mechanism. Something else might be superior in every
way. It's just that so far the arguments against a constructor-like
postblit also seem to apply to constructors as they are implemented.
For structs, using .init is a valid initialization, so it's completely
different from classes, where a constructor MUST be invoked. Indeed,
there is no mechanism to require calling struct member constructors in
the owner's ctor.
Stop thinking class constructors, and think struct constructors instead.
But more importantly, if the postblit of the member does something
crazy like stores a reference to itself as an immutable elsewhere, and
then the compiler allows overwriting, we now have problems.
I'd love to see an example of this in code. The best I can come up with
would be something like this (doesn't compile):
import std.stdio;
immutable(int)* p;
struct S
{
int x;
this(this) immutable
{
x = 42; /* First write. */
.p = &this.x;
writeln(p, " ", *p); /* Prints some address and 42. */
}
}
struct T
{
S s;
this(this) immutable
{
s = S(13); /* Second write. Breaking immutable? */
Of course this doesn't compile, because s is considered immutable by
now. What I was saying is that we can't allow postblit to modify data
that has already been postblitted, because of the reason this example is
showing.
writeln(p, " ", *p); /* Same address, but 13. */
}
}
void main()
{
immutable T foo;
immutable bar = foo;
}
But that's essentially the same as the class example I posted. `*p`
would only change values during the postblit run. Just like a
constructor chain can write to the same field multiple times.
I don't think you should be able to write to the same field multiple
times in an immutable/const constructor. If so, that's a bug.
That's kinda iffy, but I can't find a way to demonstrate some real,
obvious damage.
Any place where an immutable can be observed to change between two reads
is breaking immutable.
-Steve
Re: Deprecating this(this)
On 01/04/18 04:56, Jonathan M Davis wrote: Another potential issue is whether any of this does or should relate to https://github.com/dlang/DIPs/pull/109 and it's solution for hooking into to moves. I'm not at all sure that what happens with that needs to be related to this at all, but it might. - Jonathan M Davis I was actually going to start a new thread about it. On the one hand, nothing in the opMove DIP is directly affected by this. On the other, if we're moving away from "copy and then fix" mentality, then opMove should also reflect this. The problem is that the alternative solution has a much bigger impact on backward compatibility. I'm really tempted to try and push this DIP as is, only renaming "opMove" to "opPostMove". This way, we can push the simpler version as is, and when (and it will take a while) "this(this)" deprecation finally comes, we can implement "opMove" as an in-process user hook. Shachar
Re: Deprecating this(this)
On 04/03/2018 10:51 PM, Steven Schveighoffer wrote:
On 4/3/18 4:26 PM, ag0aep6g wrote:
[...]
If there's a problem with running two postblits on the same field,
then I think constructors probably have similar issue. I'm having a
hard time finding a good example, though. One where we could break
immutable in an obvious way or some such.
You may NOT want to run a postblit on the member. If all you are going
to do, for example, is reassign a variable, then running the postblit,
and then the destructor, just so you can overwrite it is pointless.
Same with class constructors: You may not want to run `super` when
you're just going to overwrite what it did. But the language doesn't
give you a choice. It'll be called one way or another.
I'm not saying that imitating how constructors work will make the best
possible copying mechanism. Something else might be superior in every
way. It's just that so far the arguments against a constructor-like
postblit also seem to apply to constructors as they are implemented.
So I'm thinking that a postblit modeled after constructors could work as
well as they do. But maybe the real takeaway is that constructors don't
work very well, and shouldn't be imitated.
But more importantly, if the postblit of the member does something crazy
like stores a reference to itself as an immutable elsewhere, and then
the compiler allows overwriting, we now have problems.
I'd love to see an example of this in code. The best I can come up with
would be something like this (doesn't compile):
import std.stdio;
immutable(int)* p;
struct S
{
int x;
this(this) immutable
{
x = 42; /* First write. */
.p = &this.x;
writeln(p, " ", *p); /* Prints some address and 42. */
}
}
struct T
{
S s;
this(this) immutable
{
s = S(13); /* Second write. Breaking immutable? */
writeln(p, " ", *p); /* Same address, but 13. */
}
}
void main()
{
immutable T foo;
immutable bar = foo;
}
But that's essentially the same as the class example I posted. `*p`
would only change values during the postblit run. Just like a
constructor chain can write to the same field multiple times.
That's kinda iffy, but I can't find a way to demonstrate some real,
obvious damage.
I think the better mechanism for immutable copying would be to have a
copy constructor that starts off with T.init, and is passed the object
to copy from. That seems to be a direction Andrei is considering.
No objection from me. If Andrei et al. can find a better solution, great.
Re: Deprecating this(this)
On 04/03/2018 04:29 PM, ag0aep6g wrote: But if postblit goes away for other reasons anyway (like the atomic copy thing, or another mechanism being simply superior), then there's no point in pursuing this, of course. The DIP will definitely need to make a solid case supporting whatever it proposes.
Re: Deprecating this(this)
On 04/03/2018 04:26 PM, ag0aep6g wrote:
On 04/03/2018 05:13 PM, Steven Schveighoffer wrote:
Unfortunately, I found out that it's not just "pre-filled with some
values". Member postblits are run before the containing postblit.
https://run.dlang.io/is/mt6eGa
So this means, the data that is available to the postblit has already
been processed.
There's a similar situation with constructors: A constructor can call
another constructor, which can lead to double initialization of fields.
Example:
class C
{
int x;
this() immutable
{
this(42); /* Initializes x. */
x = 13; /* Breaking immutable, or ok? */
}
this(int x) immutable
{
this.x = x;
}
}
Let's replace "int" with an UDT:
struct S
{
int x = -1;
this(int y) immutable { x = y; }
void opAssign(int) immutable;
}
class C
{
S x;
this() immutable
{
this(42); /* Initializes x. */
x = 13; /* Breaking immutable, or ok? */
}
this(int x) immutable
{
this.x = x;
}
}
This code compiles, and calls the constructor twice for the same object.
Clearly that shouldn't be allowed to pass. I've submitted
https://issues.dlang.org/show_bug.cgi?id=18719 - thanks! (The problem
seems to occur even without immutable, it's endemic to forwarding
constructors.)
Andre
Re: Deprecating this(this)
On 4/3/18 4:26 PM, ag0aep6g wrote:
On 04/03/2018 05:13 PM, Steven Schveighoffer wrote:
Unfortunately, I found out that it's not just "pre-filled with some
values". Member postblits are run before the containing postblit.
https://run.dlang.io/is/mt6eGa
So this means, the data that is available to the postblit has already
been processed.
There's a similar situation with constructors: A constructor can call
another constructor, which can lead to double initialization of fields.
Example:
class C
{
int x;
this() immutable
{
this(42); /* Initializes x. */
x = 13; /* Breaking immutable, or ok? */
IMO, breaking immutable.
}
this(int x) immutable
{
this.x = x;
}
}
If there's a problem with running two postblits on the same field, then
I think constructors probably have similar issue. I'm having a hard time
finding a good example, though. One where we could break immutable in an
obvious way or some such.
You may NOT want to run a postblit on the member. If all you are going
to do, for example, is reassign a variable, then running the postblit,
and then the destructor, just so you can overwrite it is pointless.
But more importantly, if the postblit of the member does something crazy
like stores a reference to itself as an immutable elsewhere, and then
the compiler allows overwriting, we now have problems.
I think the better mechanism for immutable copying would be to have a
copy constructor that starts off with T.init, and is passed the object
to copy from. That seems to be a direction Andrei is considering.
-Steve
Re: Deprecating this(this)
On 04/03/2018 08:57 PM, Andrei Alexandrescu wrote:
Well... not really. This is because .init is really an inert state -
null indirections, no state allocated etc.
.init can have non-null indirections:
struct S { int[] a = [1, 2, 3]; }
static assert(S.init.a.ptr !is null); /* passes */
But maybe it shouldn't. It leads to problems with immutable, because the
same .init with the same `a` is used for both mutable and immutable
objects. https://issues.dlang.org/show_bug.cgi?id=10376
As far as I see, a const/immutable postblit working like a constructor
wouldn't have that particular problem, because the copy is always made
from another const/immutable object, so there wouldn't be aliasing with
mutable data.
But there might of course be other problems with indirections, which I
don't see right now. I'd love to see any examples, though. They probably
reveal weaknesses in .init/constructors, too.
Makes typechecking easy, and
calling constructor on top of .init is what happens already. In
contrast, the postblit situash is very different - the fields already
contain "interesting" data, allocated resources etc. Calling a
constructor on top of that is not defined.
Well, the idea would be to define it. But if postblit goes away for
other reasons anyway (like the atomic copy thing, or another mechanism
being simply superior), then there's no point in pursuing this, of course.
Re: Deprecating this(this)
On 04/03/2018 05:13 PM, Steven Schveighoffer wrote:
Unfortunately, I found out that it's not just "pre-filled with some
values". Member postblits are run before the containing postblit.
https://run.dlang.io/is/mt6eGa
So this means, the data that is available to the postblit has already
been processed.
There's a similar situation with constructors: A constructor can call
another constructor, which can lead to double initialization of fields.
Example:
class C
{
int x;
this() immutable
{
this(42); /* Initializes x. */
x = 13; /* Breaking immutable, or ok? */
}
this(int x) immutable
{
this.x = x;
}
}
If there's a problem with running two postblits on the same field, then
I think constructors probably have similar issue. I'm having a hard time
finding a good example, though. One where we could break immutable in an
obvious way or some such.
It would only make sense to allow const postblits to have the same
constructor mechanism if the members all had no postblits.
Less drastically, we could also disallow writing to those fields that
have their own postblit.
The analog with constructors would be disallowing writing to fields that
have already been initialized by an implicit `super` call. Actually,
that seems to be how it works, kinda-sorta:
class C
{
int x;
}
class D : C
{
int y;
this() immutable
{
y = 1; /* accepted */
x = 2; /* Error: cannot modify this.x in immutable function */
}
}
Obviously, DMD doesn't check what C's constructor actually does (because
it generally can't check that). It just considers initializing x to be
C's responsibility.
Re: Deprecating this(this)
On Saturday, 31 March 2018 at 23:38:06 UTC, Andrei Alexandrescu
wrote:
[snip]
* immutable and const are very difficult, but we have an attack
(assuming copy construction gets taken care of)
Would it be easier if the const/immutable containers were
considered separate types? For instance, in the code below, there
is InoutFoo and then Foo takes InoutFoo as an alias this (you
could do the same thing with immutable, but then you’d have to
include two get functions). This would be like inheriting the
InoutFoo.
With better syntax, InoutFoo would be something like inout(Foo)
and the compiler could recognize that the mutable constructor is
also defined and to call that when appropriate.
struct InoutFoo
{
int a;
this(int b) inout
{
this.a = b;
}
int get() inout
{
return a;
}
}
struct Foo
{
InoutFoo inoutfoo;
alias inoutfoo this;
this(int b)
{
a = b;
}
void set(int b)
{
a = b;
}
}
void main()
{
auto x = immutable InoutFoo(1);
auto y = Foo(1);
assert(is(typeof(y) : typeof(x)));
//x.a++; //not allowed
y.a++;
assert(x.a == 1);
assert(y.a == 2);
assert(x.get == 1);
y.set(3);
assert(y.get == 3);
}
Re: Deprecating this(this)
On 04/03/2018 10:21 AM, ag0aep6g wrote: On Tuesday, 3 April 2018 at 12:52:00 UTC, Andrei Alexandrescu wrote: On 04/03/2018 07:36 AM, ag0aep6g wrote: For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost. What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data. So constructors, including const/immutable ones, basically work the same as postblit already? You get an object pre-filled with some values, and then you can "initialize" the fields some more if you want. Well... not really. This is because .init is really an inert state - null indirections, no state allocated etc. Makes typechecking easy, and calling constructor on top of .init is what happens already. In contrast, the postblit situash is very different - the fields already contain "interesting" data, allocated resources etc. Calling a constructor on top of that is not defined.
Re: Deprecating this(this)
On 4/3/18 10:21 AM, ag0aep6g wrote: On Tuesday, 3 April 2018 at 12:52:00 UTC, Andrei Alexandrescu wrote: On 04/03/2018 07:36 AM, ag0aep6g wrote: For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost. What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data. So constructors, including const/immutable ones, basically work the same as postblit already? You get an object pre-filled with some values, and then you can "initialize" the fields some more if you want. Unfortunately, I found out that it's not just "pre-filled with some values". Member postblits are run before the containing postblit. https://run.dlang.io/is/mt6eGa So this means, the data that is available to the postblit has already been processed. It would only make sense to allow const postblits to have the same constructor mechanism if the members all had no postblits. -Steve
Re: Deprecating this(this)
On Tuesday, 3 April 2018 at 12:52:00 UTC, Andrei Alexandrescu wrote: On 04/03/2018 07:36 AM, ag0aep6g wrote: For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost. What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data. So constructors, including const/immutable ones, basically work the same as postblit already? You get an object pre-filled with some values, and then you can "initialize" the fields some more if you want. [...] What if the user code reads the value? * Often people use this(this) to bump a reference count a la "if (pcnt) ++*pcnt;" Because of the indirection, that can only be done in a mutable `this(this)`. Otherwise you violate the const/immutable guarantee of the original object. * People may pass the field by reference to an opaque function. What type does the field have? Fully const. Same as in a constructor. [...] In case (1) things can get quite confusing. Inside a postblit, field = TypeOfField(100); is a call to the constructor, whereas field = TypeOfField(field.x + 100); is a call to the assignment operator. A const constructor currently accepts both of those. So the second one can apparently be considered "initialization" as well? Or should a const constructor not be allowed to do that?
Re: Deprecating this(this)
On 04/03/2018 07:36 AM, ag0aep6g wrote: For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost. What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data. We can do the same with the postblit function: First assignment is actually initialization. When the compiler sees that the postblit function initializes a field, it can skip that field when blitting. What if the user code reads the value? * Often people use this(this) to bump a reference count a la "if (pcnt) ++*pcnt;" * People may pass the field by reference to an opaque function. What type does the field have? But it can also just blit the whole struct, because it doesn't matter if the value just gets overwritten. In other words, a postblit function can either: 1) use the blitted value as a starting point, like a constructor can use the .init value, or it can 2) initialize the field itself. Would make perfect sense to me. In case (1) things can get quite confusing. Inside a postblit, field = TypeOfField(100); is a call to the constructor, whereas field = TypeOfField(field.x + 100); is a call to the assignment operator. Andrei
Re: Deprecating this(this)
On 04/02/2018 02:47 PM, Marco Leise wrote: Am Mon, 2 Apr 2018 11:57:55 -0400 schrieb Andrei Alexandrescu : Problem is we don't have head-mutable in the language. Yes, for built-in slices the mechanism is simple - just change qualifier(T[]) to qualifier(T)[]. For a struct S, there is no way to convert from qualifier(S) to tailqualifier(S). I plan to attack this directly in the DIP - provide a way for structs to express "here's what implicit conversion should be applied when doing template matching". Andrei You are hitting a prominent type system flaw here. What may look like a hurdle on the path to fix this(this) is also at the core of getting "shared" into a good shape and probably affects how we will discuss "immutable destructors" and their kin in the future. The question is "How transitive is a qualifier when we strip it top-level on an aggregate?" Roger. My hope is to solve that for primitive types, then use that to typecheck constructors and destructors, then use the signatures of (typechecked) constructors and destructors to address composition. Ideally we'd get away without defining another kind of qualifier - @tail(const) or whatever. That would complicate the language a great deal. Andrei
Re: Deprecating this(this)
On 04/03/2018 09:39 AM, Kagamin wrote: On Monday, 2 April 2018 at 14:42:17 UTC, ag0aep6g wrote: The way it works in a const constructor is that `this.foo = bar;` is considered initialization, not assignment. Do you mean the spec? Andrei complained about implementation. Andrei complained about both, no? His words: "Too many bugs in design and implementation." Design = spec. I'm also talking about the implementation. `this.foo = bar;` in a constructor is at least not normal assignment. If foo has opAssign, that won't be called. Const constructors are already implemented as needed for postblit. Maybe. But we can't explain the special assignment semantics with it being initialization. Or can we? For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost. We can do the same with the postblit function: First assignment is actually initialization. When the compiler sees that the postblit function initializes a field, it can skip that field when blitting. But it can also just blit the whole struct, because it doesn't matter if the value just gets overwritten. In other words, a postblit function can either: 1) use the blitted value as a starting point, like a constructor can use the .init value, or it can 2) initialize the field itself. Would make perfect sense to me.
Re: Deprecating this(this)
On Monday, 2 April 2018 at 14:42:17 UTC, ag0aep6g wrote: The way it works in a const constructor is that `this.foo = bar;` is considered initialization, not assignment. Do you mean the spec? Andrei complained about implementation. Const constructors are already implemented as needed for postblit. In a postblit function, we can't say it's initialization, because the field already has a value that can't be ignored. Fields are initialized before const constructor too.
Re: Deprecating this(this)
On Monday, April 02, 2018 20:47:31 Marco Leise via Digitalmars-d wrote: > Am Mon, 2 Apr 2018 11:57:55 -0400 > > schrieb Andrei Alexandrescu : > > Problem is we don't have head-mutable in the language. Yes, for built-in > > slices the mechanism is simple - just change qualifier(T[]) to > > qualifier(T)[]. For a struct S, there is no way to convert from > > qualifier(S) to tailqualifier(S). > > > > I plan to attack this directly in the DIP - provide a way for structs to > > express "here's what implicit conversion should be applied when doing > > template matching". > > > > Andrei > > You are hitting a prominent type system flaw here. What may > look like a hurdle on the path to fix this(this) is also at > the core of getting "shared" into a good shape and probably > affects how we will discuss "immutable destructors" and their > kin in the future. The question is "How transitive is a > qualifier when we strip it top-level on an aggregate?" > > In https://issues.dlang.org/show_bug.cgi?id=8295 I've been > arguing for removing all qualifiers on shallow copies and the > case you mentioned where top level qualifiers are stripped for > template matching reconfirms me that there is generally some > merit to that semantic, that should be explored. > > Shared structs need elaborate code to be copied, that's for > sure. There may be a mutex to be used or values may be copied > using atomic loads. The result would be what you dubbed > "tailqualifier(S)". I.e. in case of shared, a thread-local > copy of the fields that make up the struct. > > But then it starts to become messy: > * Are there cases where we want references contained in the > struct to become unshared, too? > * If yes, what if these references were marked shared > themselves in the struct's definition? > * If all fields become unshared, shouldn't the now superfluous > mutex be removed from the struct? If so, what started out as > a bit blit, now produces a different type entirely. > > I'm interested to hear more on your thoughs on > "tailqualifier(S)". Copying shared structs is a bit sketchy in general. In theory, we've been trying to make it so non-atomic operations on shared objects aren't legal, because they're not safe unless the object is protected by a mutex. So, the normal way to deal with shared if you want to do anything with it is to protect a section of code with a mutex and temporarily cast away shared within that section of code (if we had synchronized classes, then in some cases, that cast would be automatic, but as it stands, it's always manual). And if doing a non-atomic operation on a shared object isn't legal, then copying a shared object doesn't really make sense. In that case, I would expect that copying a shared object wouldn't be legal (and as such the way to copy a shared object would be to protect that with a mutex and cast away shared when it's copied, and then cast the new object to shared if appropriate). On the other hand, that could get _really_ annoying, and if we were dealing with a copy constructor rather than a postblit constructor, then it would probably be possible to lock a mutex and inside the copy constructor so that the copy and whatever casting was necessary could be done safely inside the struct rather than having to externally protect it with a mutex and cast away shared there. So, I don't know what the answer is, but if we're trying to make operations which aren't guaranteed to be thread-safe illegal on shared objects, then we have a bit of a problem with what to do with copying shared objects (or assigning shared objects). But I suppose that part of the problem is that while the _idea_ of shared is very well defined, the actual details aren't (e.g. a number of non-atomic operations aren't legal on shared objects, but some are). And copying is definitely one area where that has not been properly sorted out yet. - Jonathan M Davis
Re: Deprecating this(this)
Am Mon, 2 Apr 2018 11:57:55 -0400 schrieb Andrei Alexandrescu : > Problem is we don't have head-mutable in the language. Yes, for built-in > slices the mechanism is simple - just change qualifier(T[]) to > qualifier(T)[]. For a struct S, there is no way to convert from > qualifier(S) to tailqualifier(S). > > I plan to attack this directly in the DIP - provide a way for structs to > express "here's what implicit conversion should be applied when doing > template matching". > > Andrei You are hitting a prominent type system flaw here. What may look like a hurdle on the path to fix this(this) is also at the core of getting "shared" into a good shape and probably affects how we will discuss "immutable destructors" and their kin in the future. The question is "How transitive is a qualifier when we strip it top-level on an aggregate?" In https://issues.dlang.org/show_bug.cgi?id=8295 I've been arguing for removing all qualifiers on shallow copies and the case you mentioned where top level qualifiers are stripped for template matching reconfirms me that there is generally some merit to that semantic, that should be explored. Shared structs need elaborate code to be copied, that's for sure. There may be a mutex to be used or values may be copied using atomic loads. The result would be what you dubbed "tailqualifier(S)". I.e. in case of shared, a thread-local copy of the fields that make up the struct. But then it starts to become messy: * Are there cases where we want references contained in the struct to become unshared, too? * If yes, what if these references were marked shared themselves in the struct's definition? * If all fields become unshared, shouldn't the now superfluous mutex be removed from the struct? If so, what started out as a bit blit, now produces a different type entirely. I'm interested to hear more on your thoughs on "tailqualifier(S)". -- Marco
Re: Deprecating this(this)
On 4/2/18 1:08 PM, Andrei Alexandrescu wrote: On 04/02/2018 12:53 PM, Steven Schveighoffer wrote: As was mentioned, because postblit on an immutable (or const) is ONLY allowed for new data, there shouldn't be an issue. The problem with postblit is there's "double construction", one done by the compiler, after which the user may want to assign something else. That's more difficult to typecheck than direct initialization. I was going to argue that it's not full construction -- it's just a copy of bits. But I am incorrect. My understanding was that postblit is called after the bits are copied, but that isn't the case. Currently, the postblit of members is called BEFORE the postblit of the container (or maybe as part of the postblit of the container, but always at the start). This means that a member is not a moved copy at that point, but a fully postblitted item. This also means that it's not safe to assume head-const-ness. This makes some sense -- you don't want to have to deal with member postblits if you don't have to. But it also makes it impossible to intercept copying to do something different (as you would need to do for this case). Indeed, something other than the current postblit mechanism looks more attractive and powerful, even if it isn't as straightforward. -Steve
Re: Deprecating this(this)
On Monday, April 02, 2018 08:56:41 H. S. Teoh via Digitalmars-d wrote: > On Sun, Apr 01, 2018 at 02:31:06PM +, Nicholas Wilson via Digitalmars- d wrote: > > On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote: > > > One issue is that postblit constructors fundamentally don't work > > > with const. The problem is that a postblit constructor works by > > > copying the object and _then_ mutating it, and you can't mutate a > > > const object. To cleanly deal with const, you need something more > > > like a copy constructor where you initialize it with the adjusted > > > values directly rather than mutating the copy. > > > > I've always wondered about that, is the difference between that > > anything more than philosophical? Put another way if a this(this) is > > weakly pure, is there any safety issues with the compiler permitting > > the mutation on a (non-shared? not sure if this would be a > > requirement) const object? I'm not sure what the spec says, but if you > > take the view that the const object is no fully initialised until the > > postblit is done, then I don't see the problem. > > Yeah I've been wondering about this too. I mean, currently, ctors are > allowed to assign to immutable fields, because, well, it's > initialization, not after-the-fact mutation. Why can't we extend this > to postblits? Now, there's certainly the issue of ctors "leaking" > mutable references to immutable fields if not implemented properly, but > since postblits are run after a built-into-the-language copying of the > original, supposedly an opaque process, it seems reasonable enough to > allow the postblit to be regarded as initialization and able to assign > to const/immutable fields once. > > At the very least, allow rebinding of const/immutable references in the > postblit. (Allowing straight-out reassignment may have adverse effects > if the copy shares a reference to an immutable object.) The core problem is that in a postblit, you're reading a member variable and then assigning to it, which violates const. For some simple cases, we could essentially make the member variables tail-const within the postblit, because we could safely say that the member variable was a distinct copy and that overwriting it wouldn't really cause problems - e.g. assigning to an int wouldn't really be a problem, and assigning to const(T)* wouldn't really be a problem. For classes, it's a bit hinky, because there's no such thing as tail-const for classes, but they're conceptually the same as const(T)*, so we could cheat in a postblit and still let the reference be assigned once. However, the real problem is once you start dealing with structs. Not only is there no real concept of tail-const with structs, and they can contain any combination of value types, reference types, pseudo-reference types, etc. but they can overload opAssign and postblit. And of course, the postblit constructor for member variables is run before the postblit constructor for the object containing them, and there's no guarantee that the postblit actually did _anything_ involving making a deep copy (e.g. it could simply have been printing out that the object was copied). So, figuring out how independent a copy the struct is from the original isn't necessarily very straightforward, and even if it were, how would we do the equivalent of const(T)* to allow the struct to be reassigned without mucking with any data that's not independent? It might contain members that have their data directly embedded in the struct and members which are reference types, but it's still one unit, and stuff like opAssign is not designed with the idea that you just overwrite some of the struct. And of course, once opAssign is overloaded, who knows what the semantics of assigning to the struct are. As such, how do you reason about the safety of relaxing the type system to allow assignment to a struct that has overloaded opAssign? At one point, Kenji was working on a solution to the problem (I _think_ that he had a DIP on it, but it's been a while, so I don't remember), and as I recall, Walter and Andrei vetoed it, because it was too complicated. And I don't even know if he actually, fully solved the problem. Ultimately, this is all much, much cleaner if copying an object involves initializing it directly rather than doing a shallow copy and then doing a deep copy and reassigning pieces of the object. That sucks for the simple cases, because if you don't care about stuff like const, and your object has a bunch of members in it, with a postblit constructor, you might only have to manually do something with a few of them (whereas in C++, you'd have to list them all individually), but as far as I can tell, it's pretty much required for the complex cases. Off the top of my head, what I would probably do if I were redesigning this would be to go with copy constructors but improve the syntax so that you don't have to list any of the member variables unless you're actually going to
Re: Deprecating this(this)
On 04/02/2018 12:53 PM, Steven Schveighoffer wrote:
On 4/2/18 11:57 AM, Andrei Alexandrescu wrote:
On 04/02/2018 10:59 AM, ag0aep6g wrote:
That wouldn't be possible if `innocent` were only head-mutable in the
postblit function, instead of fully mutable as it is currently (bug).
`innocent` would be typed as `immutable(int)[]`, and you could not
assign it to the fully mutable `sneaky`.
Problem is we don't have head-mutable in the language.
This is the wrong way to look at it. Head mutable is a specialized
concept already implemented in constructors:
struct S
{
int x;
int *ptr;
int *ptr2;
this(int xval, immutable int *ptrval, int *cantuse) immutable
{
x = xval; // ok, head mutable
//x = x + 1; // error, immutable field initialized multiple times
ptr = ptrval; // ok
// ptr2 = cantuse; // error, can't assign mutable to immutable
}
}
I've asked Razvan to document how immutable constructors are exactly
typechecked. My understanding is that they don't rely on some form of
internal "head const" but instead on simple data flow - the first
assignment counts as a constructor call. Consider the following example,
which contains opaque methods instead of built-ins:
struct S
{
int[] payload;
S* another;
this(int) immutable;
}
struct HasS
{
S member;
this(int x) immutable
{
member = immutable S(1);
}
}
This fails to link; the assignment is just a call to the (declared but
undefined) constructor.
Such behavior is nice and easy to generalize to copy construction.
We could have the same mechanism for postblit. Essentially, you should
be able to assign immutable fields once, but they shouldn't lose their
const protections (note the cantuse example).
Yes, with the distinction that is not "assign" but really "construction
with assignment syntax".
As was mentioned, because postblit on an immutable (or const) is ONLY
allowed for new data, there shouldn't be an issue.
The problem with postblit is there's "double construction", one done by
the compiler, after which the user may want to assign something else.
That's more difficult to typecheck than direct initialization.
Andrei
Re: Deprecating this(this)
On 4/2/18 11:57 AM, Andrei Alexandrescu wrote:
On 04/02/2018 10:59 AM, ag0aep6g wrote:
That wouldn't be possible if `innocent` were only head-mutable in the
postblit function, instead of fully mutable as it is currently (bug).
`innocent` would be typed as `immutable(int)[]`, and you could not
assign it to the fully mutable `sneaky`.
Problem is we don't have head-mutable in the language.
This is the wrong way to look at it. Head mutable is a specialized
concept already implemented in constructors:
struct S
{
int x;
int *ptr;
int *ptr2;
this(int xval, immutable int *ptrval, int *cantuse) immutable
{
x = xval; // ok, head mutable
//x = x + 1; // error, immutable field initialized multiple times
ptr = ptrval; // ok
// ptr2 = cantuse; // error, can't assign mutable to immutable
}
}
We could have the same mechanism for postblit. Essentially, you should
be able to assign immutable fields once, but they shouldn't lose their
const protections (note the cantuse example).
As was mentioned, because postblit on an immutable (or const) is ONLY
allowed for new data, there shouldn't be an issue.
-Steve
Re: Deprecating this(this)
On 04/02/2018 12:14 PM, Paolo Invernizzi wrote: On Monday, 2 April 2018 at 16:00:11 UTC, bachmeier wrote: On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote: Andrei wrote in the message I am looking for folks to assist me in creating a DIP for that. There will be a _lot_ of work involved, so don't take it lightly. Andrei is asking others to write a DIP to formalize a decision There's not even an attempt made to pretend there's symmetry. The only way for Manu (and basically anyone else) to propose a change is to write a DIP. Andrei won't even participate in discussions without a DIP. That's probably a good idea. What's not a good idea is to make unilateral decisions about major breaking changes, posting in the forum, and then asking others to write the DIP. That's corporate software development, and it's very discouraging to potential contributors. I think you are plain wrong on this: the 'P' in a DIP stands for Proposal, so any decision is not taken yet. And I'll bet: - Andrei will be the main author or he will partecipate in the writing. - the DIP will follow the usual proceeding, exactly like the others. Affirmative. No need to discuss this further, it's a simple misunderstanding. I'd agree with bachmeier if his perception was correct.
Re: Deprecating this(this)
On Monday, 2 April 2018 at 16:00:11 UTC, bachmeier wrote: On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote: Andrei wrote in the message I am looking for folks to assist me in creating a DIP for that. There will be a _lot_ of work involved, so don't take it lightly. Andrei is asking others to write a DIP to formalize a decision There's not even an attempt made to pretend there's symmetry. The only way for Manu (and basically anyone else) to propose a change is to write a DIP. Andrei won't even participate in discussions without a DIP. That's probably a good idea. What's not a good idea is to make unilateral decisions about major breaking changes, posting in the forum, and then asking others to write the DIP. That's corporate software development, and it's very discouraging to potential contributors. I think you are plain wrong on this: the 'P' in a DIP stands for Proposal, so any decision is not taken yet. And I'll bet: - Andrei will be the main author or he will partecipate in the writing. - the DIP will follow the usual proceeding, exactly like the others. /Paolo
Re: Deprecating this(this)
On 04/02/2018 12:00 PM, bachmeier wrote: On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote: Andrei wrote in the message I am looking for folks to assist me in creating a DIP for that. There will be a _lot_ of work involved, so don't take it lightly. So, let's keep the discussion factual. I'm pretty sure that every aspect will be taken in account and pondered prior to a decision. I'm +1 on major breaking changes if they drive D towards a better shape. Andrei is asking others to write a DIP to formalize a decision he has already made. Yet when Manu posts here, he responds: Apologies for the misunderstanding. I'll be the first author of the DIP and plan to dedicate a lot of time to it. I was just asking for others to join me in this important and urgent effort. Andrei
Re: Deprecating this(this)
On 04/02/2018 10:42 AM, ag0aep6g wrote: On Monday, 2 April 2018 at 14:01:22 UTC, Kagamin wrote: On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote: 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust. Do the same as in const constructor. The way it works in a const constructor is that `this.foo = bar;` is considered initialization, not assignment. Affirmative. First assignment is a call to the member's constructor. We typecheck that reasonably well already in qualified constructors, and it's the most promising approach for the DIP. In a postblit function, we can't say it's initialization, because the field already has a value that can't be ignored. Affirmative. That's what makes it so darn difficult to typecheck. If we don't let the compiler do the initial blitting (and instead start with T.init), the copy ctor is typechecked exactly like the regular ctor. Andrei
Re: Deprecating this(this)
On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote: Andrei wrote in the message I am looking for folks to assist me in creating a DIP for that. There will be a _lot_ of work involved, so don't take it lightly. So, let's keep the discussion factual. I'm pretty sure that every aspect will be taken in account and pondered prior to a decision. I'm +1 on major breaking changes if they drive D towards a better shape. Andrei is asking others to write a DIP to formalize a decision he has already made. Yet when Manu posts here, he responds: The good news is there is a way to ensure your proposal gets a fair shake of the stick: write a DIP. Filing a DIP is like filing a police report: once it's in the system, we're obligated to work on it. There's a guarantee of a response. In the case of acceptance, we commit to implementing the proposal. In the case of rejection, we give a clear motivation of the reasons we had. In the case we ask for further review, we provide clear feedback of what would take the DIP through another iteration. Forum discussions are the equivalent of complaining loudly in a bar to people you know and also to strangers within earshot that your house was broken into. Until you file a report, the police will not look into it. There's not even an attempt made to pretend there's symmetry. The only way for Manu (and basically anyone else) to propose a change is to write a DIP. Andrei won't even participate in discussions without a DIP. That's probably a good idea. What's not a good idea is to make unilateral decisions about major breaking changes, posting in the forum, and then asking others to write the DIP. That's corporate software development, and it's very discouraging to potential contributors.
Re: Deprecating this(this)
On 04/02/2018 10:59 AM, ag0aep6g wrote: That wouldn't be possible if `innocent` were only head-mutable in the postblit function, instead of fully mutable as it is currently (bug). `innocent` would be typed as `immutable(int)[]`, and you could not assign it to the fully mutable `sneaky`. Problem is we don't have head-mutable in the language. Yes, for built-in slices the mechanism is simple - just change qualifier(T[]) to qualifier(T)[]. For a struct S, there is no way to convert from qualifier(S) to tailqualifier(S). I plan to attack this directly in the DIP - provide a way for structs to express "here's what implicit conversion should be applied when doing template matching". Andrei
Re: Deprecating this(this)
On Sun, Apr 01, 2018 at 02:31:06PM +, Nicholas Wilson via Digitalmars-d wrote: > On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote: > > One issue is that postblit constructors fundamentally don't work > > with const. The problem is that a postblit constructor works by > > copying the object and _then_ mutating it, and you can't mutate a > > const object. To cleanly deal with const, you need something more > > like a copy constructor where you initialize it with the adjusted > > values directly rather than mutating the copy. > > I've always wondered about that, is the difference between that > anything more than philosophical? Put another way if a this(this) is > weakly pure, is there any safety issues with the compiler permitting > the mutation on a (non-shared? not sure if this would be a > requirement) const object? I'm not sure what the spec says, but if you > take the view that the const object is no fully initialised until the > postblit is done, then I don't see the problem. Yeah I've been wondering about this too. I mean, currently, ctors are allowed to assign to immutable fields, because, well, it's initialization, not after-the-fact mutation. Why can't we extend this to postblits? Now, there's certainly the issue of ctors "leaking" mutable references to immutable fields if not implemented properly, but since postblits are run after a built-into-the-language copying of the original, supposedly an opaque process, it seems reasonable enough to allow the postblit to be regarded as initialization and able to assign to const/immutable fields once. At the very least, allow rebinding of const/immutable references in the postblit. (Allowing straight-out reassignment may have adverse effects if the copy shares a reference to an immutable object.) T -- What is Matter, what is Mind? Never Mind, it doesn't Matter.
Re: Deprecating this(this)
On Monday, 2 April 2018 at 13:55:25 UTC, bachmeier wrote: No, the reason nothing gets done is because "that would break code" is used to kill every proposal that comes along. Someone that only responds to proposals with "write a DIP" proceeds to announce a major piece of the language will be deprecated without writing a DIP himself. Corporate leadership doesn't work with an open source project. I could have gotten more involved long ago, but I'd rather not jump on a ship that's sailing in circles. From the many comments I've seen, I'm not the only one. Andrei wrote in the message I am looking for folks to assist me in creating a DIP for that. There will be a _lot_ of work involved, so don't take it lightly. So, let's keep the discussion factual. I'm pretty sure that every aspect will be taken in account and pondered prior to a decision. I'm +1 on major breaking changes if they drive D towards a better shape. /Paolo
Re: Deprecating this(this)
On 4/2/18 10:24 AM, Nicholas Wilson wrote: On Monday, 2 April 2018 at 14:07:21 UTC, Steven Schveighoffer wrote: On 4/1/18 10:34 AM, ag0aep6g wrote: On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote: One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway. Yes, precisely what I had been arguing here: https://issues.dlang.org/show_bug.cgi?id=18417#c5, however const/immutable postblit was recently deprecated by: https://github.com/dlang/dmd/pull/8032 So I don't think D is going to allow const postblit any more. Maybe not even postblit any more. Andrei did post an example where treating the designation as mutable, stuffing references to mutable data into is and then treating it as immutable leaves you with an immutable reference to immutable data. I think you meant immutable reference to mutable data. However i think that loophole is fixed if you only allow assignment to const/immutable from a pure postblit. It's pretty straightforward I would think: If a postblit is being called on an immutable, it *must* be new data, because if it was data that just got overwritten, that data couldn't possibly be immutable. Therefore, there's leeway to mutate the head at that point, because nothing else can see it yet. Doesn't need to be pure. Same leeway as an immutable constructor I think. But all this discussion is moot if the postblit is going away. -Steve
Re: Deprecating this(this)
On Monday, 2 April 2018 at 14:24:20 UTC, Nicholas Wilson wrote:
On Monday, 2 April 2018 at 14:07:21 UTC, Steven Schveighoffer
wrote:
On 4/1/18 10:34 AM, ag0aep6g wrote:
[...]
I'm not so sure if that's fundamental. Can't we just say that
the copy is head-mutable at the time when the postblit
function is called, and it only becomes fully const after
that?
[...]
Yes, precisely what I had been arguing here:
https://issues.dlang.org/show_bug.cgi?id=18417#c5
[...]
Andrei did post an example where treating the designation as
mutable, stuffing references to mutable data into is and then
treating it as immutable leaves you with an immutable reference
to immutable data.
This one?
int[] sneaky;
struct A
{
private int[] innocent;
this(this)
{
sneaky = innocent;
}
}
void main()
{
immutable a = A([1, 2, 3]);
auto b = a;
sneaky[1] = 42; // oops
import std.stdio;
writeln(a.innocent); // ops
}
That wouldn't be possible if `innocent` were only head-mutable in
the postblit function, instead of fully mutable as it is
currently (bug).
`innocent` would be typed as `immutable(int)[]`, and you could
not assign it to the fully mutable `sneaky`.
However i think that loophole is fixed if you only allow
assignment to const/immutable from a pure postblit.
Still must be head-mutable only. With fully mutable, you can
mutate data that is seen as immutable elsewehre, even if the
postblit function is pure:
struct S
{
int* x;
this(this) pure { *x = 13; }
}
void main()
{
auto s = immutable S(new int(42));
auto s2 = s;
assert(*s.x == 42); /* fails; immutability has been broken */
}
Issue 18357 covers both Andrei's example, and this one.
https://issues.dlang.org/show_bug.cgi?id=18357
Re: Deprecating this(this)
On Monday, 2 April 2018 at 14:01:22 UTC, Kagamin wrote: On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote: 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust. Do the same as in const constructor. The way it works in a const constructor is that `this.foo = bar;` is considered initialization, not assignment. In a postblit function, we can't say it's initialization, because the field already has a value that can't be ignored.
Re: Deprecating this(this)
On Monday, 2 April 2018 at 14:07:21 UTC, Steven Schveighoffer wrote: On 4/1/18 10:34 AM, ag0aep6g wrote: On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote: One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway. Yes, precisely what I had been arguing here: https://issues.dlang.org/show_bug.cgi?id=18417#c5, however const/immutable postblit was recently deprecated by: https://github.com/dlang/dmd/pull/8032 So I don't think D is going to allow const postblit any more. Maybe not even postblit any more. -Steve Andrei did post an example where treating the designation as mutable, stuffing references to mutable data into is and then treating it as immutable leaves you with an immutable reference to immutable data. However i think that loophole is fixed if you only allow assignment to const/immutable from a pure postblit.
Re: Deprecating this(this)
On 4/1/18 10:34 AM, ag0aep6g wrote: On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote: One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway. Yes, precisely what I had been arguing here: https://issues.dlang.org/show_bug.cgi?id=18417#c5, however const/immutable postblit was recently deprecated by: https://github.com/dlang/dmd/pull/8032 So I don't think D is going to allow const postblit any more. Maybe not even postblit any more. -Steve
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote: There's a mix of fundamental flaws and bugs. I'll get to the flaws in a second. About the bugs: people have altered their code in various ways to work with the bizarre semantics of this(this). Now, if we fix various bugs in this(this) by virtually redefining it, then we'll break a lot of code in a lot of ways. To wit, we fixed a small issue and it already created problems: https://github.com/dlang/dmd/pull/8032. That didn't contribute to the decision but is quite illustrative. Since poorly thought decisions keep being made for the language it's doubtful that "no breakage" policy has real backing.
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote: 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust. Do the same as in const constructor. 2. For shared objects, the part done by the compiler and the part done by this(this) should be synchronized together. This makes it impossible for the user to e.g. define a struct that gets copied atomically. If you mean reference counting, the easy solution with copy constructor is bad performance wise. Good solution can be done architecturally now and doesn't rely on copy constructor.
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 11:17:38 UTC, Patrick Schluter wrote: What I was wondering too. I mean, breaking changes just don't happen to this language. Now there will be, without even an indication of how existing code would have to be rewritten, or how this large-scale breakage is different than the breakages that just can't happen because reasons. I guess that's why there's always the disclaimer, "We'll only break code if there's a really good reason." That reason is "in case we want to". Nothing has been lay out yet and people are already freaking out. No wonder nothing gets done anymore. No, the reason nothing gets done is because "that would break code" is used to kill every proposal that comes along. Someone that only responds to proposals with "write a DIP" proceeds to announce a major piece of the language will be deprecated without writing a DIP himself. Corporate leadership doesn't work with an open source project. I could have gotten more involved long ago, but I'd rather not jump on a ship that's sailing in circles. From the many comments I've seen, I'm not the only one.
Re: Deprecating this(this)
On 02.04.2018 10:04, Shachar Shemesh wrote:
On 02/04/18 10:45, Jonathan M Davis wrote:
Honestly, I think at this point pure is easier to understand if you
think of
it as @noglobal and don't think about functional purity at all.
That's fine. My point was that the only optimizations possible are
possible on strictly pure functions (the immutable cast one included).
Weakly pure functions add nothing.
...
That point is wrong. It would be wrong even if your next point was true.
But merely having them around means that when I annotate a function with
"pure", I do not promise any guarantees that the compiler can actually
use to perform optimizations.
Shachar
Yes, actually you do.
For example, if you have:
void foo(int[] x)pure;
void main(){
auto x = [1];
auto y = x.dup;
foo(x);
foo(y);
}
This can be transformed into:
void main(){
auto x = [1];
foo(x);
auto y = x.dup;
}
Pure functions have deterministic side-effects.
Re: Deprecating this(this)
On 02.04.2018 08:56, Shachar Shemesh wrote: On 01/04/18 03:32, H. S. Teoh wrote: The one nagging question I've been having about pure is: how much are we actually taking advantage of the guarantees provided by pure? My problem is that pure do not provide many guarantees. ... It guarantees that no global state is accessed. We have developed very clever ways of extending the traditional definition of pure and invented creative ways of making more things pure, which is all great. Can anyone explain to me what good are the guarantees provided by a function that is pure but not strictly pure? I couldn't find them. ... You can use weakly pure functions to compose strongly pure functions. But AFAIK the only place where it's actually taken advantage of is to elide some redundant function calls inside a single expression. You cannot even do that unless the function is strictly pure. For all D's extension of the pure concept, it weakened, rather than enhanced, what it means. ... There is no such weakening. And perhaps infer uniqueness in some cases for implicit casting to immutable. Can you expand on that one? Shachar
Re: Deprecating this(this)
On 4/2/18 4:04 AM, Shachar Shemesh wrote: On 02/04/18 10:45, Jonathan M Davis wrote: Honestly, I think at this point pure is easier to understand if you think of it as @noglobal and don't think about functional purity at all. That's fine. My point was that the only optimizations possible are possible on strictly pure functions (the immutable cast one included). Weakly pure functions add nothing. But merely having them around means that when I annotate a function with "pure", I do not promise any guarantees that the compiler can actually use to perform optimizations. Shachar This is a good article motivating the relaxed purity model we have: http://klickverbot.at/blog/2012/05/purity-in-d/
Re: Deprecating this(this)
On Saturday, 31 March 2018 at 23:38:06 UTC, Andrei Alexandrescu wrote: * should work with mutable, const, immutable, and shared The problem is that postblit is not overloadable, so make it overloadable, and problems with overloading will be solved. * immutable and const are very difficult, but we have an attack (assuming copy construction gets taken care of) Collections must be filled somehow, so they are inherently mutable, immutable collections need a whole different design approach, it doesn't look specific to postblit. * pure is difficult Purity depends on written code. Running impure code in copy constructor won't make it pure.
Re: Deprecating this(this)
By sheer coincidence, I've just stumbled upon another limitation of
this(this). I am not sure whether it is already documented.
Let's define struct S1 with no copying allowed, and put it as a member
of struct S2. Under C++ as well as under D, this automatically makes S2
also non-copyable.
Under C++, however, I can do this:
struct S1 {
S1() {}
// Make S1 non-copyable
S1(const S1 &that) = delete;
S1 &operator=(const S1 &that) = delete;
};
struct S2 {
int a;
S1 s;
S2(int _a) : a(_a) {
}
S2(const S2 &that) : a(that.a) {
}
};
int main() {
S2 s(17);
S2 v(s); // This compiles, invoking S2's copy ctor
}
In other words, I can tell the compiler that I know how to copy S2
without having to copy the member S1.
Under D, this simply doesn't work. If S1 has @disable this(this), any
struct that has S1 as a member will be uncopyable, and this is not
overridable.
Shachar
Re: Deprecating this(this)
On Monday, April 02, 2018 11:04:08 Shachar Shemesh via Digitalmars-d wrote: > On 02/04/18 10:45, Jonathan M Davis wrote: > > Honestly, I think at this point pure is easier to understand if you > > think of it as @noglobal and don't think about functional purity at > > all. > > That's fine. My point was that the only optimizations possible are > possible on strictly pure functions (the immutable cast one included). > Weakly pure functions add nothing. > > But merely having them around means that when I annotate a function with > "pure", I do not promise any guarantees that the compiler can actually > use to perform optimizations. It means that a strongly pure function could call the function, which is the entire reason that the definition of pure was widened to simply mean that it guarantees that the function doesn't access global, mutable state instead of also including the requirements about parameters which are placed on strongly pure functions. So, a weakly pure function _can_ help with optimizations in that it helps to implement strongly pure functions, and without weakly pure functions, what you can do with strongly pure functions can be very limited, making weakly pure functions very important even if all you care about is optimizations, but no, a call to a weakly pure function cannot be elided based on the fact that it's weakly pure. Now, pure combined with const could provide some optimizations in rare cases, since the compiler can guarantee that a pure function doesn't mutate a const argument via another reference if no such mutable reference could be accessed through one of the arguments, but I doubt that such optimizations are done at this point, and it wouldn't involve eliding function calls. But in theory, the fact that the compiler knows that a function can't access anything except through its arguments could allow the compiler to optimize some code, even if it doesn't involve eliding function calls. Ultimately, I think that it's a mistake to think about pure having much to do with optimizations. Much as such optimizations do exist, they're just too limited. The primary advantage is that when you see that a function is pure, you know that the function is just using what it's given via its arguments, just like when you see a variable is immutable, you know that it can't be mutated. Any optimizations that can be gotten via pure are therefore mostly just gravy. So, if your only motivation in dealing with pure is optimizations, then there's a good chance that it really isn't worth your time, but personally, I think that it's quite valuably simply for guaranteeing the that function doesn't access global, mutable state. - Jonathan M Davis
Re: Deprecating this(this)
On Monday, 2 April 2018 at 00:44:14 UTC, Jonathan M Davis wrote:
On Monday, April 02, 2018 00:25:52 Nicholas Wilson via
Digitalmars-d wrote:
On Sunday, 1 April 2018 at 17:08:37 UTC, Andrei Alexandrescu
wrote:
> On 4/1/18 10:59 AM, Nicholas Wilson wrote:
> [...]
> int[] sneaky;
> struct A
> {
>
> private int[] innocent;
> this(this)
> {
>
> sneaky = innocent;
>
> }
>
> }
> void main()
> {
>
> immutable a = A([1, 2, 3]);
> auto b = a;
> sneaky[1] = 42; // oops
> import std.stdio;
> writeln(a.innocent); // ops
>
> }
>
> Sadly this (and many similar ones) compiles and runs
> warning-free on today's compiler. We really need to close
> this loop, like, five years ago.
How much of this class of bug would be eliminated by requiring
that `this(this)` be pure for assignment to const and
immutable objects? Arguably this(this) should always be pure
in any sane program. The only reason I can think of is if
you're trying to perf the number of copies you're making, but
there is compiler help for that.
All kinds of things could be done with a postlbit costructor
that don't actually involve copying. These include logging or
printing something, and they could include stuff like reference
counting, which may or may not need access to something
external. debug statements would solve some uses cases but not
all. Requiring that this(this) be pure has some of the same
issues that requiring opEquals to be pure or const or whatever
has. It makes sense in _most_ cases, but occasionally, there
are good reasons for it not to be - especially in a systems
language.
I wasn't suggesting this a global requirement of all this(this)s,
only to the _postblit assignment to const and immutable objects_
( where
being pure would to disallow the above bug). Note that this should
still be able to be worked around by cast()s (which are un@safe)
and therefore require
@trusted to work in @safe code, in which case the programmer has
presumably
thought about the situation and knows what he's doing.
Re: Deprecating this(this)
On 02/04/18 10:45, Jonathan M Davis wrote: Honestly, I think at this point pure is easier to understand if you think of it as @noglobal and don't think about functional purity at all. That's fine. My point was that the only optimizations possible are possible on strictly pure functions (the immutable cast one included). Weakly pure functions add nothing. But merely having them around means that when I annotate a function with "pure", I do not promise any guarantees that the compiler can actually use to perform optimizations. Shachar
Re: Deprecating this(this)
On Monday, April 02, 2018 09:56:19 Shachar Shemesh via Digitalmars-d wrote:
> On 01/04/18 03:32, H. S. Teoh wrote:
> > The one nagging question I've been having about pure is: how much are we
> > actually taking advantage of the guarantees provided by pure?
>
> My problem is that pure do not provide many guarantees.
>
> > We have
> >
> > developed very clever ways of extending the traditional definition of
> > pure and invented creative ways of making more things pure, which is all
> > great.
>
> Can anyone explain to me what good are the guarantees provided by a
> function that is pure but not strictly pure? I couldn't find them.
Honestly, I think at this point pure is easier to understand if you think of
it as @noglobal and don't think about functional purity at all. What pure
does is make it so that the function cannot access global, mutable state
except through its arguments. So, all that it's working with (except for
constants such as enums or immutable, module-level variables) is what it's
given via its arguments. As such, the primary benefit of pure is that you
know that no global state is being mucked with unless it was passed to the
function as an argument. _That_ is the guarantee that pure provides.
Everything else about pure is just built on top of that guarantee and what
the compiler can infer from it.
If the function is "strongly" pure (i.e. its parameters are immutable or
implicitly convertible to immutable) then the compiler knows that if the
function is called multiple times with the exact same arguments, then it
knows that the result will be the same each time (though it does have to
take into account the fact that each call could return a newly allocated
object - they'd just be equivalent objects every time). So, when you're
dealing with a strongly pure function, you're then dealing with actual,
functional purity, and the compiler can choose to optimize calls such as
foo(5) * foo(5) so that foo(5) is only called once.
Weakly pure functions (i.e. pure functions that aren't strongly pure) don't
have those same optimization benefits, because calling them could result in
the arguments being mutated, but because weakly pure functions don't access
global, mutable state, the compiler can safely call them from a strongly
pure function without violating the guarantee that multiple calls with the
same arguments to a strongly pure function are supposed to give the same
result.
Most pure functions are weakly pure (e.g. unless a pure member function is
immutable, then it's weakly pure), so the optimization benefits are pretty
minimal in most cases. And even if a function is strongly pure, the
optimization is only done within an expression (or maybe statement - I can
never remember which), because going beyond that would require data flow
analysis, which the compiler rarely does. As such, pretty much the only time
you end up with function calls being elided thanks to pure is when you have
a strongly pure function where you do something like foo(5) * foo(5). So,
the optimization benefits of pure are pretty minimal. It's that guarantee
about not touching global, mutable state which is the main benefit. As such,
if we were adding pure now, I would strongly argue for calling it something
like @noglobal. I think that it would reduce the confusion considerably. As
it stands, while it helps make functional purity possible in some cases, it
ultimately doesn't have much to do with functional purity in spite of its
name.
> > And perhaps infer uniqueness in some cases for implicit casting to
> > immutable.
>
> Can you expand on that one?
int[] foo()
{
return [1, 2, 3, 4, 5];
}
void main()
{
immutable arr = foo();
}
does not compile, because the return value is mutable, and you can't
implicitly convert int[] to immutable int[]. However,
int[] foo() pure
{
return [1, 2, 3, 4, 5];
}
void main()
{
immutable arr = foo();
}
compiles just fine. Because foo is pure, and the compiler knows that the
return value could not possibly have come from the function's arguments, it
knows that the return value is unique and that it won't violate the type
system to implicitly cast it to immutable. As such, you can write a function
as complicated as you want to create the return value, and so long as the
function is pure, and the compiler can determine that the return value did
not come via an argument, you can implicitly convert the return value to
immutable rather than having to use something like
std.exception.assumeUnique or an explicit cast, which relies on the
programmer verifying that the object being cast is indeed unique rather than
having the compiler guarantee it.
Whether the implicit cast is allowed ultimately depends on the types of the
pure function's parameters and the actual arguments, so it's not always
obvious whether it will work or not, but in general, it works very well for
initializing complex, immutable objects without having to rely on getting
casts right. e.g. this still compiles
Re: Deprecating this(this)
On 01/04/18 03:32, H. S. Teoh wrote: The one nagging question I've been having about pure is: how much are we actually taking advantage of the guarantees provided by pure? My problem is that pure do not provide many guarantees. We have developed very clever ways of extending the traditional definition of pure and invented creative ways of making more things pure, which is all great. Can anyone explain to me what good are the guarantees provided by a function that is pure but not strictly pure? I couldn't find them. But AFAIK the only place where it's actually taken advantage of is to elide some redundant function calls inside a single expression. You cannot even do that unless the function is strictly pure. For all D's extension of the pure concept, it weakened, rather than enhanced, what it means. And perhaps infer uniqueness in some cases for implicit casting to immutable. Can you expand on that one? Shachar
Re: Deprecating this(this)
On Monday, April 02, 2018 00:25:52 Nicholas Wilson via Digitalmars-d wrote:
> On Sunday, 1 April 2018 at 17:08:37 UTC, Andrei Alexandrescu
> wrote:
> > On 4/1/18 10:59 AM, Nicholas Wilson wrote:
> > [...]
> > int[] sneaky;
> > struct A
> > {
> >
> > private int[] innocent;
> > this(this)
> > {
> >
> > sneaky = innocent;
> >
> > }
> >
> > }
> > void main()
> > {
> >
> > immutable a = A([1, 2, 3]);
> > auto b = a;
> > sneaky[1] = 42; // oops
> > import std.stdio;
> > writeln(a.innocent); // ops
> >
> > }
> >
> > Sadly this (and many similar ones) compiles and runs
> > warning-free on today's compiler. We really need to close this
> > loop, like, five years ago.
>
> How much of this class of bug would be eliminated by requiring
> that `this(this)` be pure for assignment to const and immutable
> objects? Arguably this(this) should always be pure in any sane
> program. The only reason I can think of is if you're trying to
> perf the number of copies you're making, but there is compiler
> help for that.
All kinds of things could be done with a postlbit costructor that don't
actually involve copying. These include logging or printing something, and
they could include stuff like reference counting, which may or may not need
access to something external. debug statements would solve some uses cases
but not all. Requiring that this(this) be pure has some of the same issues
that requiring opEquals to be pure or const or whatever has. It makes sense
in _most_ cases, but occasionally, there are good reasons for it not to be -
especially in a systems language.
We have to be _very_ careful about requiring any particular attribute much
of anywhere. Doing so typically causes problems - e.g. those we have with
Object's opEquals, opCmp, toHash, and toString. That decision really needs
to be left up to a particular code base. It's also a big part of why
templates infer attributes. Someone can write their code in such a way that
their code base requires a particular attribute, but in general, language
features shouldn't be requiring any specific attributes, or we'll just be
backing ourselves into another corner.
- Jonathan M Davis
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 17:08:37 UTC, Andrei Alexandrescu
wrote:
On 4/1/18 10:59 AM, Nicholas Wilson wrote:
for 1. consider
immutable foo = ...;
immutable bar = foo;
to be
immutable foo = ...;
immutable bar = () {mutable _ = bitcopy(foo); _.__postblit();
return _;}();
Negative. The problem is typechecking postblit itself, not its
invocation.
Nit sure that I follow but, see comment below on your immutable
example.
for 2. you would have to synchronize anyway for shared, it
makes no difference.
Negative. Consider:
shared struct Point
{
private long x, y, z;
private Mutex mutex;
...
}
Task: define the copy primitive of Point so atomically copy x,
y, and z using mutex. The problem is, the compiler will memcpy
the three longs non-atomically before the user even gets a
crack at intercepting the operation.
What I meant was:
Point p = ...;
auto pp = p; // bit copy + postblit = WRONG
Point pp;
synchronized (p.mutex) { pp = p; } // synchronised: bitcopy is
inseparable from the postblit
Perhaps there is a way to make that automatic / nicer to use?
[...]
int[] sneaky;
struct A
{
private int[] innocent;
this(this)
{
sneaky = innocent;
}
}
void main()
{
immutable a = A([1, 2, 3]);
auto b = a;
sneaky[1] = 42; // oops
import std.stdio;
writeln(a.innocent); // ops
}
Sadly this (and many similar ones) compiles and runs
warning-free on today's compiler. We really need to close this
loop, like, five years ago.
How much of this class of bug would be eliminated by requiring
that `this(this)` be pure for assignment to const and immutable
objects? Arguably this(this) should always be pure in any sane
program. The only reason I can think of is if you're trying to
perf the number of copies you're making, but there is compiler
help for that.
Re: Deprecating this(this)
On 4/1/18 10:59 AM, Nicholas Wilson wrote:
On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote:
There's a mix of fundamental flaws and bugs. I'll get to the flaws in
a second. About the bugs: people have altered their code in various
ways to work with the bizarre semantics of this(this). Now, if we fix
various bugs in this(this) by virtually redefining it, then we'll
break a lot of code in a lot of ways. To wit, we fixed a small issue
and it already created problems:
https://github.com/dlang/dmd/pull/8032. That didn't contribute to the
decision but is quite illustrative.
I found two fundamental flaws with this(this):
1. For immutable objects, typechecking in the presence of successive
modifications of data (first assignment by the compiler, then
modification by the user) is very difficult if not impossible. I don't
know how to do it. The single initialization model (raw/cooked) used
currently in regular immutable constructors works reasonably well and
is robust.
2. For shared objects, the part done by the compiler and the part done
by this(this) should be synchronized together. This makes it
impossible for the user to e.g. define a struct that gets copied
atomically.
See my other reply: but why is it necessary to consider the blit
logically distinct from the postblit w.r.t to program flow observability?
for 1. consider
immutable foo = ...;
immutable bar = foo;
to be
immutable foo = ...;
immutable bar = () {mutable _ = bitcopy(foo); _.__postblit(); return _;}();
Negative. The problem is typechecking postblit itself, not its invocation.
for 2. you would have to synchronize anyway for shared, it makes no
difference.
Negative. Consider:
shared struct Point
{
private long x, y, z;
private Mutex mutex;
...
}
Task: define the copy primitive of Point so atomically copy x, y, and z
using mutex. The problem is, the compiler will memcpy the three longs
non-atomically before the user even gets a crack at intercepting the
operation.
There'd be an additional issue - this(this) is non-templated, which
requires combinatorial additions when qualifiers are present on the
source or destination side.
(Perhaps this is what you're referring to, but all you have said so far
is "this doesn't work and we need to fix it") the post blit is surely
like a destructor: there's only one way to do it, irrespective of the
attributes, especially of the intermediate is considered mutable until
the end of post blit, like static module constructors initialising
global immutables.
Negative. Ignoring qualifiers during copying opens holes in the type
system the size of China. Or at least Australia as it were :o). Consider:
int[] sneaky;
struct A
{
private int[] innocent;
this(this)
{
sneaky = innocent;
}
}
void main()
{
immutable a = A([1, 2, 3]);
auto b = a;
sneaky[1] = 42; // oops
import std.stdio;
writeln(a.innocent); // ops
}
Sadly this (and many similar ones) compiles and runs warning-free on
today's compiler. We really need to close this loop, like, five years ago.
I agree that we should fix any type checking bugs that may be present,
and that we should strive to not make the same mistake twice, however I
wouldn't call either of the above cases a showstopper. You will need to
show more of what is broken and why it is broken given the expected
breakage.
Such discussions will be indeed present in the DIP.
Andrei
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote: Now, if we fix various bugs in this(this) by virtually redefining it, then we'll break a lot of code in a lot of ways. To wit, we fixed a small issue and it already created problems: https://github.com/dlang/dmd/pull/8032. That didn't contribute to the decision but is quite illustrative. Yeah, I absolutely see value in starting fresh with different syntax, even if you were just implementing the same postblit idea again. I found two fundamental flaws with this(this): 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust. I'd think that just letting the const/immutable postblit function see head-mutable fields would work. But maybe that's way harder to implement than writing it down the forum. If that's so, then fair enough. I know that I won't be able to implement it. 2. For shared objects, the part done by the compiler and the part done by this(this) should be synchronized together. This makes it impossible for the user to e.g. define a struct that gets copied atomically. Interesting. I've got no armchair expertise on this one. There'd be an additional issue - this(this) is non-templated, which requires combinatorial additions when qualifiers are present on the source or destination side. I think I don't understand this one. Could you give an example in code? Are you saying that we'd need to define all these: this(this) this(this) const this(this) immutable even if they do the same thing? Wouldn't `this(this) inout` take care of this?
Re: Deprecating this(this)
On Sunday, April 01, 2018 10:31:46 Andrei Alexandrescu via Digitalmars-d wrote: > On 4/1/18 9:37 AM, Jonathan M Davis wrote: > > One issue is that postblit constructors fundamentally don't work with > > const. > Actually they do... How so? In the postblit, you're dealing with a copy of an object where everything is already initialized. Mutating the object would violate const. It could be made to work for primitive types that the compiler understands and knows that the member variable is truly independent - e.g. it could be allowed to mutate an int, or it could be allowed to mutate a pointer while treating what it points to as const, but as soon as you're dealing with user-defined types, you're screwed - especially if you're dealing with something like a struct with a user-defined opAssign. You're reading an existing value and then mutating it, and it has to be at least tail-const, because the original was const - and tail-const is pretty meaningless for structs and can't really be represented in the type system for classes. So, I don't see how postblit could be made to work with a const object of any real complexity. It can be made to work in some corner cases but not in general. Kenji worked on a solution to the problem with const and postblit several years ago (and I'm not sure how close he got to really solving it), but as I recall, you and Walter shot it down because it was overly complicated. How are you proposing that const work with postblit? - Jonathan M Davis
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 14:34:01 UTC, ag0aep6g wrote: On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote: One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway. Ah, you said it much better than I did.
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu
wrote:
There's a mix of fundamental flaws and bugs. I'll get to the
flaws in a second. About the bugs: people have altered their
code in various ways to work with the bizarre semantics of
this(this). Now, if we fix various bugs in this(this) by
virtually redefining it, then we'll break a lot of code in a
lot of ways. To wit, we fixed a small issue and it already
created problems: https://github.com/dlang/dmd/pull/8032. That
didn't contribute to the decision but is quite illustrative.
I found two fundamental flaws with this(this):
1. For immutable objects, typechecking in the presence of
successive modifications of data (first assignment by the
compiler, then modification by the user) is very difficult if
not impossible. I don't know how to do it. The single
initialization model (raw/cooked) used currently in regular
immutable constructors works reasonably well and is robust.
2. For shared objects, the part done by the compiler and the
part done by this(this) should be synchronized together. This
makes it impossible for the user to e.g. define a struct that
gets copied atomically.
See my other reply: but why is it necessary to consider the blit
logically distinct from the postblit w.r.t to program flow
observability?
for 1. consider
immutable foo = ...;
immutable bar = foo;
to be
immutable foo = ...;
immutable bar = () {mutable _ = bitcopy(foo); _.__postblit();
return _;}();
for 2. you would have to synchronize anyway for shared, it makes
no difference.
There'd be an additional issue - this(this) is non-templated,
which requires combinatorial additions when qualifiers are
present on the source or destination side.
(Perhaps this is what you're referring to, but all you have said
so far is "this doesn't work and we need to fix it") the post
blit is surely like a destructor: there's only one way to do it,
irrespective of the attributes, especially of the intermediate is
considered mutable until the end of post blit, like static module
constructors initialising global immutables.
Please note that fixing one or two of these issues doesn't make
this(this) viable - I'm mentioning various issues, each of
which is a showstopper. Nevertheless knowing them is necessary
so we don't make the same mistake again!
Andrei
I agree that we should fix any type checking bugs that may be
present, and that we should strive to not make the same mistake
twice, however I wouldn't call either of the above cases a
showstopper. You will need to show more of what is broken and why
it is broken given the expected breakage.
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote: One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. To cleanly deal with const, you need something more like a copy constructor where you initialize it with the adjusted values directly rather than mutating the copy. I've always wondered about that, is the difference between that anything more than philosophical? Put another way if a this(this) is weakly pure, is there any safety issues with the compiler permitting the mutation on a (non-shared? not sure if this would be a requirement) const object? I'm not sure what the spec says, but if you take the view that the const object is no fully initialised until the postblit is done, then I don't see the problem.
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote: One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that? The destination can't be const/immutable already, or you wouldn't be able to write there anyway.
Re: Deprecating this(this)
On 4/1/18 9:37 AM, Jonathan M Davis wrote: One issue is that postblit constructors fundamentally don't work with const. Actually they do...
Re: Deprecating this(this)
On 4/1/18 8:55 AM, ag0aep6g wrote: On 04/01/2018 03:08 AM, Andrei Alexandrescu wrote: On 3/31/18 8:32 PM, H. S. Teoh wrote: [...] What exactly is it about this(this) that blocks us from doing that? See the updated docs. Too many bugs in design and implementation. Removing this(this) is going to be a huge breaking change far bigger than, say, removing autodecoding ever will be. We're not removing it as much as evolving it: we define an alternate copying mechanism, and once that is in tip-top shape, we deprecate this(this). Is there a fundamental flaw in the postblit idea, or are you just going to give postblit a new syntax, and try to avoid all the issues that `this(this)` currently has? If there's a fundamental flaw, I'd be interested in what it is. I can't make it out in your additions to the spec, if it's in there. I can see that `this(this)` is a mess, but it also looks like a lot could be fixed. For example, how it interacts with const/immutable is ridiculous, but that could probably be fixed. If you're just going for a clean slate, I can see the appeal. You avoid dealing with the hard breakage that fixing `this(this)` would most probably bring. There's a mix of fundamental flaws and bugs. I'll get to the flaws in a second. About the bugs: people have altered their code in various ways to work with the bizarre semantics of this(this). Now, if we fix various bugs in this(this) by virtually redefining it, then we'll break a lot of code in a lot of ways. To wit, we fixed a small issue and it already created problems: https://github.com/dlang/dmd/pull/8032. That didn't contribute to the decision but is quite illustrative. I found two fundamental flaws with this(this): 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust. 2. For shared objects, the part done by the compiler and the part done by this(this) should be synchronized together. This makes it impossible for the user to e.g. define a struct that gets copied atomically. There'd be an additional issue - this(this) is non-templated, which requires combinatorial additions when qualifiers are present on the source or destination side. Please note that fixing one or two of these issues doesn't make this(this) viable - I'm mentioning various issues, each of which is a showstopper. Nevertheless knowing them is necessary so we don't make the same mistake again! Andrei
Re: Deprecating this(this)
On Sunday, April 01, 2018 14:55:07 ag0aep6g via Digitalmars-d wrote: > On 04/01/2018 03:08 AM, Andrei Alexandrescu wrote: > > On 3/31/18 8:32 PM, H. S. Teoh wrote: > [...] > > >> What exactly is it about this(this) that blocks us from doing that? > > > > See the updated docs. Too many bugs in design and implementation. > > > >> Removing this(this) is going to be a huge breaking change far bigger > >> than, say, removing autodecoding ever will be. > > > > We're not removing it as much as evolving it: we define an alternate > > copying mechanism, and once that is in tip-top shape, we deprecate > > this(this). > > Is there a fundamental flaw in the postblit idea, or are you just going > to give postblit a new syntax, and try to avoid all the issues that > `this(this)` currently has? > > If there's a fundamental flaw, I'd be interested in what it is. I can't > make it out in your additions to the spec, if it's in there. I can see > that `this(this)` is a mess, but it also looks like a lot could be > fixed. For example, how it interacts with const/immutable is ridiculous, > but that could probably be fixed. One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. To cleanly deal with const, you need something more like a copy constructor where you initialize it with the adjusted values directly rather than mutating the copy. > If you're just going for a clean slate, I can see the appeal. You avoid > dealing with the hard breakage that fixing `this(this)` would most > probably bring. Avoiding any breakage would be ideal, but it's unlikely that that can be done if we want to make copying const objects work properly - not unless we did something like use postblit constructors for mutable objects and copy constructors for const objects, and that would just cause other problems (including having to duplicate the code that deals with copying an object if it's going to work with const). On the bright side, if we're replacing postblit constructors with some other type of constructor for copying, it should be a pretty straightforward process. But there isn't much point in worrying about how much breakage there's going to be before we really know where we want to go with this. At this point, it's just clear that as things stand, postblit constructors have some definite problems (some which are implementation issues and some which are language design issues), and based on past discussions on this and previous attempts to fix some of the problems with postblit constructors, it seems pretty unlikely that we can fully fix postblit constructors. I'm sure that we could fix some of the issues, but others (most notably, the issues with const) seem pretty intractable. - Jonathan M Davis
Re: Deprecating this(this)
On 04/01/2018 03:08 AM, Andrei Alexandrescu wrote: On 3/31/18 8:32 PM, H. S. Teoh wrote: [...] What exactly is it about this(this) that blocks us from doing that? See the updated docs. Too many bugs in design and implementation. Removing this(this) is going to be a huge breaking change far bigger than, say, removing autodecoding ever will be. We're not removing it as much as evolving it: we define an alternate copying mechanism, and once that is in tip-top shape, we deprecate this(this). Is there a fundamental flaw in the postblit idea, or are you just going to give postblit a new syntax, and try to avoid all the issues that `this(this)` currently has? If there's a fundamental flaw, I'd be interested in what it is. I can't make it out in your additions to the spec, if it's in there. I can see that `this(this)` is a mess, but it also looks like a lot could be fixed. For example, how it interacts with const/immutable is ridiculous, but that could probably be fixed. If you're just going for a clean slate, I can see the appeal. You avoid dealing with the hard breakage that fixing `this(this)` would most probably bring.
Re: Deprecating this(this)
On Sunday, April 01, 2018 12:23:29 Per Nordlöw via Digitalmars-d wrote:
> On Sunday, 1 April 2018 at 01:56:40 UTC, Jonathan M Davis wrote:
> > Another potential issue is whether any of this does or should
> > relate to
> >
> > https://github.com/dlang/DIPs/pull/109
> >
> > and it's solution for hooking into to moves. I'm not at all
> > sure that what happens with that needs to be related to this at
> > all, but it might.
> >
> > - Jonathan M Davis
>
> And before we think about `opMove` we should, IMO, make the
> compiler pass by move in more cases, for instance, in range
> constructors such as
>
> this(Source source)
> {
> this.source = source; // last occurrence of `source` can be
> moved
> }
>
> I'd be happy to help out with adding this in dmd. Andrei has
> already showed interest in this idea.
I don't see what that has to do with opMove other than the fact that in such
cases, you'd then end up with opMove being called instead of the postblit
constructor. It seems to me that whether opMove is part of the language or
not when such improvements are made is irrelevant. They should be able to be
done independently of one other.
As I understand it, the motivations behind opMove really have nothing to do
with how often moves are made vs copying. It has to do with types that have
serious issues if they're moved. The Weka guys (and probably others) have
use cases where the fact that an object is moved without any way for them to
detect it and do stuff like update pointers to the object is a serious
problem. If anything, their use case might be better off right now if _no_
moves were ever done (at least until something like opMove is in the
language), since the moves result in bugs.
- Jonathan M Davis
Re: Deprecating this(this)
On Sunday, April 01, 2018 11:37:21 Guillaume Piolat via Digitalmars-d wrote: > On Sunday, 1 April 2018 at 01:01:24 UTC, Jonathan M Davis wrote: > > So, I think that the only large-scale benefit thet exists for > > pure and really can exist for pure is the fact that you know > > that the function doesn't access global, mutable state. > > Everything else it does is just gravy and too limited to be a > > "large-scale" benefit. Certainly, optimizations are clearly > > _not_ the main benefit of pure, since they almost don't exist. > > But over time, we have managed to add more gravy here and there > > as we've figured out assumptions that can be made based on pure > > (like the case where we can convert the result of a pure > > function to immutable). > > > > - Jonathan M Davis > > Great write-up. > > Why keep pure when the benefits obviously don't outweight the > costs? > Here is what I interpret reading this: "lost productivity". Personally, I don't think that the costs outweigh the benefits. Yes, if your primary goal is optimizations, then pure falls flat on its face, but personally, I think that the fact that pure allows you to prove that a function doesn't access any global, mutable state except through its arguments is quite valuable. There are still some things in druntime and Phobos which don't work with pure like they should, but in general, I haven't found that it's really a big problem. Some code can't be pure, and that's life, but a _lot_ can be. If anything, I find that const is the attribute that causes problems, not pure. But if you don't want to use pure, then don't use pure. Nothing is forcing you to use it, and if someone else insists on making their code work with pure, it's only a problem for you if they're trying to call your code, and they complain about the fact that your code doesn't work with pure. Certainly, code that isn't pure has no problems calling code that is, so if other folks go to the effort of making their code work with pure, it won't cause you problems. - Jonathan M Davis
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 01:56:40 UTC, Jonathan M Davis wrote:
Another potential issue is whether any of this does or should
relate to
https://github.com/dlang/DIPs/pull/109
and it's solution for hooking into to moves. I'm not at all
sure that what happens with that needs to be related to this at
all, but it might.
- Jonathan M Davis
And before we think about `opMove` we should, IMO, make the
compiler pass by move in more cases, for instance, in range
constructors such as
this(Source source)
{
this.source = source; // last occurrence of `source` can be
moved
}
I'd be happy to help out with adding this in dmd. Andrei has
already showed interest in this idea.
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 01:01:24 UTC, Jonathan M Davis wrote: So, I think that the only large-scale benefit thet exists for pure and really can exist for pure is the fact that you know that the function doesn't access global, mutable state. Everything else it does is just gravy and too limited to be a "large-scale" benefit. Certainly, optimizations are clearly _not_ the main benefit of pure, since they almost don't exist. But over time, we have managed to add more gravy here and there as we've figured out assumptions that can be made based on pure (like the case where we can convert the result of a pure function to immutable). - Jonathan M Davis Great write-up. Why keep pure when the benefits obviously don't outweight the costs? Here is what I interpret reading this: "lost productivity".
Re: Deprecating this(this)
On Saturday, 31 March 2018 at 23:38:06 UTC, Andrei Alexandrescu wrote: * immutable and const are very difficult, but we have an attack (assuming copy construction gets taken care of) * pure is difficult How about removing pure, const and immutable?
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 10:49:22 UTC, bachmeier wrote: On Sunday, 1 April 2018 at 10:04:04 UTC, Johannes Loher wrote: This seems really sudden, april fool's joke? Not really sure, as there are real problems with this(this)... What I was wondering too. I mean, breaking changes just don't happen to this language. Now there will be, without even an indication of how existing code would have to be rewritten, or how this large-scale breakage is different than the breakages that just can't happen because reasons. I guess that's why there's always the disclaimer, "We'll only break code if there's a really good reason." That reason is "in case we want to". Nothing has been lay out yet and people are already freaking out. No wonder nothing gets done anymore.
Re: Deprecating this(this)
On Sunday, 1 April 2018 at 10:04:04 UTC, Johannes Loher wrote: This seems really sudden, april fool's joke? Not really sure, as there are real problems with this(this)... What I was wondering too. I mean, breaking changes just don't happen to this language. Now there will be, without even an indication of how existing code would have to be rewritten, or how this large-scale breakage is different than the breakages that just can't happen because reasons. I guess that's why there's always the disclaimer, "We'll only break code if there's a really good reason." That reason is "in case we want to".
Re: Deprecating this(this)
On 4/1/18 6:04 AM, Johannes Loher wrote: This seems really sudden, april fool's joke? Not really sure, as there are real problems with this(this)... I'm glad I've sent it yesterday then at least in my time zone :o). This looks sudden but isn't. Eduard and I have been blocked by this problem seriously whilst working on the collections library. Andrei
Re: Deprecating this(this)
Am 01.04.2018 um 01:38 schrieb Andrei Alexandrescu: > We need to have a simple recipe on how to define a canonical object. > That would include requirements such as: > > * should work with mutable, const, immutable, and shared > * the right way to define constructor(s) > * the right way to define copying > * the right way to define destructor(s) > > I've asked my student Razvan to document the behavior of this(this). His > work: > > https://github.com/dlang/dmd/pull/8055 > https://github.com/dlang/dlang.org/pull/2281 > https://github.com/dlang/dlang.org/pull/2299 > > ... reveals a puzzling array of behaviors. Sometimes the typechecking is > wrong, too. > > I think it's very important for us to have a simple, correct, and > canonical way of defining structs in the D language that work with the > language features: qualifiers, pure, safe, and nogc. > > Once we have that, we can encapsulate desirable abstractions (such as > @nogc safe collections that work in pure code), regardless of how > difficult their implementations might be. It seems that currently > this(this) does not allow us to do that. > > Eduard, another student I work with, has made steps toward a collections > library that rant into difficulties: > > * @safe is achievable with relative ease > > * immutable and const are very difficult, but we have an attack > (assuming copy construction gets taken care of) > > * @nogc is doable with a couple of conventions for allocators > > * pure is difficult > > * making them work together is very difficult > > We need to offer features, tools, and guidance toward creating simple > encapsulated types that work well with D's own abstractions. Once we > have that we can build libraries to work satisfactorily for any domain. > > I think the way to move forward is to deprecate this(this) entirely and > create a DIP that allows people to define truly encapsulated structs. > This is important, urgent, and of huge impact. > > I am looking for folks to assist me in creating a DIP for that. There > will be a _lot_ of work involved, so don't take it lightly. > > > Thanks, > > Andrei This seems really sudden, april fool's joke? Not really sure, as there are real problems with this(this)...
Re: Deprecating this(this)
On Saturday, March 31, 2018 19:38:06 Andrei Alexandrescu via Digitalmars-d wrote: > We need to have a simple recipe on how to define a canonical object. > That would include requirements such as: > > * should work with mutable, const, immutable, and shared > * the right way to define constructor(s) > * the right way to define copying > * the right way to define destructor(s) > > I've asked my student Razvan to document the behavior of this(this). His > work: > > https://github.com/dlang/dmd/pull/8055 > https://github.com/dlang/dlang.org/pull/2281 > https://github.com/dlang/dlang.org/pull/2299 > > ... reveals a puzzling array of behaviors. Sometimes the typechecking is > wrong, too. > > I think it's very important for us to have a simple, correct, and > canonical way of defining structs in the D language that work with the > language features: qualifiers, pure, safe, and nogc. > > Once we have that, we can encapsulate desirable abstractions (such as > @nogc safe collections that work in pure code), regardless of how > difficult their implementations might be. It seems that currently > this(this) does not allow us to do that. > > Eduard, another student I work with, has made steps toward a collections > library that rant into difficulties: > > * @safe is achievable with relative ease > > * immutable and const are very difficult, but we have an attack > (assuming copy construction gets taken care of) > > * @nogc is doable with a couple of conventions for allocators > > * pure is difficult > > * making them work together is very difficult > > We need to offer features, tools, and guidance toward creating simple > encapsulated types that work well with D's own abstractions. Once we > have that we can build libraries to work satisfactorily for any domain. > > I think the way to move forward is to deprecate this(this) entirely and > create a DIP that allows people to define truly encapsulated structs. > This is important, urgent, and of huge impact. > > I am looking for folks to assist me in creating a DIP for that. There > will be a _lot_ of work involved, so don't take it lightly. Another potential issue is whether any of this does or should relate to https://github.com/dlang/DIPs/pull/109 and it's solution for hooking into to moves. I'm not at all sure that what happens with that needs to be related to this at all, but it might. - Jonathan M Davis
Re: Deprecating this(this)
On 01/04/2018 11:38 AM, Andrei Alexandrescu wrote:
We need to have a simple recipe on how to define a canonical object.
That would include requirements such as:
* should work with mutable, const, immutable, and shared
* the right way to define constructor(s)
* the right way to define copying
* the right way to define destructor(s)
We will also need to make this consistent for classes.
***
So lets go back to the basics.
From the primordial goo that is My Own Little Language (MOLL heavily WIP):
__userType [Identifier:name] {
(Declaration|Definition)*
}
Is a value type, acts pretty much like struct.
Why do I not call it a struct? Because it shouldn't be used in user
code. This exists only in mental models for D users currently.
So something like:
__userType Foo {
int x;
void __constructor(int x) {
this.x = x;
}
void __postblit() {
this.x = x+1;
}
void __deconstructor() {
// free?
}
}
Is comparable to D code of:
struct Foo {
int x;
this(this) {
this.x = x + 1;
}
~this() {
// free?
}
}
So what does a class look like?
__userType Foo {
enum __IS_CLASS = true;
static TypeInfo_Class TypeInfo = TypeInfo_Class(...);
static void*[][] __vtables = [
[&TypeInfo, ...],
[&TypeInfo, ...]
];
void*[] __vtable;
void* __ptr;
auto __cast(T)() if(T.__IS_CLASS) {
// ...
}
}
Why is this important? Because it shows that our current lifetime
management strategies for ref counting with structs can be applied
directly to classes.
***
At this point in time, lets look at how each qualifier is perceived:
mutable: default, good, everything else is a pain
const: library code and the author doesn't want you to go touchy
immutable: in binary only, quite yucky
shared: a message from the library author and not much else
I do not believe we can continue this conversation with the above
perceptions.
First and foremost qualifiers are there to tell other developers (and
yourself in the future) what you intend on doing with a given bit of
memory. Now we do need a way to express head-const. Because it allows us
to say that we personally can't modify a bit of memory but we can call a
method that does (from it).
Good engineering requires us to consider psychology on these issues. To
create a single unified view of this, will be tricky I think, but well
worth it.
Re: Deprecating this(this)
On 3/31/18 8:32 PM, H. S. Teoh wrote:
On Sat, Mar 31, 2018 at 07:38:06PM -0400, Andrei Alexandrescu via Digitalmars-d
wrote:
[...]
Once we have that, we can encapsulate desirable abstractions (such as
@nogc safe collections that work in pure code), regardless of how
difficult their implementations might be. It seems that currently
this(this) does not allow us to do that.
What exactly is it about this(this) that blocks us from doing that?
See the updated docs. Too many bugs in design and implementation.
Removing this(this) is going to be a huge breaking change far bigger
than, say, removing autodecoding ever will be.
We're not removing it as much as evolving it: we define an alternate
copying mechanism, and once that is in tip-top shape, we deprecate
this(this).
A lot of us here have essentially given up on const except for a few
very narrow cases. The transitive nature of const makes it extremely
difficult to work with in the general case, even though the simplest use
cases are workable.
Immutable is where it's at, in terms of usefulness. Const is a mere
servant of it (and of mutable).
One of the biggest stumbling blocks is that whenever ranges are
involved, const is practically out of the question, because even though
it can be made to work for most cases, there will almost always be that
one pathological case where it's impossible / too hard to work around,
and that ruins it for everything else, so that it's much simpler to just
avoid it altogether.
Yah, the DIP might address that, too. Consider:
void fun(R)(R arr)
{
pragma(msg, typeof(arr));
}
void main()
{
immutable(int[]) arr = [ 1, 2, 3 ];
pragma(msg, typeof(arr));
fun(arr);
}
The program prints during compilation:
immutable(int[])
immutable(int)[]
Interesting! So the type of the array changes during template matching,
which is an exception to the rule that templates always glom to the
exact type passed.
This is a hack introduced in the compiler in response to the issues you
mention. But we don't need hacks and special casing - we need a means
for types to say "here's what needs to happen when a template parameter
is matched against this type".
So the DIP would address manipulating qualified ranges as a perk.
The one nagging question I've been having about pure is: how much are we
actually taking advantage of the guarantees provided by pure?
Very little, but that doesn't matter. The problem is it's
underspecified. So now it's like a vague threat - whenever we mess with
fear somebody comes asking, but what about an aggressive compiler doing
some unexpected optimizations based on such and such interpretation?
We need to lock pure down.
Andrei
Re: Deprecating this(this)
On 3/31/18 9:01 PM, Jonathan M Davis wrote: And as far as auto-decoding goes Let's keep this about construction and not about auto-decoding. Thanks.
Re: Deprecating this(this)
On Saturday, March 31, 2018 17:32:10 H. S. Teoh via Digitalmars-d wrote: > On Sat, Mar 31, 2018 at 07:38:06PM -0400, Andrei Alexandrescu via > Digitalmars-d wrote: [...] > > > Once we have that, we can encapsulate desirable abstractions (such as > > @nogc safe collections that work in pure code), regardless of how > > difficult their implementations might be. It seems that currently > > this(this) does not allow us to do that. > > What exactly is it about this(this) that blocks us from doing that? > > Removing this(this) is going to be a huge breaking change far bigger > than, say, removing autodecoding ever will be. Well, as far const goes, you have the inherent problem that the object you have in this(this) has already been initialized, and you can't mutate const. I think that other attribute issues are mostly implementation issues. And as far as auto-decoding goes, removing postblit constructors would actually be far easier, because it's simply a matter of deprecating a particular function. Some traits that check for postblits would have to be adjusted, but all those really care about is that there's extra code beyond the default copying code, and they could easily be made to work with both postblits and whatever the new solution is at the same time until postblits are actually gone. Auto-decoding on the other hand, affects far more than just the functions in std.range.primitives, and there is no clear deprecation path. So, while it _might_ be true that deprecating postblits would break more code (and I'm honestly not convinced that it would), postblit constructors would actually have a clean deprecation path. So, while it breaks code, it does so in a way that can easily be managed, whereas the removal of auto-decoding is not straightforward at all. It could be done by just flipping the switch so-to-speak, but AFAIK, no one has yet presented a workable deprecation path. And ultimately, I think that _that_ is what's preventing us from fixing auto-decoding. If a clean deprecation path were found, then we could discuss whether the resulting breakage would be worth it, but without a clean deprecation path, I don't see how we could ever do it. Also, unless we can fix postblit constructors (and the evidence thus far is that if we can, it's way too complicated - e.g. Kenji's attempt several years ago was rejected because it was way too complicated), if we don't switch to a differnt solution, we're talking about permanently not supporting copying const types that require user-defined copying. > A lot of us here have essentially given up on const except for a few > very narrow cases. The transitive nature of const makes it extremely > difficult to work with in the general case, even though the simplest use > cases are workable. > > One of the biggest stumbling blocks is that whenever ranges are > involved, const is practically out of the question, because even though > it can be made to work for most cases, there will almost always be that > one pathological case where it's impossible / too hard to work around, > and that ruins it for everything else, so that it's much simpler to just > avoid it altogether. I don't think that it's even the case that it can be made to work in most cases - or if it can, it involves a lot of static ifs. The range API does not require const for _anything_ (and really can't due to how restricted const is), so _no_ generic range-based code can assume that even something like length or empty can be called if the range is const or inout. As such, the only ranges that can mark anything with const are ones that either aren't generic or which have a bunch of static ifs presenting const and non-const versions depending on the template arguments, and IMHO, that's just not workable. I've done it before, and it's a mess. As such, ranges and const really don't work together at all. const really only works when you're dealing with a very constrainted set of types where you can actually guarantee that they work with const. > > * pure is difficult > > [...] > > The one nagging question I've been having about pure is: how much are we > actually taking advantage of the guarantees provided by pure? We have > developed very clever ways of extending the traditional definition of > pure and invented creative ways of making more things pure, which is all > great. But AFAIK the only place where it's actually taken advantage of > is to elide some redundant function calls inside a single expression. > And perhaps infer uniqueness in some cases for implicit casting to > immutable. > > While these are indisputably useful, they seem so far to be only > relatively minor benefits. If pure is indeed so difficult to support > generically, it begs the question, is it worth the effort just to gain > these niggling benefits? Whatever happened to larger-scale benefits > conferred by purity? Honestly, I think that the main benefit of pure is that if you know that a function is pure, you know that it doesn'
Re: Deprecating this(this)
On 3/31/18 8:25 PM, Jonathan M Davis wrote: So, is the idea then that we'd switch to copy constructors? Something like that. We'll need to define them carefully to obey safety and purity.
Re: Deprecating this(this)
On Sat, Mar 31, 2018 at 07:38:06PM -0400, Andrei Alexandrescu via Digitalmars-d wrote: [...] > Once we have that, we can encapsulate desirable abstractions (such as > @nogc safe collections that work in pure code), regardless of how > difficult their implementations might be. It seems that currently > this(this) does not allow us to do that. What exactly is it about this(this) that blocks us from doing that? Removing this(this) is going to be a huge breaking change far bigger than, say, removing autodecoding ever will be. > Eduard, another student I work with, has made steps toward a > collections library that rant into difficulties: > > * @safe is achievable with relative ease > > * immutable and const are very difficult, but we have an attack > (assuming copy construction gets taken care of) A lot of us here have essentially given up on const except for a few very narrow cases. The transitive nature of const makes it extremely difficult to work with in the general case, even though the simplest use cases are workable. One of the biggest stumbling blocks is that whenever ranges are involved, const is practically out of the question, because even though it can be made to work for most cases, there will almost always be that one pathological case where it's impossible / too hard to work around, and that ruins it for everything else, so that it's much simpler to just avoid it altogether. Also, as far as containers are concerned, the lack of a standard way to construct head-mutable / tail-const types that works analogously with built-in arrays makes it very difficult to write generic containers that work well with const/immutable. It's not too hard to make it work for specific types, but very difficult to write a truly *generic* container that can be deployed in all situations where const/immutable are involved. [...] > * pure is difficult [...] The one nagging question I've been having about pure is: how much are we actually taking advantage of the guarantees provided by pure? We have developed very clever ways of extending the traditional definition of pure and invented creative ways of making more things pure, which is all great. But AFAIK the only place where it's actually taken advantage of is to elide some redundant function calls inside a single expression. And perhaps infer uniqueness in some cases for implicit casting to immutable. While these are indisputably useful, they seem so far to be only relatively minor benefits. If pure is indeed so difficult to support generically, it begs the question, is it worth the effort just to gain these niggling benefits? Whatever happened to larger-scale benefits conferred by purity? T -- Computers aren't intelligent; they only think they are.
Re: Deprecating this(this)
On Saturday, March 31, 2018 19:38:06 Andrei Alexandrescu via Digitalmars-d wrote: > We need to have a simple recipe on how to define a canonical object. > That would include requirements such as: > > * should work with mutable, const, immutable, and shared > * the right way to define constructor(s) > * the right way to define copying > * the right way to define destructor(s) > > I've asked my student Razvan to document the behavior of this(this). His > work: > > https://github.com/dlang/dmd/pull/8055 > https://github.com/dlang/dlang.org/pull/2281 > https://github.com/dlang/dlang.org/pull/2299 > > ... reveals a puzzling array of behaviors. Sometimes the typechecking is > wrong, too. > > I think it's very important for us to have a simple, correct, and > canonical way of defining structs in the D language that work with the > language features: qualifiers, pure, safe, and nogc. > > Once we have that, we can encapsulate desirable abstractions (such as > @nogc safe collections that work in pure code), regardless of how > difficult their implementations might be. It seems that currently > this(this) does not allow us to do that. > > Eduard, another student I work with, has made steps toward a collections > library that rant into difficulties: > > * @safe is achievable with relative ease > > * immutable and const are very difficult, but we have an attack > (assuming copy construction gets taken care of) > > * @nogc is doable with a couple of conventions for allocators > > * pure is difficult > > * making them work together is very difficult > > We need to offer features, tools, and guidance toward creating simple > encapsulated types that work well with D's own abstractions. Once we > have that we can build libraries to work satisfactorily for any domain. > > I think the way to move forward is to deprecate this(this) entirely and > create a DIP that allows people to define truly encapsulated structs. > This is important, urgent, and of huge impact. > > I am looking for folks to assist me in creating a DIP for that. There > will be a _lot_ of work involved, so don't take it lightly. So, is the idea then that we'd switch to copy constructors? - Jonathan M Davis
Deprecating this(this)
We need to have a simple recipe on how to define a canonical object. That would include requirements such as: * should work with mutable, const, immutable, and shared * the right way to define constructor(s) * the right way to define copying * the right way to define destructor(s) I've asked my student Razvan to document the behavior of this(this). His work: https://github.com/dlang/dmd/pull/8055 https://github.com/dlang/dlang.org/pull/2281 https://github.com/dlang/dlang.org/pull/2299 ... reveals a puzzling array of behaviors. Sometimes the typechecking is wrong, too. I think it's very important for us to have a simple, correct, and canonical way of defining structs in the D language that work with the language features: qualifiers, pure, safe, and nogc. Once we have that, we can encapsulate desirable abstractions (such as @nogc safe collections that work in pure code), regardless of how difficult their implementations might be. It seems that currently this(this) does not allow us to do that. Eduard, another student I work with, has made steps toward a collections library that rant into difficulties: * @safe is achievable with relative ease * immutable and const are very difficult, but we have an attack (assuming copy construction gets taken care of) * @nogc is doable with a couple of conventions for allocators * pure is difficult * making them work together is very difficult We need to offer features, tools, and guidance toward creating simple encapsulated types that work well with D's own abstractions. Once we have that we can build libraries to work satisfactorily for any domain. I think the way to move forward is to deprecate this(this) entirely and create a DIP that allows people to define truly encapsulated structs. This is important, urgent, and of huge impact. I am looking for folks to assist me in creating a DIP for that. There will be a _lot_ of work involved, so don't take it lightly. Thanks, Andrei
