Re: Head Const
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. I'll leave to others to discuss whether a language solution is worth it, but I just wanted to point out there are two library solutions in this area: HeadConst [1] - PR pending, and Rebindable (tail const) [2] - already part of Phobos. Maybe if the language could express head const, it also makes sense to have tail const. [1]: https://github.com/D-Programming-Language/phobos/pull/3862 [2]: http://dlang.org/phobos-prerelease/std_typecons.html#.Rebindable
Re: Head Const
On 2/15/2016 3:11 PM, ZombineDev wrote: I'll leave to others to discuss whether a language solution is worth it, but I just wanted to point out there are two library solutions in this area: HeadConst [1] - PR pending, and Rebindable (tail const) [2] - already part of Phobos. Maybe if the language could express head const, it also makes sense to have tail const. [1]: https://github.com/D-Programming-Language/phobos/pull/3862 [2]: http://dlang.org/phobos-prerelease/std_typecons.html#.Rebindable Transitive const is tail const.
Re: Head Const
On Monday, 15 February 2016 at 23:28:28 UTC, Walter Bright wrote: Transitive const is tail const. const(char)[] foo is tail const const(Object) ref is the analog the language is missing so phobos tries to catch up
Re: Head Const
On Monday, 15 February 2016 at 23:28:28 UTC, Walter Bright wrote: On 2/15/2016 3:11 PM, ZombineDev wrote: I'll leave to others to discuss whether a language solution is worth it, but I just wanted to point out there are two library solutions in this area: HeadConst [1] - PR pending, and Rebindable (tail const) [2] - already part of Phobos. Maybe if the language could express head const, it also makes sense to have tail const. [1]: https://github.com/D-Programming-Language/phobos/pull/3862 [2]: http://dlang.org/phobos-prerelease/std_typecons.html#.Rebindable Transitive const is tail const. I used the term "tail const" as a mutable pointer/reference to const/immutable object, where transitivity starts from the object (not from the pointer, which is what currently happens if you type `const Object o`).
Re: Head Const
On 2/15/2016 3:39 PM, ZombineDev wrote: I used the term "tail const" as a mutable pointer/reference to const/immutable object, where transitivity starts from the object (not from the pointer, which is what currently happens if you type `const Object o`). Makes sense.
Re: Head Const
On Monday, 15 February 2016 at 23:39:56 UTC, ZombineDev wrote: On Monday, 15 February 2016 at 23:28:28 UTC, Walter Bright wrote: On 2/15/2016 3:11 PM, ZombineDev wrote: I'll leave to others to discuss whether a language solution is worth it, but I just wanted to point out there are two library solutions in this area: HeadConst [1] - PR pending, and Rebindable (tail const) [2] - already part of Phobos. Maybe if the language could express head const, it also makes sense to have tail const. [1]: https://github.com/D-Programming-Language/phobos/pull/3862 [2]: http://dlang.org/phobos-prerelease/std_typecons.html#.Rebindable Transitive const is tail const. I used the term "tail const" as a mutable pointer/reference to const/immutable object, where transitivity starts from the object (not from the pointer, which is what currently happens if you type `const Object o`). Just to clarify: struct S; const(S)* mutablePointerToS; const(S)[] mutableArrayOfConstObjects; Are examles of tail const. The only missing piece is the ability to have a tail-const references to class objects.
Re: Head Const
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. so keep const as it is and add headconst?
Re: Head Const
On Mon, Feb 15, 2016 at 11:45:26PM +, ZombineDev via Digitalmars-d wrote: [...] > >I used the term "tail const" as a mutable pointer/reference to > >const/immutable object, where transitivity starts from the object > >(not from the pointer, which is what currently happens if you type > >`const Object o`). > > Just to clarify: > struct S; > const(S)* mutablePointerToS; > const(S)[] mutableArrayOfConstObjects; > > Are examles of tail const. The only missing piece is the ability to > have a tail-const references to class objects. Just out of curiosity, since we already have Rebindable in Phobos and HeadConst is being proposed, what are the disadvantages / shortcomings of a library solution that would justify adding yet another feature to the language? T -- Today's society is one of specialization: as you grow, you learn more and more about less and less. Eventually, you know everything about nothing.
Re: Head Const
On 16/02/2016 9:48 AM, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) I'd much rather improve pragma(mangle) than add more C++ features to D. 2. supports single assignment style of programming, even if the data is otherwise mutable Like 'final'? We did get rid of that...
Re: Head Const
On Mon, 15 Feb 2016 16:14:23 -0800, H. S. Teoh via Digitalmars-d wrote: > Just out of curiosity, since we already have Rebindable in Phobos and > HeadConst is being proposed, what are the disadvantages / shortcomings > of a library solution that would justify adding yet another feature to > the language? C++ interfacing requires either a well-known type that the compiler special-cases or a builtin attribute. D could ignore this for all purposes besides name mangling, though, and that would be an okay compromise.
Re: Head Const
On Tue, Feb 16, 2016 at 12:21:38AM +, Chris Wright via Digitalmars-d wrote: > On Mon, 15 Feb 2016 16:14:23 -0800, H. S. Teoh via Digitalmars-d wrote: > > Just out of curiosity, since we already have Rebindable in Phobos > > and HeadConst is being proposed, what are the disadvantages / > > shortcomings of a library solution that would justify adding yet > > another feature to the language? > > C++ interfacing requires either a well-known type that the compiler > special-cases or a builtin attribute. D could ignore this for all > purposes besides name mangling, though, and that would be an okay > compromise. What about besides C++ integration? 'cos I remember some people were complaining that a library solution is bad, but I've forgotten what the reasons were. My thinking is that if the current library solution is bad, or messy, or whatever, and other such library solutions exhibit similar problems, then perhaps what we should be looking at is how to make the language more conducive to implementing nice, usable library solutions to this and a host of other problems, rather than throwing everything and the kitchen sink into the language and end up with an unmaintainable mess. T -- This sentence is false.
Re: Head Const
On 2/15/2016 4:15 PM, Daniel Murphy wrote: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) I'd much rather improve pragma(mangle) than add more C++ features to D. It's currently difficult to interface with C++, and always will be, but smoothing out what we can can be a big opportunity for D. 2. supports single assignment style of programming, even if the data is otherwise mutable Like 'final'? We did get rid of that... Maybe we should resurrect it.
Re: Head Const
On Mon, 15 Feb 2016 16:57:44 -0800, H. S. Teoh via Digitalmars-d wrote: > What about besides C++ integration? 'cos I remember some people were > complaining that a library solution is bad, but I've forgotten what the > reasons were. The first problem mentioned was C++ integration, and the minimal thing required to make that work in a nice way is to supply a @cppconst attribute with no semantics that is used only to alter name mangling. It's the least complex solution available. I did get tripped up not too long ago on a lack of a `final` storage class in the context of members. It is sometimes useful. There's also a question of whether we want the same thing for C++ name mangling as for preventing reassignment. It might be clearer to have both final and @cppconst.
Re: Head Const
On Tuesday, 16 February 2016 at 01:04:44 UTC, Walter Bright wrote: On 2/15/2016 4:15 PM, Daniel Murphy wrote: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) I'd much rather improve pragma(mangle) than add more C++ features to D. It's currently difficult to interface with C++, and always will be, but smoothing out what we can can be a big opportunity for D. Hmm, I wonder if Stroustrup said something similar.
Re: Head Const
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. Straight up head-const is utterly useless IMHO. That's what Java has. C++ has something far more complicated where any individual piece (or pieces) of a pointer type can be declared const without necessarily making the whole thing const. So, _that_ is what we'd need to interact with C++, and it's a downright ugly feature IMHO. Tail-const has some value, because it allows you to refer to a const object with a mutable pointer or reference, whereas head-const just makes things ugly. So, if it weren't for interacting with C++, I would give a resounding "no" to this. It's not even vaguely worth the complexity. As for interoperability with C++, I don't know. Previously, you stated that we weren't going to do things like support interacting with C++ templates (aside from specific instantiations which weren't typed as templates), because it would mean putting a C++ compiler into D, which you didn't want to do. But increasingly, it seems like you're heading in the direction of doing that in an attempt to be able to have fantastic C++ interoperability. On the one on hand, that seems great, since being able to have your C++ code work with your D code is great, but on the other, it seems like it's going to make it so that D is contaminated by a lot of extra C++ muck just to be able to interoperate. At some point, we either need to decide that we're just not going to interoperate with C++ in some manner and lose out on some capability, or we're going to need to fully interoperate with C++ and pretty much put a C++ compiler in the D compiler, and I'd prefer that we didn't go that far. In this particular case, doesn't this really just come down to mangling? It's undefined behavior in D to mutate a const object (even if it was constructed as mutable), so we can't just slap D const on C++ types and have that work, since the C++ code could legally mutate the object. But since C++ considers it defined behavior to mutate a const object by casting away const (or at least it does in all but some very specific cases), can we just get away with the D code treating D const as mutable? If so, then it's purely a matter of mangling. And for that, we could do something like add @cppconst or cppconst that the compiler recognizes and which is only valid in extern(C++) declarations. It would unfortunately have to be more than a simple attribute, because it would have to apply to parts of a parameter's type like C++'s const does (instead of the whole type at once), but if it were implemented such that it was just something that went on the extern(C++) function parameter types (or return types) for mangling purposes, and the D code just treated it as mutable, then it would at least restrict the muck to extern(C++). Now, that does have the downside that the D code has no protection against mutating a C++ object that the C++ code marks as const (whereas while the C++ code can mutate, you at least have to cast away const or use the mutable keyword first), but that seems a whale of a lot better to me than trying to add non-transitive const to D. - Jonathan M Davis
Re: Head Const
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. Maybe you can get away with adding "mutable", which is needed for intrusive ref counting as well: C++: Type * const ptr; struct ConstType { mutable int rc; } D: const mutable(Type)* ptr; struct ConstType { mutable(int) rc; }
Re: Head Const
I think the point about name mangling is very true. That's the most important thing, being able to call all of the C++ functions. I personally love that const and immutable are transitive in D. I get annoyed in other languages when I have const objects containing mutable objects, and no real protection from the compiler from stopping the objects in the inside from being mutated. The way I see it, C++ const is a weaker form of const which doesn't complain about mutations enough. I think if we could avoid putting head const in the language, we'd be better off. If you ever really, really need to mutate something inside a const type, you can cast away const. It will rightly look ugly, and the compiler can complain that it's not @safe.
Re: Head Const
On Tuesday, 16 February 2016 at 07:46:11 UTC, w0rp wrote: I think if we could avoid putting head const in the language, we'd be better off. If you ever really, really need to mutate something inside a const type, you can cast away const. It will rightly look ugly, and the compiler can complain that it's not @safe. Actually, in D, it's undefined behavior to cast away const and mutate. D's const is supposed to guarantee that the data cannot be mutated via a const reference to the data. Given that very few optimizations can be made based solely on const (and I'm not sure that the compiler currently does any of them), it will almost certainly work as long as the underlying data is mutable rather than immutable, but it _is_ undefined behavior, and you're violating the guarantees that the type system provides when you go around it like that. - Jonathan M Davis
Re: Head Const
On Tuesday, 16 February 2016 at 07:20:00 UTC, Ola Fosheim Grøstad wrote: On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. Maybe you can get away with adding "mutable", which is needed for intrusive ref counting as well: C++: Type * const ptr; struct ConstType { mutable int rc; } D: const mutable(Type)* ptr; struct ConstType { mutable(int) rc; } For that to work in D, we'd lose out on the guarantee that a const object can't be mutated via a const reference to that data - though with the current state of things, const does become limiting enough that you can't use it under a number of circumstances that you might like to (e.g. ref-counting), and some folks mistakenly think that it's defined behavior to cast away const and mutate and do it in their code. So, such a change might be worth it, but it's not something that should be considered lightly. If we did go that route though, having mutable or @mutable would allow for it to be much more controlled than simply casting away const and mutating (e.g. it could guarantee that no immutable objects were involved, since an object with an @mutable member couldn't be immutable, since that would violate immutable). But even if we went that route, that's still a whole other ballgame from non-transitive const. There are some major implications for const if we making it legal to cast it away and mutate as long as the underlying data is mutable rather than immutable, but from a language complexity standpoint, it's not particularly problematic, whereas non-transitive const is. - Jonathan M Davis
Re: Head Const
On Tuesday, 16 February 2016 at 08:04:32 UTC, Jonathan M Davis wrote: For that to work in D, we'd lose out on the guarantee that a const object can't be mutated via a const reference to that data - though with the current state of things, const does become limiting enough that you can't use it under a number of circumstances that you might like to (e.g. ref-counting), and some folks mistakenly think that it's defined behavior to cast away const and mutate and do it in their code. Yes, casting away transitive const should be impossible. The real issue is how to deal with calling overloaded C++ functions. Like, some C++ functions implements transitive const, others do head const combined with mutable. So it is insufficient to look at the C++ sigature... Which means D ought to have some stronger way to specifing constness for C++ signatures than C++ provides. But even if we went that route, that's still a whole other ballgame from non-transitive const. There are some major implications for const if we making it legal to cast it away and mutate as long as the underlying data is mutable rather than immutable, but from a language complexity standpoint, it's not particularly problematic, whereas non-transitive const is. Yes, please don't allow casting from const to mutable, D should aim for stronger typing and better optimization.
Re: Head Const
On 2/15/2016 10:04 PM, Jonathan M Davis wrote: Straight up head-const is utterly useless IMHO. That's what Java has. C++ has something far more complicated where any individual piece (or pieces) of a pointer type can be declared const without necessarily making the whole thing const. So, _that_ is what we'd need to interact with C++, and it's a downright ugly feature IMHO. That's right. 'headconst' cannot be just a storage class, it has to be a type constructor. You're right that mangling is the big issue with C++ interoperability in regards to const.
Re: Head Const
On 16/02/2016 12:04 PM, Walter Bright wrote: It's currently difficult to interface with C++, and always will be, but smoothing out what we can can be a big opportunity for D. I'm starting to think we should give up on implementing C++ support in the language and move it to the library. eg mixin(cppFunctionBinding("unsigned long long NameSpace::myFunc(char * const && x, long double y)"); expands to some combination of pragma(mangle) and extern(C++) With the limitation that only declarations can be parsed, it's not that bad to implement a ctfe C++ parser, and we can stop the creep of C++ features and hack into D. As a bonus, it could generate wrappers when we really can't match the semantics well enough.
Re: Head Const
On Tuesday, 16 February 2016 at 09:04:48 UTC, Daniel Murphy wrote: I'm starting to think we should give up on implementing C++ support in the language and move it to the library. eg mixin(cppFunctionBinding("unsigned long long NameSpace::myFunc(char * const && x, long double y)"); expands to some combination of pragma(mangle) and extern(C++) With the limitation that only declarations can be parsed, it's not that bad to implement a ctfe C++ parser, and we can stop the creep of C++ features and hack into D. As a bonus, it could generate wrappers when we really can't match the semantics well enough. I agree with the principle, but not as a library function, because: 1. you want virtual functions to work out ok 2. making D more reliant on macroish string processing is not good You would need something along the lines of: 1. «extern "C++"» the essence of the class definition in plain C++ syntax 2. add to this syntax a translation for each parameter what it means in D. E.g. extern "C++" { class X { mutable int rc; virtual func1(const A* ptr); @reinterpret(ptr, head_const_ptr!A) virtual func2(const A* ptr); @reinterpret(ptr, const A*) virtual func3(A* ptr); virtual func4(const A* ptr); @reinterpret(ptr, const_rc!A*) }; }
Re: Head Const
Am 16.02.2016 um 08:20 schrieb Ola Fosheim Grøstad: On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. Maybe you can get away with adding "mutable", which is needed for intrusive ref counting as well: C++: Type * const ptr; struct ConstType { mutable int rc; } D: const mutable(Type)* ptr; struct ConstType { mutable(int) rc; } As a bonus, this would also provide a natural syntax to define tail-const class references: mutable const(C) something;
Re: Head Const
On Tuesday, 16 February 2016 at 08:03:31 UTC, Jonathan M Davis wrote: ... Given that very few optimizations can be made based solely on const (and I'm not sure that the compiler currently does any of them) ... I think that the fact that we have shared can help a lot in the area of optimization. Taking a non-shared const parameter should be as good as taking an immutable one (if the function pure-like - only does computation based on the parameter and doesn't escape it). Or am I missing something?
Re: Head Const
On Tuesday, 16 February 2016 at 09:45:34 UTC, ZombineDev wrote: On Tuesday, 16 February 2016 at 08:03:31 UTC, Jonathan M Davis wrote: ... Given that very few optimizations can be made based solely on const (and I'm not sure that the compiler currently does any of them) ... I think that the fact that we have shared can help a lot in the area of optimization. Taking a non-shared const parameter should be as good as taking an immutable one (if the function pure-like - only does computation based on the parameter and doesn't escape it). Or am I missing something? The fact that variables are thread-local by default in D definitely helps, but it's still far from sufficient. If you have auto result = func(foo); where foo is const, for the compiler to know that foo was not mutated by the call to func, it has to know that there's no way that func could have accessed a mutable reference to the object that foo refers to. The fact that we have shared means that it doesn't have to worry about other, arbitrary code mutating foo while func is called, but it still has to be able to determine that neither func no anything that it accesses is able to mutate foo. If func is pure, that gets us a good chunk of the way, because then the only way that func can access a mutable reference to foo is via its arguments. If it's just foo, then - unless I'm missing something - that should be enough, since foo would not be able to access a mutable reference to itself except via a non-pure function, which wouldn't be callable from func if it's pure. But if func is a member function, or if it took addition, mutable arguments, e.g. auto result = func(foo, bar); then the compiler has to know that func can't access a mutable foo via the other arguments - and if they're not const, that's hard to do. In general, I think that the compiler pretty much has to be able to see where foo was created and that no mutable references _can_ exist that func would have access to. e.g. const foo = new Foo(42); auto result = func(foo, bar); or auto mutableFoo = new Foo(42); const foo = mutableFoo; auto result = func(foo, bar); would be enough. But without a lot of code flow analysis, it doesn't take much for it to be too much for the compiler to figure out, and unless it has access to the bodies of the functions that are being called and does deep inspection of what they do and call, simply calling a function (especially a non-pure one) very quickly makes it so that the compiler doesn't know enough to guarantee that func can't possibly access foo via a mutable reference to the same data. e.g. in auto mutableFoo = new Foo(42); auto result1 = otherFunc(mutableFoo, bar); const foo = mutableFoo; auto result2 = func(foo, bar); otherFunc might have given bar access to mutableFoo which it then accessed inside of func. So, yes, the combination of non-shared, pure, and const does allow for some optimizations, but it really doesn't take much to make it so that the compiler can't guarantee that const object hasn't been changed by a function call without it doing a lot of deep analysis that compilers just don't normally do. - Jonathan M Davis
Re: Head Const
On Tuesday, 16 February 2016 at 09:39:31 UTC, Sönke Ludwig wrote: Am 16.02.2016 um 08:20 schrieb Ola Fosheim Grøstad: On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. Maybe you can get away with adding "mutable", which is needed for intrusive ref counting as well: C++: Type * const ptr; struct ConstType { mutable int rc; } D: const mutable(Type)* ptr; struct ConstType { mutable(int) rc; } As a bonus, this would also provide a natural syntax to define tail-const class references: mutable const(C) something; Another bonus to introducing the mutable keyword is the option to make everything immutable by default (in a future version of D) and allow the users to have mutable objects only if they use the mutable keyword.
Re: Head Const
On Tuesday, 16 February 2016 at 10:06:12 UTC, ZombineDev wrote: Another bonus to introducing the mutable keyword is the option to make everything immutable by default (in a future version of D) and allow the users to have mutable objects only if they use the mutable keyword. While some folks do bring that up from time to time, I think that it's pretty clear that that would be so restrictive that it would risk killing D. As it is, many programmers avoid const altogether, because it's too restrictive. Heck, ranges are designed in such a way that they require mutation to work, and they're everywhere. immutable has its uses to be sure, but I don't see how it's anything but a pipe dream to expect any version of D to be immutable by default. For most programmers, it would be way too annoying and way too verbose, because they'd be forced to slap mutable on most everything. Regardless, there isn't much point in planning for a future version of D. We don't know what we're going to want to do at that point, and if we're actually willing to break backwards compatibility in a serious way, what D2 looks like doesn't really matter much for D3. And we don't even know whether there will ever be a D3. What matters to us now is what we do with D2 for making it a good language now and not what we may or may not do with a future version of the language. Planning for D3 now would be like planning for D when working on finishing up C++98. - Jonathan M Davis
Re: Head Const
On Tuesday, 16 February 2016 at 01:04:44 UTC, Walter Bright wrote: 2. supports single assignment style of programming, even if the data is otherwise mutable Like 'final'? We did get rid of that... Maybe we should resurrect it. I'm trying to say it politely. D2 const story is more complicated than its competitors. Both D1 "final" and C++ const always felt more useful and practical to me that the whole D2 immutable/const/inout thing. The current scheme seems to have marginal value in practice, lots of complexity, and is harder to use well (Unqual, inout) etc. Constructors can break it. I don't know why we should be that happy about our constness, maybe someone can explain.
Re: Head Const
On 16.02.2016 01:14, H. S. Teoh via Digitalmars-d wrote: Just out of curiosity, since we already have Rebindable in Phobos and HeadConst is being proposed, what are the disadvantages / shortcomings of a library solution that would justify adding yet another feature to the language? struct S{ int[] a; void foo()headconst{ a[0]=1; // ok // arr.length=2 // error } } void main(){ headconst(S) s([0,2,3]); s.foo(); assert(s.a==[1,2,3]); } How to do this in the library?
Re: Head Const
On Tuesday, 16 February 2016 at 10:09:19 UTC, Jonathan M Davis wrote: On Tuesday, 16 February 2016 at 09:45:34 UTC, ZombineDev wrote: On Tuesday, 16 February 2016 at 08:03:31 UTC, Jonathan M Davis wrote: ... Given that very few optimizations can be made based solely on const (and I'm not sure that the compiler currently does any of them) ... I think that the fact that we have shared can help a lot in the area of optimization. Taking a non-shared const parameter should be as good as taking an immutable one (if the function pure-like - only does computation based on the parameter and doesn't escape it). Or am I missing something? The fact that variables are thread-local by default in D definitely helps, but it's still far from sufficient. If you have auto result = func(foo); where foo is const, for the compiler to know that foo was not mutated by the call to func, it has to know that there's no way that func could have accessed a mutable reference to the object that foo refers to. The fact that we have shared means that it doesn't have to worry about other, arbitrary code mutating foo while func is called, but it still has to be able to determine that neither func no anything that it accesses is able to mutate foo. If func is pure, that gets us a good chunk of the way, because then the only way that func can access a mutable reference to foo is via its arguments. If it's just foo, then - unless I'm missing something - that should be enough, since foo would not be able to access a mutable reference to itself except via a non-pure function, which wouldn't be callable from func if it's pure. But if func is a member function, or if it took addition, mutable arguments, e.g. auto result = func(foo, bar); then the compiler has to know that func can't access a mutable foo via the other arguments - and if they're not const, that's hard to do. In general, I think that the compiler pretty much has to be able to see where foo was created and that no mutable references _can_ exist that func would have access to. e.g. const foo = new Foo(42); auto result = func(foo, bar); or auto mutableFoo = new Foo(42); const foo = mutableFoo; auto result = func(foo, bar); would be enough. But without a lot of code flow analysis, it doesn't take much for it to be too much for the compiler to figure out, and unless it has access to the bodies of the functions that are being called and does deep inspection of what they do and call, simply calling a function (especially a non-pure one) very quickly makes it so that the compiler doesn't know enough to guarantee that func can't possibly access foo via a mutable reference to the same data. e.g. in auto mutableFoo = new Foo(42); auto result1 = otherFunc(mutableFoo, bar); const foo = mutableFoo; auto result2 = func(foo, bar); otherFunc might have given bar access to mutableFoo which it then accessed inside of func. So, yes, the combination of non-shared, pure, and const does allow for some optimizations, but it really doesn't take much to make it so that the compiler can't guarantee that const object hasn't been changed by a function call without it doing a lot of deep analysis that compilers just don't normally do. - Jonathan M Davis I agree with everything you said. What I meant was that **in theory** const + thread-local can give similar guarantees as immutable (if the compiler can perform the necessary analysis). That means it's more of an implementation problem than a langauge problem. Given that separate compilation and .di header-only libraries are far more rare in D that in C++ (D has faster compilation and templates with attribute inference provide even more incentive), whole program optimization seems more easily doable in D. After all the Rust compiler has proven that immutable borrowing (of a mutable object) is not hard to enforce, so I don't see a hard obstacle for this to be implemented in D. At least not an obstacle comming from the language design.
Re: Head Const
On Tuesday, 16 February 2016 at 10:17:05 UTC, Jonathan M Davis wrote: version of D. We don't know what we're going to want to do at that point, and if we're actually willing to break backwards compatibility in a serious way, what D2 looks like doesn't really matter much for D3. And we don't even know whether there will ever be a D3. What matters to us now is what we do with D2 for making it a good language now and not what we may or may not do with a future version of the language. Planning for D3 now would be like planning for D when working on finishing up C++98. This is true, in a way, but also a bit too pessimistic. Here is a possible strategy: 1. add semantics that are desirable to D2 2. express both D2 and D3 in a single intermediate representation 3. work on a new D3 syntax that does not support deprecated features 4. have two parsers, one for D2 and one for D3 Let compiler vendors decide whether they want to support D2 or not.
Re: Head Const
On 16.02.2016 11:17, Jonathan M Davis wrote: On Tuesday, 16 February 2016 at 10:06:12 UTC, ZombineDev wrote: Another bonus to introducing the mutable keyword is the option to make everything immutable by default (in a future version of D) and allow the users to have mutable objects only if they use the mutable keyword. While some folks do bring that up from time to time, I think that it's pretty clear that that would be so restrictive that it would risk killing D. As it is, many programmers avoid const altogether, because it's too restrictive. Immutable by default would not be what we have now (transitive immutable).
Re: Head Const
On 02/16/2016 12:48 AM, Walter Bright wrote: > rears its head again :-) > > Head Const is what C++ has for const, i.e. it is not transitive, applies > to one level only. D has transitive const. > > What head const will do for us: > > 1. make it easy to interface to C++ code that uses const, as currently > it is not very practical to do so, you have to resort to pragma(mangle) > > 2. supports single assignment style of programming, even if the data is > otherwise mutable > > The downside is, of course, language complexity. Moderately skeptical here. Adding features to the language for the sake of C++ compatibility is good way to ensure C++ compatibility will be its only domain of triumph. I'd expect more justification for users who don't care about C++ but want language which doesn't collapse under own weight.
Re: Head Const
On Tuesday, 16 February 2016 at 11:02:38 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 16 February 2016 at 10:17:05 UTC, Jonathan M Davis wrote: version of D. We don't know what we're going to want to do at that point, and if we're actually willing to break backwards compatibility in a serious way, what D2 looks like doesn't really matter much for D3. And we don't even know whether there will ever be a D3. What matters to us now is what we do with D2 for making it a good language now and not what we may or may not do with a future version of the language. Planning for D3 now would be like planning for D when working on finishing up C++98. This is true, in a way, but also a bit too pessimistic. Here is a possible strategy: 1. add semantics that are desirable to D2 2. express both D2 and D3 in a single intermediate representation 3. work on a new D3 syntax that does not support deprecated features 4. have two parsers, one for D2 and one for D3 Let compiler vendors decide whether they want to support D2 or not. A small comment on the way you express most of your opinions: You have many ideas on how and why D should change, however all of your proposals are so vague that no one can figure out any concrete action items in them. Furthermore you don't seem to be familiar with implementation details and even don't seem to have any experience in writting large programs in D, because you never talk about concrete problems and never contribute DIPs or PRs (which other people serious about using D do). It seems that you're not interested in using D for anything at all, which is very strange considering how much time you spend on the forums. Instead of trying critisize other people's concrete plans/ideas by saying that instead they should be focusing on something very abstract that only you understand, I think it would be a lot more beneficial for the D community and your agenda, if you turn your words into actions and show **by example** that your design/implementation has **objective, measurable** benefits. Otherwise no one will find practical real-world benefits from your theoretical ideas and we would just waste time arguing with each other, while something practical could be accomplished in the same time. Be the change you want to see in the world. Back to your concrete comment: It sounds nice in theory, however there are too many refactorings in the AST and semantic analysis done for each release, which prevents creating a stable intermediate representation. The only stable IRs are the DMD backend IR, the GCC IR and the LLVM IR. By the way those sementics (maybe) are already representable in the current AST abstractions in the frontend. What benefits would a dedicated frontend IR bring? Hint: I have already watched the LLVM talk about the Swift IR and I'm familiar with the reasons for Rust introducing MIR. I just don't see the value such a large initiative would bring to D. The best way to defend your point would be look at the DMD source code and propose specific API changes/additions which would bring objective benefits. Otherwise you sound like someone telling a car manufacturer that they should be focusing on flying cars or teleportation, because clearly that's the future :D
Re: Head Const
On Tuesday, 16 February 2016 at 12:03:48 UTC, ZombineDev wrote: familiar with implementation details and even don't seem to have any experience in writting large programs in D, because Of course I don't. My goal for D is that it becomes a solid base language which it makes sense to build upon. My D programming is currently only language related experiments on a rather low level (syntax and basic functionality), I have used D for other things in the past, but currently it does not stack up against Python or C++ for production in my case. It would be nice if it did, of course. Otherwise no one will find practical real-world benefits from your theoretical ideas and we would just waste time arguing with each other, while something practical could be accomplished in the same time. The foundation has to be solid. Clang went against common wisdom when compiling directly to a low level IR without. Common wisdom is to compile to a high level IR first. In the case of C++ it has worked out ok for clang. That does not mean that there are no benefits to using a high level IR. GCC has a higher level IR at the top level, and I suspect they have benefitted from that on some optimizations. would bring objective benefits. Otherwise you sound like someone telling a car manufacturer that they should be focusing on flying cars or teleportation, because clearly that's the future :D No, it is more like discussing the chassis rather than the polish. Car manufacturers can design many models over the same chassis, if they put extra effort into it.
Re: Head Const
On Tuesday, 16 February 2016 at 12:15:03 UTC, Ola Fosheim Grøstad wrote: Common wisdom is to compile to a high level IR first. In the case of C++ it has worked out ok for clang. That does not mean that there are no benefits to using a high level IR. Let me expand on this: 1. it makes the compiler easier to maintain and debug as the compiler becomes more modular with clear separation 2. it makes it possible to verify @safe (hopefully) 3. it makes it easier to use multiple backends 4. it can make it easier to exploit multithreading 5. you can have separate compilation that retains more type information for whole program linkage or optimization 6. It makes it much easier to get people to work on the compiler as you only need to understand the IR when implementing features, i.e. static analysis.
Re: Head Const
On Tuesday, 16 February 2016 at 10:32:45 UTC, Timon Gehr wrote: struct S{ int[] a; void foo()headconst{ a[0]=1; // ok // arr.length=2 // error } } void main(){ headconst(S) s([0,2,3]); s.foo(); assert(s.a==[1,2,3]); } How to do this in the library? Maybe make foo a free function: struct S{ int[] a; } void foo(HConst!S self){ self.a[0]=1; // ok self.arr.length=2 // no effect } Haven't tried, but maybe HConst can allow access to fields of S by value, not ref. That could prevent mutation like C++ const, but it wouldn't disallow mutation of the temporary copy. Not sure if that's good enough. I suppose the compiler could warn about unused temporary mutation.
Re: Head Const
On Tuesday, 16 February 2016 at 11:27:56 UTC, Dicebot wrote: On 02/16/2016 12:48 AM, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. Moderately skeptical here. Adding features to the language for the sake of C++ compatibility is good way to ensure C++ compatibility will be its only domain of triumph. I'd expect more justification for users who don't care about C++ but want language which doesn't collapse under own weight. `@mutable` OTOH would be a useful for both C++, reference counting, caching, lazy initialization... But we need to find a way to keep most of the existing guarantees, especially concerning shareability.
Re: Head Const
On Tuesday, 16 February 2016 at 10:17:05 UTC, Jonathan M Davis wrote: On Tuesday, 16 February 2016 at 10:06:12 UTC, ZombineDev wrote: Another bonus to introducing the mutable keyword is the option to make everything immutable by default (in a future version of D) and allow the users to have mutable objects only if they use the mutable keyword. While some folks do bring that up from time to time, I think that it's pretty clear that that would be so restrictive that it would risk killing D. As it is, many programmers avoid const altogether, because it's too restrictive. Not if immutable by default is opt-in - read below (my second paragraph). Heck, ranges are designed in such a way that they require mutation to work, and they're everywhere. Ranges are not a hard problem, they're more of a too much good code written with them to be worth the breakage. Since ranges are mostly meant to be cheap value types, we can just make popFront return a new range, just like slicing an array slice returns a new slice. However for this to work we would need to rewrite a ton of std.range, std.algorithm, etc. so it probably won't provide a good enough cost/benefit ratio. I'm working on a different proposal to generalize ranges to support push data flow model, in addition to the current pull-only, which would allow them to be used for processing async data (e.g. comming from I/O or another thread performing expensive computation for each element) in a **non-blocking** way. Processing async data with the current range primitives implies polling/blocking on popFront or front, which is a performance killer and diminishes the value of composition. I hope my proposal would provide enough value to be included in phobos as an alternative to the current InputRange primitives. If it proves to be good enough, maybe it would be worth adding new versions of most of the current range algorithms/transformations that leverage the new prmitives. I haven't yet figured out the best way to bring these API ideas from Clojure and C++ to D (see http://forum.dlang.org/thread/xfjoktyjtoojhszrl...@forum.dlang.org for more info on me exploring one possibility of using higher-order functions), because there are plenty of ways to do this which I want to explore, but I hope to have something concrete in a couple of months. immutable has its uses to be sure, but I don't see how it's anything but a pipe dream to expect any version of D to be immutable by default. For most programmers, it would be way too annoying and way too verbose, because they'd be forced to slap mutable on most everything. Immutable by default has two sides: 1) The programmer writing the code 2) The user of the code The change to immutable by default should only affect 1). 2) should be the same as if currently you make something immutable. This means **no code breakage**. A good way forward, IMO, is to be able to indicate at the beginning of the module your general preferences e.g. immutable, @nogc, pure, non-virtual by default. This should affect **only** the current declaration scope, just as if you have put `nothrow:` at the beginning. The only thing that's needed is a way to revert this. The proposed solution is `pure(false)`, `final(false)`, etc., which Andrei already approved. Regardless, there isn't much point in planning for a future version of D. We don't know what we're going to want to do at that point, and if we're actually willing to break backwards compatibility in a serious way, what D2 looks like doesn't really matter much for D3. And we don't even know whether there will ever be a D3. What matters to us now is what we do with D2 for making it a good language now and not what we may or may not do with a future version of the language. Planning for D3 now would be like planning for D when working on finishing up C++98. - Jonathan M Davis I agree. I think we should focus on features that have strong benefits and don't break backwards compatibility. There are plenty of stuff to explore in this area. From what I've read, the D1/D2 and Phobos/Tango devisions was hurtful for the community. We should instead focus on unitying and improving what we already have in order to bring real benefits to our community.
Re: Head Const
On 02/16/2016 02:49 PM, Marc Schütz wrote: > `@mutable` OTOH would be a useful for both C++, reference counting, > caching, lazy initialization... But we need to find a way to keep most > of the existing guarantees, especially concerning shareability. In my opinion @mutable would be a disaster of much higher destructive impact than head const. I am very opposed to it no matter how it is designed. Once you start considering it, you are better at simply throwing away existing const system and starting it all from scratch with D3. Logical const is harmful as it doesn't give and serious guarantees but gives developer a false sense of confidence.
Re: Head Const
On Tuesday, 16 February 2016 at 13:35:56 UTC, Dicebot wrote: matter how it is designed. Once you start considering it, you are better at simply throwing away existing const system and starting it all from scratch with D3. Logical const is harmful as it doesn't give and serious guarantees but gives developer a false sense of confidence. Why is that? Doesn't non-shared const promise that the protected values aren't modified, until the const reference is no longer alive? Meaning: does the language allow non-const aliasing with const in function calls? // prototype f(const A *, A*); f(ptr,ptr); Is this disallowed? Or is it legal D? (not talking about the compiler, but the language spec)
Re: Head Const
On 16.02.2016 14:41, Ola Fosheim Grøstad wrote: On Tuesday, 16 February 2016 at 13:35:56 UTC, Dicebot wrote: matter how it is designed. Once you start considering it, you are better at simply throwing away existing const system and starting it all from scratch with D3. Logical const is harmful as it doesn't give and serious guarantees but gives developer a false sense of confidence. Why is that? Doesn't non-shared const promise that the protected values aren't modified, until the const reference is no longer alive? Meaning: does the language allow non-const aliasing with const in function calls? // prototype f(const A *, A*); f(ptr,ptr); Is this disallowed? Or is it legal D? (not talking about the compiler, but the language spec) Legal.
Re: Head Const
On Tuesday, 16 February 2016 at 15:10:40 UTC, Timon Gehr wrote: Legal. Ouch... :-/
Re: Head Const
On Tuesday, 16 February 2016 at 15:12:00 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 16 February 2016 at 15:10:40 UTC, Timon Gehr wrote: Legal. Ouch... :-/ const guarantees that the data will not be mutated via that reference to the data. It guarantees nothing about other references to that data. For that, you need immutable. But as long as const guarantees that the data can't be mutated via that reference, then there's plenty of code where the programmer can look at it and know for sure that that data isn't being mutated, because they know that the code in question can't have access to the same data via a mutable reference (the compiler can see that too, but in a much more limited fashion than a programmer is going to be able to know). If casting away const and mutating is defined behavior, then you lose that guarantee, and instead of knowing that some code doesn't have access to a mutable reference to that data being enough, you have to actually know what the code does to know that it's not casting away const and mutating. Having @mutable would be better than allowing casting away const and mutating to be defined behavior (at least when the underlying data is mutable rather than immutable), because it would at least restrict the data that can be mutated, and it would guarantee that you don't accidentally mutate an immutable variable (since it would make no sense to have a type with an @mutable member be immutable, whereas if you're casting away const, getting that right would be completely up to the programmer). But having @mutable would still mean that you can't rely on a const variable not being mutated when you pass it to a function that you know doesn't have access to a mutable reference to the same data. If you knew that no @mutable was involved, it would be the same as now, but you'd have to know for sure that @mutable was involved, and even if it isn't now, it could be later after refactoring. So, having @mutable would be far better than having it be defined behavior to cast away const and mutate (assuming that the underlying data is mutable rather than immutable), but you're still losing out on guarantees that we have now. After all, most code isn't going to do the equivalent of // prototype f(const A *, A*); f(ptr,ptr); Ultimately, I think that the question is whether the guarantees that we currently get with const are worth more or whether the idioms that don't work with const right now but would work with @mutable are worth more. But regardless, the guarantees provided by const obviously pale in comparison to the ones that immutable provides. - Jonathan M Davis
Re: Head Const
On Tuesday, 16 February 2016 at 06:04:42 UTC, Jonathan M Davis wrote: On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. In this particular case, doesn't this really just come down to mangling? It's undefined behavior in D to mutate a const object (even if it was constructed as mutable), so we can't just slap D const on C++ types and have that work, since the C++ code could legally mutate the object. But since C++ considers it defined behavior to mutate a const object by casting away const (or at least it does in all but some very specific cases), can we just get away with the D code treating D const as mutable? If so, then it's purely a matter of mangling. And for that, we could do something like add @cppconst or cppconst that the compiler recognizes and which is only valid in extern(C++) declarations. It would unfortunately have to be more than a simple attribute, because it would have to apply to parts of a parameter's type like C++'s const does (instead of the whole type at once), but if it were implemented such that it was just something that went on the extern(C++) function parameter types (or return types) for mangling purposes, and the D code just treated it as mutable, then it would at least restrict the muck to extern(C++). Actually, one could argue that the fact that we support const at all with other languages - even with C - is broken in that C/C++ code can cast away our const and mutate the variable we passed in, breaking our type system. We rely on the programmer to know that the C/C++ code isn't going to do that, and most of the time, you really have no clue. You just assume that it won't - and it probably won't, but you don't know for sure. So, most of the time, it will work just fine, but const with extern(C) and extern(C++) really does not provide the guarantees that const is supposed to provide. Of course, there's also the issue that with extern(C) and extern(C++), everything is really __gshared, and yet we typically treat it like it's thread-local just like the D code is. Arguably, all parameters and return types with extern(C) and extern(C++) should be automatically treated as shared, and not doing so is risking a world of pain if the C/C++ code misbehaves, but having them be treated as shared would be really annoying to deal with, and most C/C++ code that's being called probably isn't doing anything with threads that could violate D's guarantees about thread-local variables (just like most C/C++ code presumably isn't casting away const and mutating when we pass it something from D) - but there's no guarantee that it isn't. So, as far as compiler guarantees and the type system go, extern(C) and extern(C++) are a bit of a disaster, much as they generally work in practice. But I don't know what we can really do about that other than trying to educate people about the risks and possibly have the compiler avoid certain types of optimizations around extern(C) and extern(C++) code (which it may already do). - Jonathan M Davis
Re: Head Const
On Tuesday, 16 February 2016 at 15:33:03 UTC, Jonathan M Davis wrote: const guarantees that the data will not be mutated via that reference to the data. It guarantees nothing about other references to that data. For that, you need immutable. Yes, it's as weak as C++ const in most cases from an optimization POV. In C (and some C++ compilers) you can say that there is no aliasing by using "restrict", so I guess what I want is something like "const restrict". https://en.wikipedia.org/wiki/Restrict But as long as const guarantees that the data can't be mutated via that reference, then there's plenty of code where the programmer can look at it and know for sure that that data isn't being mutated, because they know that the code in When you are calling a function that is no-global-access "pure", yes. If casting away const and mutating is defined behavior, then you lose that guarantee, and instead of knowing that some code doesn't have access to a mutable reference to that data being enough, you have to actually know what the code does to know that it's not casting away const and mutating. Yes, I agree that casting away const as is possible in C++ is bad. The only reasonable setting for doing so is when you call legacy functions that are not defined to be const, but implemented as such. Having @mutable would be better than allowing casting away const and mutating to be defined behavior (at least when the underlying data is mutable rather than immutable), because it would at least restrict the data that can be mutated, and it would guarantee that you don't accidentally mutate an immutable variable (since it would make no sense to have a type with an @mutable member be immutable, whereas if you're casting away const, getting that right would be completely up to the programmer). Maybe I misunderstand, but I am likely to want an immutable object with a mutable field, e.g. ref-counted caching. You use a CAS instruction on the mutable ref-counter and share the data. Then you can flush the cache and release all objects that have a refcount of only 1 (the cache index pointer). But having @mutable would still mean that you can't rely on a const variable not being mutated when you pass it to a function that you know doesn't have access to a mutable reference to the same data. If you knew that no @mutable was involved, it would be the same as now, but you'd have to know for sure that @mutable was involved, and even if it isn't now, it could be later after refactoring. Well, yes, this is where you have to be very clear about what kind of semantics a struct is meant to have. I don't feel this is very clear in the current language. To me the value of immutable is that fields that are immutable will remain immutable forever. Like for a key that is inserted into a hash table. You know that it cannot change so you never need to recompute the hash value. Then I'd like to see a "limited time immutable", like the "restrict const" suggested above. But apparently there are use-cases where you'd want to say "whole object is immutable" and other use-cases where you want to say "all non-forced-mutable fields are immutable". So, having @mutable would be far better than having it be defined behavior to cast away const and mutate (assuming that the underlying data is mutable rather than immutable), but you're still losing out on guarantees that we have now. Yes. But I think you could distinguish between what you specify and what you require. So you could specify "I only want wholly immutable things" in a function signature, and specify "this object is mostly immutable except for these mutable exceptions" when defining it. So I think you can have both?
Re: Head Const
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. plz plz plz! +1 Bit
Re: Head Const
On Tuesday, 16 February 2016 at 16:01:26 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 16 February 2016 at 15:33:03 UTC, Jonathan M Davis wrote: So, having @mutable would be far better than having it be defined behavior to cast away const and mutate (assuming that the underlying data is mutable rather than immutable), but you're still losing out on guarantees that we have now. Yes. But I think you could distinguish between what you specify and what you require. So you could specify "I only want wholly immutable things" in a function signature, and specify "this object is mostly immutable except for these mutable exceptions" when defining it. So I think you can have both? const and immutable are transitive. This is particularly critical with immutable, which is implicitly shared. Having portions of an immutable object be mutable wouldn't play nicely at all with how immutable works and would complicate things considerably. Andrei recently posted about possibly having a reference count be in the memory next to an immutable object but not having it in the object itself, and that might work: http://forum.dlang.org/post/n9larg$23c9$1...@digitalmars.com But I really don't think that getting rid of transivity is going to fly for immutable (e.g. it's not like you can share part of object across threads and have part of it be thread-local), and I'm opposed to introducing non-transitive const like Walter brought up with this thread. It's just not worth the complexity. - Jonathan M Davis
Re: Head Const
On Tuesday, 16 February 2016 at 16:27:54 UTC, Jonathan M Davis wrote: considerably. Andrei recently posted about possibly having a reference count be in the memory next to an immutable object but not having it in the object itself, and that might work: I don't think I understand the semantic difference. Transitive immutability is mostly useless. You usually want at least to be able to have a back pointer to a mutable index so that you can move the object to another index. This is also not something you can prevent. I can simply have a global set of indices and refer to that global set through an index ID in the immutable object. That is equivalent to breaking immutable transitivity. So... you cannot have transitive immutability if you allow global data structures. Then you would have to require "pure" functions everywhere. I also don't see how a reinterpret cast to a negative offset can be a good solution. If you need that, then you have a strong sign that something important is missing in the type system. Where would this be allowed? Not in "pure" functions, for sure? But I really don't think that getting rid of transivity is going to fly for immutable (e.g. it's not like you can share part of object across threads and have part of it be thread-local) It shouldn't pose a problem for a system level language to hand memory access through a thread local pointer to another thread if need be, but not sure if I follow the argument. , and I'm opposed to introducing non-transitive const like Walter brought up with this thread. It's just not worth the complexity. But you do at the very least need a mechanism for triggering overloads on "const A&", "A&&" etc. C++ libraries tend to be heavily overloaded...
Re: Head Const
On Tuesday, 16 February 2016 at 06:04:42 UTC, Jonathan M Davis wrote: On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. Previously, you stated that we weren't going to do things like support interacting with C++ templates (aside from specific instantiations which weren't typed as templates), because it would mean putting a C++ compiler into D, which you didn't want to do. But increasingly, it seems like you're heading in the direction of doing that in an attempt to be able to have fantastic C++ interoperability. On the one on hand, that seems great, since being able to have your C++ code work with your D code is great, but on the other, it seems like it's going to make it so that D is contaminated by a lot of extra C++ muck just to be able to interoperate. At some point, we either need to decide that we're just not going to interoperate with C++ in some manner and lose out on some capability, or we're going to need to fully interoperate with C++ and pretty much put a C++ compiler in the D compiler, and I'd prefer that we didn't go that far. - Jonathan M Davis Hahaha. Well, I think it is already happening. Like the reincarnation of C to C++ story.
Re: Head Const
On Tuesday, 16 February 2016 at 18:06:05 UTC, karabuta wrote: Hahaha. Well, I think it is already happening. Like the reincarnation of C to C++ story. The focus on interfacing D with C++ lately has been very disconcerting, especially considering features from TDPL are still unfinished nearly 6 years later.
Re: Head Const
On Tuesday, 16 February 2016 at 13:35:56 UTC, Dicebot wrote: On 02/16/2016 02:49 PM, Marc Schütz wrote: `@mutable` OTOH would be a useful for both C++, reference counting, caching, lazy initialization... But we need to find a way to keep most of the existing guarantees, especially concerning shareability. In my opinion @mutable would be a disaster of much higher destructive impact than head const. I am very opposed to it no matter how it is designed. Once you start considering it, you are better at simply throwing away existing const system and starting it all from scratch with D3. Logical const is harmful as it doesn't give and serious guarantees but gives developer a false sense of confidence. The last sentence in my post is crucial: "keep most of the existing guarantees". If we can ensure that access to @mutable data is strongly restricted and properly encapsulated, we don't lose anything. The type system would stay as it is, there would just be a way for it to be broken locally without provoking undefined behaviour. For example, it's always possible to use a global mutable associative array to store additional data connected with an immutable or const object (ignoring purity issues for the moment). That's safe because from the outside, there's no observable change to the state of the object itself, and the global AA's type (shared/thread-local) prevents race conditions. There's no reason why we can't have the same guarantees for embedded members, without resorting to an external AA. Have a look at my DIP [1] for lazy initialization of const members, which I now realize can be adapted to this use case. Basically, just replace `lazy` by `@mutable`, and most of the DIP is still valid. I'll try updating it when I find the time. [1] http://wiki.dlang.org/DIP85
Re: Head Const
speaking as a lay user - if this is going to be done maybe it should something like __gconst? similiar to __gshared which is ugly enough to make you concider not using it, but it's there if you really need it. On Wed, Feb 17, 2016 at 5:29 AM, Marc Schütz via Digitalmars-d wrote: > On Tuesday, 16 February 2016 at 13:35:56 UTC, Dicebot wrote: >> >> On 02/16/2016 02:49 PM, Marc Schütz wrote: >>> >>> `@mutable` OTOH would be a useful for both C++, reference counting, >>> caching, lazy initialization... But we need to find a way to keep most of >>> the existing guarantees, especially concerning shareability. >> >> >> In my opinion @mutable would be a disaster of much higher destructive >> impact than head const. I am very opposed to it no matter how it is >> designed. Once you start considering it, you are better at simply throwing >> away existing const system and starting it all from scratch with D3. Logical >> const is harmful as it doesn't give and serious guarantees but gives >> developer a false sense of confidence. > > > The last sentence in my post is crucial: "keep most of the existing > guarantees". If we can ensure that access to @mutable data is strongly > restricted and properly encapsulated, we don't lose anything. The type > system would stay as it is, there would just be a way for it to be broken > locally without provoking undefined behaviour. > > For example, it's always possible to use a global mutable associative array > to store additional data connected with an immutable or const object > (ignoring purity issues for the moment). That's safe because from the > outside, there's no observable change to the state of the object itself, and > the global AA's type (shared/thread-local) prevents race conditions. > > There's no reason why we can't have the same guarantees for embedded > members, without resorting to an external AA. Have a look at my DIP [1] for > lazy initialization of const members, which I now realize can be adapted to > this use case. Basically, just replace `lazy` by `@mutable`, and most of the > DIP is still valid. I'll try updating it when I find the time. > > [1] http://wiki.dlang.org/DIP85
Re: Head Const
On 02/16/2016 09:29 PM, Marc Schütz wrote: > The last sentence in my post is crucial: "keep most of the existing > guarantees". If we can ensure that access to @mutable data is strongly > restricted and properly encapsulated, we don't lose anything. The type > system would stay as it is, there would just be a way for it to be > broken locally without provoking undefined behaviour. One example of a existing guarantee you won't be able to keep, for example, is that any immutable allocation can be completely put into separate read-only memory. A very important property for building optimized allocators if you keep in mind sharing goals of immutability. > For example, it's always possible to use a global mutable associative > array to store additional data connected with an immutable or const > object (ignoring purity issues for the moment). That's safe because from > the outside, there's no observable change to the state of the object > itself, and the global AA's type (shared/thread-local) prevents race > conditions. Yes and this is how I believe it must be done. > There's no reason why we can't have the same guarantees for embedded > members, without resorting to an external AA. Have a look at my DIP [1] > for lazy initialization of const members, which I now realize can be > adapted to this use case. Basically, just replace `lazy` by `@mutable`, > and most of the DIP is still valid. I'll try updating it when I find the > time. > > [1] http://wiki.dlang.org/DIP85 The problem with this DIP is that it speaks about type system semantics and what matters first is memory model (which is currently very under-defined as soon as you step from a "the GC" world). Physical immutability is demanding but it also has great value in its simplicity and being hard to fool. Any language change that is going to reject this notion has to be really strongly justified in terms of what is gained and what is lost and not come simply because expressing such semantics is possible.
Re: Head Const
On 2/16/2016 5:35 AM, Dicebot wrote: In my opinion @mutable would be a disaster of much higher destructive impact than head const. I am very opposed to it no matter how it is designed. Once you start considering it, you are better at simply throwing away existing const system and starting it all from scratch with D3. Logical const is harmful as it doesn't give and serious guarantees but gives developer a false sense of confidence. I agree with you on that, and I've argued from that position before. Note that head const does not introduce any watering down nor destruction of the const/immutable/sharing type system. The main downside of head const would be language complexity.
Re: Head Const
On 2/16/2016 11:29 AM, Marc Schütz wrote: For example, it's always possible to use a global mutable associative array to store additional data connected with an immutable or const object (ignoring purity issues for the moment). That's safe because from the outside, there's no observable change to the state of the object itself, and the global AA's type (shared/thread-local) prevents race conditions. The trouble with that is you're relying on the programmer to ensure correctness. It'll revert D to C++ "trust the programmer" semantics. Furthermore, with the current mechanical guarantee of immutability, it opens up the possibility of relying on such beyond merely observable behavior - such as immutable data being placed in ROM, or special GC semantics.
Re: Head Const
On 2/16/2016 6:20 PM, Dicebot wrote: The problem with this DIP is that it speaks about type system semantics and what matters first is memory model (which is currently very under-defined as soon as you step from a "the GC" world). Physical immutability is demanding but it also has great value in its simplicity and being hard to fool. Any language change that is going to reject this notion has to be really strongly justified in terms of what is gained and what is lost and not come simply because expressing such semantics is possible. Yeah, I think we're on the same page with this. Most of what people find frustrating about D's transitive const/immutable is its very nature of not being sloppy and full of holes.
Re: Head Const
On 02/17/2016 04:44 AM, Walter Bright wrote: > Note that head const does not introduce any watering down nor > destruction of the const/immutable/sharing type system. The main > downside of head const would be language complexity. Yes, I agree - it isn't like head const is bad on its own, it simply feels unnecessary addition outside of C++ bindings for non-trivial language complication.
Re: Head Const
On Wednesday, 17 February 2016 at 02:51:06 UTC, Walter Bright wrote: The trouble with that is you're relying on the programmer to ensure correctness. It'll revert D to C++ "trust the programmer" semantics. You might end up in a worse position than C++, as people will cast away constness / immutability in third party libraries to get around the limitations. C++ added "mutable" keyword for a reason... with head const which is much less limiting to begin with... Furthermore, with the current mechanical guarantee of immutability, it opens up the possibility of relying on such beyond merely observable behavior - such as immutable data being placed in ROM, or special GC semantics. A nonissue. 1. The compiler has access to all reachable types. It can easily determine if there are reachable types with @mutable fields. 2. This is not a good argument for transitivity of immutability as head-immutability is sufficient.
Re: Head Const
On 2/16/2016 9:56 PM, Ola Fosheim Grøstad wrote: 1. The compiler has access to all reachable types. Nope. Opaque types are supported.
Re: Head Const
On 02/16/2016 09:44 PM, Walter Bright wrote: On 2/16/2016 5:35 AM, Dicebot wrote: In my opinion @mutable would be a disaster of much higher destructive impact than head const. I am very opposed to it no matter how it is designed. Once you start considering it, you are better at simply throwing away existing const system and starting it all from scratch with D3. Logical const is harmful as it doesn't give and serious guarantees but gives developer a false sense of confidence. I agree with you on that, and I've argued from that position before. Note that head const does not introduce any watering down nor destruction of the const/immutable/sharing type system. The main downside of head const would be language complexity. I profoundly oppose such an outlook. It has a name - prejudice, pure and simple. Rejecting possible future ideas "no matter what" even before they exist is extremely damaging. Consider: "I oppose implicit narrowing conversions regardless how they are designed" "static if is fundamentally flawed" "Variadics cannot be both simple and safe" etc. Andrei
Re: Head Const
On 02/16/2016 09:51 PM, Walter Bright wrote: On 2/16/2016 11:29 AM, Marc Schütz wrote: For example, it's always possible to use a global mutable associative array to store additional data connected with an immutable or const object (ignoring purity issues for the moment). That's safe because from the outside, there's no observable change to the state of the object itself, and the global AA's type (shared/thread-local) prevents race conditions. The trouble with that is you're relying on the programmer to ensure correctness. It'll revert D to C++ "trust the programmer" semantics. That doesn't seem the case to me. -- Andrei
Re: Head Const
On 02/16/2016 09:53 PM, Walter Bright wrote: Most of what people find frustrating about D's transitive const/immutable is its very nature of not being sloppy and full of holes. I thought it was the constructors and postblit that are the matter. (It may seem I'm on a snarky streak, but I do think that's the remaining issue with immutable). -- Andrei
Re: Head Const
On Monday, 15 February 2016 at 22:48:16 UTC, Walter Bright wrote: rears its head again :-) Head Const is what C++ has for const, i.e. it is not transitive, applies to one level only. D has transitive const. What head const will do for us: 1. make it easy to interface to C++ code that uses const, as currently it is not very practical to do so, you have to resort to pragma(mangle) 2. supports single assignment style of programming, even if the data is otherwise mutable The downside is, of course, language complexity. How about disallowing immutable data with extern(C++) types? With extern(C++) data always mutable, `const` could safely be reused to mean C++ const in bindings. Any `mutable`-style code would be implemented in C++.
Re: Head Const
On Wednesday, 17 February 2016 at 11:56:29 UTC, Andrei Alexandrescu wrote: On 02/16/2016 09:53 PM, Walter Bright wrote: Most of what people find frustrating about D's transitive const/immutable is its very nature of not being sloppy and full of holes. I thought it was the constructors and postblit that are the matter. (It may seem I'm on a snarky streak, but I do think that's the remaining issue with immutable). -- Andrei Yes, those are design problems with D's const that we really need to figure out how to fix, but they're not necessarily what people complain about. Many folks want to be able to do things like lazy initialization or caching or have a mutable reference count in a const/immutable object. The problems that usually come up basically boil down to folks wanting logical const rather than the compiler actually guaranteeing that data cannot be mutated via a const reference to that data without violating the type system. Lots of people basically want holes in const, because that's what C++ has, and not having holes in const prevents a number of potentially useful idioms (heck, you were trying to get around it with a container that you posted in the main newsgroup semi-recently and in RCString when you posted about it in dlang-study the other day). We need to solve the constructor/postblit issues in order for D's const to really work like it's supposed to, but even if that problem were solved, the basic tenet of D's design for const - that it has no backdoors - is what really makes it so that many folks avoid const completely, whereas others use it but violate the type system by casting away const and mutating, not realizing that unlike C++, it's undefined behavior in D. Regardless of whether Walter's (and thus D's) stance on const is the right one, it clearly doesn't jive with what a lot of folks (particularly those from C++) are looking for or expecting. - Jonathan M Davis
Re: Head Const
On Wednesday, 17 February 2016 at 12:46:08 UTC, Jonathan M Davis wrote: but violate the type system by casting away const and mutating, not realizing that unlike C++, it's undefined behavior in D. Regardless of whether Walter's (and thus D's) stance on const is the right one, it clearly doesn't jive with what a lot of folks (particularly those from C++) are looking for or expecting. - Jonathan M Davis struct (bla1, bla2) Test { int rc; bla1 int number; (bla2) Payload payload; void doSomething() (shared, immutable) { } } (shared, immutable) Test obj; bla1 becomes shared and bla2 becomes immutable, but the payload might not be immutable because it is (bla2) instead of just bla2. --- Can it be made in a way that the blablas do not participate in the static if? Could "shared", "immutable" be separated from the type int, float? (shared, immutable) Test!(int, float) obj; --- I did some "const(immutable) int" and the the compiler says basic type expected. What is the basic type from "const(shared, immutable)", would that be "const(shared const, const)"? And then the basic type from "const(const, immutable)" would be "const". --- Also would it be possible to determine the implicit conversion rules based solely on these qualifiers and not on the content of the object? (const, const) Test t = new (const, immutable) Test(); const Test t = new (const,const) Test(); ---
Re: Head Const
On 16/02/16 12:31, Guillaume Piolat wrote: I'm trying to say it politely. D2 const story is more complicated than its competitors. Both D1 "final" and C++ const always felt more useful and practical to me that the whole D2 immutable/const/inout thing. The current scheme seems to have marginal value in practice, lots of complexity, and is harder to use well (Unqual, inout) etc. Constructors can break it. I don't know why we should be that happy about our constness, maybe someone can explain. Seconded. It is one of the main tools I was using in C++ to avoid introducing bugs that I find completely unusable in D. I've tried to introduce it into code I'm writing. I'm used to such things being a problem to add later for code that was not written this way in advance. That's the case in C++ too. Here, however, there are many cases where I simply couldn't get it to the finish line. Between not being able to have head const, needing to mutate vars in a const object (a la refcount) and the compiler deciding functions are pure, I often end up facing a problem where I say "okay, forget it". And that's a real shame. We have at least one case where we're introducing runtime checks to make sure no one is accidentally changing a global var put there for reference, where declaring it const would have provided compile time assurance that we're okay. Shachar
Re: Head Const
On Tuesday, 16 February 2016 at 10:31:05 UTC, Guillaume Piolat wrote: On Tuesday, 16 February 2016 at 01:04:44 UTC, Walter Bright wrote: 2. supports single assignment style of programming, even if the data is otherwise mutable Like 'final'? We did get rid of that... Maybe we should resurrect it. I'm trying to say it politely. D2 const story is more complicated than its competitors. Both D1 "final" and C++ const always felt more useful and practical to me that the whole D2 immutable/const/inout thing. The current scheme seems to have marginal value in practice, lots of complexity, and is harder to use well (Unqual, inout) etc. Constructors can break it. I don't know why we should be that happy about our constness, maybe someone can explain. +1 It's weird because usually D prefers the practical solution over the ivory tower one. Nearly every time I end up using immutable or const for anything beyond say, a trivial function parameter, I always end up removing it. My C++ code is often far, far more 'const correct' than my D code.
Re: Head Const
On Wednesday, 17 February 2016 at 17:25:40 UTC, rsw0x wrote: It's weird because usually D prefers the practical solution over the ivory tower one. Nearly every time I end up using immutable or const for anything beyond say, a trivial function parameter, I always end up removing it. My C++ code is often far, far more 'const correct' than my D code. Well, Walter's take on it is that because C++'s const doesn't actually guarantee anything, it really isn't much different from simply documenting your code, and it's essentially useless. And to a point, he's right. All C++'s const does is prevent accidental mutation and serve as documentation that the code isn't supposed to mutate the logical state of the object. It's trivially circumvented to the point that it doesn't actually guarantee anything and can't actually be relied on for anything. So, essentially the argument is that C++'s const isn't actually practical. It's just fooling you into thinking that you're getting guarantees that you're not actually getting. But in practice, it does prevent accidental mutation, and as long as the programmer is behaving themselves reasonably well, it functions as documentation. And IMHO that definitely is worth something, much as Walter thinks that it isn't. So, while C++'s const doesn't really guarantee much, I'd much rather have it than nothing. A lot of what it comes down to though from a purely practical standpoint is that because we have immutable, const naturally becomes more restrictive, and while immutable obviously can't be used in many cases, it _does_ allow for cleaner, more efficient code where it can be used. So, as soon as you have immutable, you're almost forced to go to where D is with const - at least as long as const can refer to something that's immutable. Unfortunately, the net result is that while const is still very useful, there are a lot of cases in D where you can't use it, and you have to be a lot more careful about how and when you use it such that it gets used far less than in C++, and while where it _is_ used does provide much better guarantees, you lose out on a lot of protection against accidental mutation that your typical C++ code gets. Personally, I'm quite divided on the matter. The extra guarantees of D's const are very nice to have, but since it doesn't have any backdoors, there are a number of idioms that simply can't be done in D as long as const is involved, which definitely sucks. But interestingly, the more I've used D, the less happy I've been with C++'s const. It just has too many holes. In particular, the fact that it's not transitive is incredibly annoying, making it trivial to have cases where you give mutable access to something where you intended for it to be const, but it was only partially const. Even if we were to do something like add @mutable to D, I never want to see non-transitive const in D, and I wish that C++'s const were transitive, even if all of the rest of it was the same. - Jonathan M Davis
Re: Head Const
On Wednesday, 17 February 2016 at 17:47:02 UTC, Jonathan M Davis wrote: definitely sucks. But interestingly, the more I've used D, the less happy I've been with C++'s const. It just has too many holes. In particular, the fact that it's not transitive is What you could consider in C++ is to fully encapsulate your field values and use member functions to access them. Which is considered good modelling anyway. So if you access a member as "get_child() const", you return a const reference, which makes it transitive. What is annoying in C++ is that there is no getter/setter syntax...
Re: Head Const
On Wednesday, 17 February 2016 at 02:20:15 UTC, Dicebot wrote: One example of a existing guarantee you won't be able to keep, for example, is that any immutable allocation can be completely put into separate read-only memory. Yes, and it would be rejected statically (rule 2). I therefor don't consider this a problem. A very important property for building optimized allocators if you keep in mind sharing goals of immutability. This is considered too (rule 3). An object can only be immutable if all its embedded @mutable members are marked as shared. For example, it's always possible to use a global mutable associative array to store additional data connected with an immutable or const object (ignoring purity issues for the moment). That's safe because from the outside, there's no observable change to the state of the object itself, and the global AA's type (shared/thread-local) prevents race conditions. Yes and this is how I believe it must be done. There's no reason why we can't have the same guarantees for embedded members, without resorting to an external AA. Have a look at my DIP [1] for lazy initialization of const members, which I now realize can be adapted to this use case. Basically, just replace `lazy` by `@mutable`, and most of the DIP is still valid. I'll try updating it when I find the time. [1] http://wiki.dlang.org/DIP85 The problem with this DIP is that it speaks about type system semantics and what matters first is memory model (which is currently very under-defined as soon as you step from a "the GC" world). I'd say most of the memory model issues are the same as those with shared, which is just as undefined currently. That leaves possible reordering of accesses to immutable data. I don't have a complete answer to that at the moment, but most of it is likely also covered by the fact that the @mutable members must be shared in this case. Physical immutability is demanding but it also has great value in its simplicity and being hard to fool. Any language change that is going to reject this notion has to be really strongly justified in terms of what is gained and what is lost and not come simply because expressing such semantics is possible.
Re: Head Const
On Wednesday, 17 February 2016 at 02:51:06 UTC, Walter Bright wrote: On 2/16/2016 11:29 AM, Marc Schütz wrote: For example, it's always possible to use a global mutable associative array to store additional data connected with an immutable or const object (ignoring purity issues for the moment). That's safe because from the outside, there's no observable change to the state of the object itself, and the global AA's type (shared/thread-local) prevents race conditions. The trouble with that is you're relying on the programmer to ensure correctness. It'll revert D to C++ "trust the programmer" semantics. Are you referring to the quoted passage, or my DIP? If the latter, please note that it was initially written for a different purpose. Operations with @mutables will be @system, which is our usual way to deal with situations in which the programmer has to ensure correctness. The unsafe (@trusted) part is very small and well encapsulated, the unsafety doesn't leak. Furthermore, with the current mechanical guarantee of immutability, it opens up the possibility of relying on such beyond merely observable behavior - such as immutable data being placed in ROM, or special GC semantics. That's already covered in the DIP, see my reply to Dicebot.
Re: Head Const
On Wednesday, 17 February 2016 at 18:16:54 UTC, Ola Fosheim Grøstad wrote: On Wednesday, 17 February 2016 at 17:47:02 UTC, Jonathan M Davis wrote: definitely sucks. But interestingly, the more I've used D, the less happy I've been with C++'s const. It just has too many holes. In particular, the fact that it's not transitive is What you could consider in C++ is to fully encapsulate your field values and use member functions to access them. Which is considered good modelling anyway. So if you access a member as "get_child() const", you return a const reference, which makes it transitive. As soon as you start using pointers or containers or pointers or anything like that, pretty quickly, you can get situations like where the container is const, but its elements aren't. So, you may not be able to mutate the stuff that's directly a member of the class, you can still mutate stuff that it refers to or even owns. D doesn't allow that, which I think is a definite improvement. If you return a reference or pointer to a member variable from a const member function in D, it's const through and through with no holes, and I think that that is unequivocally a good thing. Where things get annoying is the type of situation where you'd mark something mutable in C++, since those just don't work at all in D. You're forced to not use const or redesign what you're doing so that it doesn't need mutable (which usually means not using const). - Jonathan M Davis
Re: Head Const
On Wednesday, 17 February 2016 at 19:03:25 UTC, Jonathan M Davis wrote: As soon as you start using pointers or containers or pointers or anything like that, pretty quickly, you can get situations like where the container is const, but its elements aren't. Yes, generic containers is an issue in C++. You can't just recast the template parameter as const, as the internals could depend on it not being const when the container was instantiated (if-test on const in the constructor). This is a problem we get when meta-programming becomes powerful enough. The easy way out is to use a wrapper and implement the constness in the wrapper (a class with just a pointer and methods). Basically implement a reference type for container access.
Re: Head Const
On 2/17/2016 3:56 AM, Andrei Alexandrescu wrote: On 02/16/2016 09:53 PM, Walter Bright wrote: Most of what people find frustrating about D's transitive const/immutable is its very nature of not being sloppy and full of holes. I thought it was the constructors and postblit that are the matter. (It may seem I'm on a snarky streak, but I do think that's the remaining issue with immutable). -- Andrei Logical const!
Re: Head Const
On 2/17/2016 4:03 AM, Jakob Ovrum wrote: How about disallowing immutable data with extern(C++) types? With extern(C++) data always mutable, `const` could safely be reused to mean C++ const in bindings. Any `mutable`-style code would be implemented in C++. That doesn't help with trying to match the mangling for: mutable pointer to const pointer to mutable pointer to const pointer to mutable pointer to void It's still an interesting idea, but it would precluded passing any D immutable data structures to C++ code. I tend to think such needs to be possible, with the proviso that you'd have to verify that the C++ end did not violate the immutability rules. It's normal practice to write C++ code as if const were transitive.
Re: Head Const
On 2/17/2016 10:54 AM, Marc Schütz wrote: That's already covered in the DIP, see my reply to Dicebot. Note that D supports opaque types, meaning it is not always possible to transitively follow all types to see if they have mutable members.
Re: Head Const
On Wednesday, 17 February 2016 at 22:06:20 UTC, Walter Bright wrote: On 2/17/2016 10:54 AM, Marc Schütz wrote: That's already covered in the DIP, see my reply to Dicebot. Note that D supports opaque types, meaning it is not always possible to transitively follow all types to see if they have mutable members. We _could_ have something like @mutable on member variables which made them mutable in spite of being in a const object and which made it illegal for that type to ever be immutable, but we'd be forced to have an additional attribute on the type itself (e.g. @contains_mutable) because of the opaque type issue, and at that point, the compiler could just look at the attribute on the type to know that it couldn't be immutable, and similar to abstract, it could then require that any type that it's a member of t hen be marked with that attribute. So, it's uglier than simply marking a member variable with @mutable, but it's certainly feasible. The question really is whether we want such a backdoor in const and whether it's worth the extra complication. It _would_ be explicitly marked, making it fairly easy to determine when a const type might actually mutate some of its state (though presumably, if it's used responsibly, it would only mutate state that was really part of the logical state of the object - e.g. a mutex or a reference count). So, while const would be weakened on some level, it wouldn't be completely invisible, and you couldn't accidentally have a member variable with an @mutable member, since you'd be forced to mark the type containing it with @contains_mutable. And it _would_ allow a number of idioms that many folks keep wanting and complaining about without actually making it defined behavior to cast away const and mutate (even Andrei has been running into issues lately with containers and RCString where he's needed @mutable and then done stuff like cast away const and mutating, thinking that that was okay as long as the underlying object wasn't immutable). It would be a backdoor, but it would be a well-defined one rather than allowing a free-for-all. I honestly don't know whether adding something like @mutable is a good idea or not, but it is very clear that there are a number of idioms that are impossible in D when const is involved that lead a lot of programmers to either not use const or to cast it away and mutate, thinking that that's okay, because it's okay in C++, and it does tend to work as long as immutable isn't involved, much as it's undefined behavior. Having @mutable would make const usable for a lot of programs that otherwise have to mostly abandon it, but it would add some extra complication to the language, and it would mean that there's a backdoor in const, even if it's one that's actually safe to use (unlike casting away const and mutating). But what _is_ clear to me is that transitive const has been a big win, and while I can definitely live with C++'s version of const if I have to, it drives me nuts now that it's not transitive. Without transitivity, it's too easy to end up with mutable references to stuff from const functions, just because they're not directly part of the object. Transitivity fixes all that. - Jonathan M Davis
Re: Head Const
On 02/17/2016 04:07 PM, Walter Bright wrote: On 2/17/2016 3:56 AM, Andrei Alexandrescu wrote: On 02/16/2016 09:53 PM, Walter Bright wrote: Most of what people find frustrating about D's transitive const/immutable is its very nature of not being sloppy and full of holes. I thought it was the constructors and postblit that are the matter. (It may seem I'm on a snarky streak, but I do think that's the remaining issue with immutable). -- Andrei Logical const! How do you mean that? I see no connection. -- Andrei
Re: Head Const
On 2/17/2016 2:31 PM, Jonathan M Davis wrote: We _could_ have something like @mutable on member variables which made them mutable in spite of being in a const object and which made it illegal for that type to ever be immutable, but we'd be forced to have an additional attribute on the type itself (e.g. @contains_mutable) because of the opaque type issue, and at that point, the compiler could just look at the attribute on the type to know that it couldn't be immutable, and similar to abstract, it could then require that any type that it's a member of t hen be marked with that attribute. So, it's uglier than simply marking a member variable with @mutable, but it's certainly feasible. It would seem that implementing headconst as a type constructor would let people who wanted mutable members have their way, without introducing backdoors in const.
Re: Head Const
On Wednesday, 17 February 2016 at 22:44:27 UTC, Walter Bright wrote: It would seem that implementing headconst as a type constructor would let people who wanted mutable members have their way, without introducing backdoors in const. I really don't see how that's the same thing at all. The main problems with const pop up when what folks need is something like C++'s mutable, because they need to be able to do something like a reference count or a mutex which is not really part of the logical state of the object but is still part of the object. Having some form of non-transitive const allows for things like a const pointer to non-const data, but it doesn't help at all when what you need to do is treat most of the object as const while treating a small portion of it as mutable. Sure, that's not completely transitive, because parts of the object are mutable, but it's specific parts of the object, not which part of a pointer declaration is const and which isn't. Maybe I'm missing something, but I don't see how adding non-transitive const to D in order to solve the C++ declaration problem would help at all with the complaints that folks have with D's const. The complaints with D's const are almost entirely about the lack of backdoors (be they well-defined like a mutable attribute or a free-for-all like casting away const and mutating). - Jonathan M Davis
Re: Head Const
On 2/17/2016 3:12 PM, Jonathan M Davis wrote: On Wednesday, 17 February 2016 at 22:44:27 UTC, Walter Bright wrote: It would seem that implementing headconst as a type constructor would let people who wanted mutable members have their way, without introducing backdoors in const. I really don't see how that's the same thing at all. The main problems with const pop up when what folks need is something like C++'s mutable, because they need to be able to do something like a reference count or a mutex which is not really part of the logical state of the object but is still part of the object. Having some form of non-transitive const allows for things like a const pointer to non-const data, but it doesn't help at all when what you need to do is treat most of the object as const while treating a small portion of it as mutable. Sure, that's not completely transitive, because parts of the object are mutable, but it's specific parts of the object, not which part of a pointer declaration is const and which isn't. Maybe I'm missing something, but I don't see how adding non-transitive const to D in order to solve the C++ declaration problem would help at all with the complaints that folks have with D's const. The complaints with D's const are almost entirely about the lack of backdoors (be they well-defined like a mutable attribute or a free-for-all like casting away const and mutating). If the headconst was one level up, the struct can have mutating members.
Re: Head Const
On Wednesday, 17 February 2016 at 23:47:54 UTC, Walter Bright wrote: On 2/17/2016 3:12 PM, Jonathan M Davis wrote: On Wednesday, 17 February 2016 at 22:44:27 UTC, Walter Bright wrote: It would seem that implementing headconst as a type constructor would let people who wanted mutable members have their way, without introducing backdoors in const. I really don't see how that's the same thing at all. The main problems with const pop up when what folks need is something like C++'s mutable, because they need to be able to do something like a reference count or a mutex which is not really part of the logical state of the object but is still part of the object. Having some form of non-transitive const allows for things like a const pointer to non-const data, but it doesn't help at all when what you need to do is treat most of the object as const while treating a small portion of it as mutable. Sure, that's not completely transitive, because parts of the object are mutable, but it's specific parts of the object, not which part of a pointer declaration is const and which isn't. Maybe I'm missing something, but I don't see how adding non-transitive const to D in order to solve the C++ declaration problem would help at all with the complaints that folks have with D's const. The complaints with D's const are almost entirely about the lack of backdoors (be they well-defined like a mutable attribute or a free-for-all like casting away const and mutating). If the headconst was one level up, the struct can have mutating members. Yes, but that really isn't going to help much code. It would be useless for ref-counting const objects, it wouldn't allow you to put a mutex in a const object, and you couldn't do anything with caching calculated properties in a const object. All it means is that you can do something like have a const pointer to a mutable object, which in my experience is usually useless. There have been plenty of complaints about having to use Rebindable instead of having tail-const for classes, but I've rarely seen anyone complain about the lack of head-const. And what's usually the case is that folks want logical const, and I don't see how that's possible without either having something similar to C++'s mutable or make it well-defined to cast away const and mutate. Having something similar to C++'s mutable would at least solve most of the problem while making it explicit and safe, whereas allowing casting away const and mutating would just throw all of the guarantees of const out the window and risk serious problems with immutable, making it so that all const really did was prevent accidental mutation and serve as documentation of intent. Something like @mutable would at least retain the guarantees that we currently have when @mutable isn't used and restrict the effects of removing const to very well defined areas such that it wouldn't run afoul of immutable. So, if we want to actually solve the problems that folks typically complain about with const, I think that something like @mutable is the way to go. Now, we can certainly decide that that's not worth it and that those idioms simply won't work in D as long as const is used, but I really don't think that head-const is going to solve the same problem at all or really make much of anyone much happier aside from how it helps with C++ interoperability. Adding tail-const for classes to the language would make a whale of a lot more people happy than head-const would even though Rebindable ostensibly solves that problem already. - Jonathan M Davis
Re: Head Const
On 16/02/2016 8:29 PM, Ola Fosheim Grøstad wrote: I agree with the principle, but not as a library function, because: 1. you want virtual functions to work out ok virtual functions don't even need mangling. But even if they did it would work just fine anyway. 2. making D more reliant on macroish string processing is not good It's not macroish string processing, it's embedding a subset of C++ declarations like a DSL. The difference is that the C++ can be fully type-checked and semantically analysed, errors will not leak into the generated source. You would need something along the lines of: 1. «extern "C++"» the essence of the class definition in plain C++ syntax 2. add to this syntax a translation for each parameter what it means in D. E.g. extern "C++" { class X { mutable int rc; virtual func1(const A* ptr); @reinterpret(ptr, head_const_ptr!A) virtual func2(const A* ptr); @reinterpret(ptr, const A*) virtual func3(A* ptr); virtual func4(const A* ptr); @reinterpret(ptr, const_rc!A*) }; } We don't 'need' compiler support beyond what we have, for any of this.
Re: Head Const
On Wednesday, 17 February 2016 at 17:47:02 UTC, Jonathan M Davis wrote: On Wednesday, 17 February 2016 at 17 you're not actually getting. But in practice, it does prevent accidental mutation, and as long as the programmer is behaving themselves reasonably well, Unfortunately, the net result is that while const is still very useful, there are a lot of cases in D where you can't use it, and you have to be a lot more careful about how and when you use it such that it gets used far less than in C++, and while where it _is_ used does provide much better guarantees, you lose out on a lot of protection against accidental mutation that your typical C++ code gets. Things would be a bit friendlier for newcomers if the documentation were a bit clearer on the 'having to be a lot more careful on how and when you use it' so one doesnt end up finding this out the hard way.
Re: Head Const
Greetings Having coded a multithreaded library in D, I sorely miss headcost like specifier. The library code dealt with lots of composite class object hierarchy where many element objects of a parent are *effectively immutable*. You build an effectively immutable object in the constructor of the parent class and then you are done since you do not assign to these object handles again. As a result there is no need to lock the parent object while accessing an *effectively immutable* object. The parent as well as child objects are mutable otherwise. Dlang's immutable and const do not provide such a contract. So +1 for headconst if this specifier can be used with class elements as well and not just in the function parameter list. - Puneet
Re: Head Const
On 2/17/2016 2:38 PM, Andrei Alexandrescu wrote: On 02/17/2016 04:07 PM, Walter Bright wrote: On 2/17/2016 3:56 AM, Andrei Alexandrescu wrote: On 02/16/2016 09:53 PM, Walter Bright wrote: Most of what people find frustrating about D's transitive const/immutable is its very nature of not being sloppy and full of holes. I thought it was the constructors and postblit that are the matter. (It may seem I'm on a snarky streak, but I do think that's the remaining issue with immutable). -- Andrei Logical const! How do you mean that? I see no connection. -- Andrei People want to relax transitive immutability so they can implement logical const.
Re: Head Const
On 2/17/2016 4:41 PM, Jonathan M Davis wrote: Yes, but that really isn't going to help much code. It would be useless for ref-counting const objects, it wouldn't allow you to put a mutex in a const object, and you couldn't do anything with caching calculated properties in a const object. Embed a headconst pointer to the thing you want to mutate.
Re: Head Const
On 18.02.2016 08:51, Walter Bright wrote: On 2/17/2016 4:41 PM, Jonathan M Davis wrote: Yes, but that really isn't going to help much code. It would be useless for ref-counting const objects, it wouldn't allow you to put a mutex in a const object, and you couldn't do anything with caching calculated properties in a const object. Embed a headconst pointer to the thing you want to mutate. const(headconst(T)) is the same as const(T), no?
Re: Head Const
Too illustrate how headconst can help with multithreading (shared memory) code, I have created this small snippet. In my experience, there are lots of *effectively immutable* objects (like Foo.bar in the code snippet). If these variables can be declared headconst, a lot of hierarchical synchronization locking (and therefor deadlock situations) can be saved. In the present Dlang semantics we do not have any specifier to make *effective mutability* effective. So presently I have to create a naming and coding discipline when declaring and using effectively mutable hierarchical objects. class Bar { void funcBar() { } } class Frop { void funcFrop() { } } class Foo { this() { synchronized(this) { bar = new Bar(); } } void setFrop() { frop = new Frop(); } headconst Bar bar;// effectively immutable Frop frop;// mutable void funcBar() { // no locking required bar.funcBar(); // since bar is effectively immutable } void funcFrop() { synchronized(this) {// lock necessary since frop frop.funcFrop(); // is mutable } } } void main() { Foo foo = new Foo(); // yield for other threads // foo.setFrop();// in some other thread foo.funcBar(); foo.funcFrop(); }
Re: Head Const
On Wednesday, 17 February 2016 at 22:01:45 UTC, Walter Bright wrote: On 2/17/2016 4:03 AM, Jakob Ovrum wrote: How about disallowing immutable data with extern(C++) types? With extern(C++) data always mutable, `const` could safely be reused to mean C++ const in bindings. Any `mutable`-style code would be implemented in C++. That doesn't help with trying to match the mangling for: mutable pointer to const pointer to mutable pointer to const pointer to mutable pointer to void Oh yeah, I totally forgot about that. It's still an interesting idea, but it would precluded passing any D immutable data structures to C++ code. I tend to think such needs to be possible, with the proviso that you'd have to verify that the C++ end did not violate the immutability rules. It's normal practice to write C++ code as if const were transitive. It's a sizeable sacrifice. I thought maybe it could be worth it for a simple and consistent implementation, but without syntactical support for `char * const` mangling it's substantially less appealing.
Re: Head Const
On 2/17/2016 11:58 PM, Timon Gehr wrote: const(headconst(T)) is the same as const(T), no? Er, yes?
Re: Head Const
On 18.02.2016 10:24, Walter Bright wrote: On 2/17/2016 11:58 PM, Timon Gehr wrote: const(headconst(T)) is the same as const(T), no? Er, yes? Jonathan wanted to embed a mutable reference count within a const object. Contrary to your suggestion, headconst won't help with that.
Re: Head Const
On Thursday, 18 February 2016 at 06:49:34 UTC, Daniel Murphy wrote: It's not macroish string processing, it's embedding a subset of C++ declarations like a DSL. The difference is that the C++ can be fully type-checked and semantically analysed, errors will not leak into the generated source. Ok, if you can resolve everything this way. I agree that you end up spending way too much effort on bringing C++ into D, for it to be meaningful. My headerfiles used to be like 10-20% of my C++ source. Now it is at 40-50% of my C++ source... My C++ library code is at 90% header file only. Things like concepts will probably just ensure that this trend persists. So I am not sure if interfacing with compiled C++ only is particularly useful. I think you need support for generating C++ source code stubs for interfacing so that you can use inlined non-instantiated C++ code (or just stick to C). LLVM whole program optimization (linking LLVM IR) would "melt" it all together reasonably efficiently.
Re: Head Const
On 02/17/2016 08:48 PM, Marc Schütz wrote: > On Wednesday, 17 February 2016 at 02:20:15 UTC, Dicebot wrote: >> One example of a existing guarantee you won't be able to keep, for >> example, is that any immutable allocation can be completely put into >> separate read-only memory. > > Yes, and it would be rejected statically (rule 2). I therefor don't > consider this a problem. You pretty much prohibit any but typed allocators this way. I do see it as a problem. >> A very important property for building optimized allocators if you >> keep in mind sharing goals of immutability. > > This is considered too (rule 3). An object can only be immutable if all > its embedded @mutable members are marked as shared. That sounds rather weird considering immutable data neither needs nor synchronization nor CPU cache reloading and @mutable shared one needs both.
Re: Head Const
On 02/17/2016 01:53 PM, Andrei Alexandrescu wrote: > On 02/16/2016 09:44 PM, Walter Bright wrote: >> On 2/16/2016 5:35 AM, Dicebot wrote: >>> In my opinion @mutable would be a disaster of much higher destructive >>> impact than head const. I am very opposed to it no matter how it is >>> designed. Once you start considering it, you are better at simply >>> throwing away existing const system and starting it all from scratch >>> with D3. Logical const is harmful as it doesn't give and serious >>> guarantees but gives developer a false sense of confidence. >> >> >> I agree with you on that, and I've argued from that position before. >> >> Note that head const does not introduce any watering down nor >> destruction of the const/immutable/sharing type system. The main >> downside of head const would be language complexity. > > I profoundly oppose such an outlook. It has a name - prejudice, pure and > simple. Rejecting possible future ideas "no matter what" even before > they exist is extremely damaging. > > Consider: > > "I oppose implicit narrowing conversions regardless how they are designed" > > "static if is fundamentally flawed" > > "Variadics cannot be both simple and safe" > > etc. It isn't that the concept of logical const / @mutable is inherently flawed and I am rejecting any possibility of designing it decently. It is the fact that it is quite alien to existing system and adding it will inevitably be a hack of some sort. Redesigning the whole thing with logical const in mind could be feasible but is a bit too late. Retro-fitting into existing system is a different thing though. Consider "static if is fundamentally flawed" vs "static if is fundamentally flawed [when applied to existing idiomatic C++ code]". Former is opinionated prejudice. Latter can quite easily be true (even if I personally wouldn't agree with it).
Re: Head Const
On Thursday, 18 February 2016 at 10:47:57 UTC, Timon Gehr wrote: On 18.02.2016 10:24, Walter Bright wrote: On 2/17/2016 11:58 PM, Timon Gehr wrote: const(headconst(T)) is the same as const(T), no? Er, yes? Jonathan wanted to embed a mutable reference count within a const object. Contrary to your suggestion, headconst won't help with that. Exactly. The problem that folks frequently want to be able to solve that they simply cannot solve with D's const (and headconst wouldn't help) is that they want to be able to pass an object to a function that takes it as const or return a const object from a const member function and have stuff like reference counting, caching, mutexes, etc. work - stuff that has to be part of the object (at least in pure code) but which isn't part of its logical state and cannot be const. That's why people keep asking for logical const. D's const simply does not work with many use cases (especially in the more performance-driven, @system cases), which tends to mean that you just can't use it. Generic code in particular can't afford to use it, because it immediately shuts out a whole set of types (just like marking a templated function with @safe or pure is a bad idea). So, const ends up being useful with built-in stuff like integers and arrays, but it generally fails once user-defined types get involved. And remember that headconst is essentially cppconst, and C++ has to have the mutable keyword and allow casting away const and mutating in order to solve these sorts of problems. From what I can tell, fundamentally, there are some common use cases that will not work with const unless it has backdoors. Mutexes and reference counts are great examples of that. They need to be associated with the object, but they cannot be treated as const in order to work even if the actual data is treated as const. Aside from C++ interoperability, headconst really doesn't seem like it's going to solve much. I love the fact that D treats it as undefined behavior to cast away const and mutate. That's a huge win with regards to compiler guarantees and being able to trust const not to mutate stuff on its own (though other, mutable references to the same data obviously can). And it's required for const to interact well with immutable. But having no backdoors at all keeps popping up as a problem - and one that headconst clearly can't solve, because it doesn't solve it in C++. Andrei has run into problems with this as he's being working on the new containers and RCString. What he needed was the mutable keyword, and he was casting away const and mutating to do it, and I had to point out to him that that wasn't defined behavior. And if he's forgetting that, what's the lay user doing? Some things need backdoors from const, or they simply can't use const, and if you use those in your code, it quickly cascades such that you're not using const much of anywhere, because you can't. A number of folks have stated in the newsgroup that they generally avoid const in D, because it's too restrictive to be useful. We can decide that we just don't care about those problems and that you simply don't get to use const in those cases, but that means that you lose out on all of the benefits that you get with const preventing you from mutating objects that aren't supposed to be mutated, and if you have to strip out const from enough of your code base, then you lose out on its benefits entirely, meaning that C++'s const with its backdoors would have been a win in comparison, sad as that may be. The more I look at it, the more I'm inclined to think that introducing @mutable for member variables with a corresponding, required attribute on the struct or class it's in (e.g. @has_mutable) is really what we need to be able to solve this problem and make D usable in some of these high performance cases that would be using the mutable keyword in C++. It solves the logical const problem without totally throwing away the compiler guarantees. Any type without @has_mutable functions as it always has, and the cases where @mutable/@has_mutable would be required would then work with const, gaining all of its benefits for the non-@mutable members, and it would allow the compiler to prevent you from doing stupid stuff like mutating immutable objects, because an object with @has_mutable couldn't be immutable. Regardless, I really don't think that adding headconst is worth the complication. It doesn't solve enough to be worth it. I'd much rather see something specific to extern(C++) that's used for mangling and which doesn't allow you to pass const objects to it unless the C++ declaration is equivalent to transitive const (and probably which treats return types as transitive const if they're partially const). Remember that we ditched having headconst, tailconst, etc. in D ages ago, because it was deemed overcomplicated. And I don't think that that's changed. Persona
Re: Head Const
On 2/18/2016 2:47 AM, Timon Gehr wrote: On 18.02.2016 10:24, Walter Bright wrote: On 2/17/2016 11:58 PM, Timon Gehr wrote: const(headconst(T)) is the same as const(T), no? Er, yes? Jonathan wanted to embed a mutable reference count within a const object. Contrary to your suggestion, headconst won't help with that. Of course it will: headconst pointer -> pointer -> refcount -> const pointer -> ... constant stuff ...
Re: Head Const
On Thursday, 18 February 2016 at 12:03:00 UTC, Walter Bright wrote: On 2/18/2016 2:47 AM, Timon Gehr wrote: On 18.02.2016 10:24, Walter Bright wrote: On 2/17/2016 11:58 PM, Timon Gehr wrote: const(headconst(T)) is the same as const(T), no? Er, yes? Jonathan wanted to embed a mutable reference count within a const object. Contrary to your suggestion, headconst won't help with that. Of course it will: headconst pointer -> pointer -> refcount -> const pointer -> ... constant stuff ... Sure, until you need to put that headconst pointer in a const object or pass it to a function that takes const. Then the whole thing is const, and the refcount doesn't work. Remember that C++ has headconst, and it still needs the mutable keyword in order to solve these problems. Also, adding headconst is suddenly going to create this dichotomy between const and headconst where you have to figure out which to use and where they really don't interact nicely, because you can't pass a const object to something that takes headconst, and if you do pass a headconst object to something that takes const, then all of the stuff like ref-counting stops working. In a way, we'd be forking D between const and headconst. Some code would use one, and some code would use the other. And neither of them really solves the problem that the mutable keyword solves. Neither of them allows for any kind of logical const, and _that_ really is the problem that people keep complaining about and want solved. headconst may solve the extern(C++) problem, but it really doesn't solve the problems we have with const. At best, it reduces them slightly while introducing a whole extra set of complications to an already complicated language. - Jonathan M Davis