Re: Article: Why Const Sucks
On Thursday, 8 March 2018 at 15:13:09 UTC, Steven Schveighoffer wrote: On 3/8/18 9:58 AM, joe wrote: On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: /snip May be not entirely related, but a little gotcha. given: interface XY {} class Foo: XY {} class Bar: XY {} void doSomething(in XY sth) { auto foo = cast(Foo)sth; // error in @safe code } But the compiler doesn't emit a warning that const got cast away in @system code. Since @system is the default setting and casting const away is by definition undefined behavior, there should be some feedback. If you cast away const in C++ you need to be explicit about it by using const_cast, documents the intention rather than D's swiss-army-cast. This is a problem with the cast system, not const. ... I figured. Sorry for off topic. ... D has one cast mechanism, and it's used to: a) dynamic type cast b) hook user-supplied opCast c) enter undefined behavior by casting away const/immutable. I really think we should have a separate mechanism for a and b, as they are far less dangerous than c. std.conv.to does this as well, ... Thanks for pointing this out. ...but the language should have some safeguards against using "safe casting" incorrectly. -Steve yes, that's what I was aiming for :) This one got me because I assumed, with all the improved security in place, D would never silently do such a thing... Assumptions, right? ...the root of all evil. I should know better. haha. Anyways, thanks for the insight and have a nice Sunday.
Re: Article: Why Const Sucks
On Monday, 5 March 2018 Jonathan M Davis wrote at http://jmdavisprog.com/articles/why-const-sucks.html: What Java has instead is `final`, which IMHO is borderline useless In Java `final` is extremely useful for efficient threadsafe code.
Re: Article: Why Const Sucks
On 3/8/18 9:58 AM, joe wrote: On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: /snip May be not entirely related, but a little gotcha. given: interface XY {} class Foo: XY {} class Bar: XY {} void doSomething(in XY sth) { auto foo = cast(Foo)sth; // error in @safe code } But the compiler doesn't emit a warning that const got cast away in @system code. Since @system is the default setting and casting const away is by definition undefined behavior, there should be some feedback. If you cast away const in C++ you need to be explicit about it by using const_cast, documents the intention rather than D's swiss-army-cast. This is a problem with the cast system, not const. D has one cast mechanism, and it's used to: a) dynamic type cast b) hook user-supplied opCast c) enter undefined behavior by casting away const/immutable. I really think we should have a separate mechanism for a and b, as they are far less dangerous than c. std.conv.to does this as well, but the language should have some safeguards against using "safe casting" incorrectly. -Steve
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: /snip May be not entirely related, but a little gotcha. given: interface XY {} class Foo: XY {} class Bar: XY {} void doSomething(in XY sth) { auto foo = cast(Foo)sth; // error in @safe code } But the compiler doesn't emit a warning that const got cast away in @system code. Since @system is the default setting and casting const away is by definition undefined behavior, there should be some feedback. If you cast away const in C++ you need to be explicit about it by using const_cast, documents the intention rather than D's swiss-army-cast.
Re: Article: Why Const Sucks
On Tuesday, 6 March 2018 at 17:41:42 UTC, H. S. Teoh wrote: Yeah, Andrei has admitted before that this is probably what he would do today, if he were given a second chance to design ranges. But at the time, the landscape of D was rather different, and certain language features didn't exist yet (sorry, can't recall exactly which off the top of my head), so he settled with the compromise that we have today. As they say, hindsight is always 20/20. But it wasn't so easy to foresee the consequences at the time when the very concept of ranges was still brand new. Andrei's 'On Iteration'[0] was published 2009-11-09. Postblits had been in the language for about a year and a half[1], and @disable arrived early 2010[2]. Both features were probably too new to warrant being an integral part of the design of ranges. -- Simen [0]: http://www.informit.com/articles/printerfriendly/1407357 [1]: https://dlang.org/changelog/2.012.html [2]: https://dlang.org/changelog/2.040.html
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 17:38:52 UTC, H. S. Teoh wrote: struct Container { auto opSlice() const { static struct Result { private Container impl; private int n; // internal mutable state @property bool empty() { ... } ... // rest of range API } return Result(this); } } That's definitely a know problem with ranges that hasn't yet been solved. The known workaround is to implement Range and ConstRange. This idiom could prolly be encapsulated in a safe and generic template type to avoid boilerplate. If we had some covariant mechansim to implement sth. similar to struct Range(T) if (is(T InoutT == inout)) { inout(InoutT) opIndex(size_t i) inout; } that would be ideal, but that isn't exactly trivial. I think we had a better issue for this topic, but here is one ticket asking for this feat. [9983 – inout type can not be used as a parameter for structure template](https://issues.dlang.org/show_bug.cgi?id=9983)
Re: Article: Why Const Sucks
On Tuesday, March 06, 2018 19:06:25 Martin Nowak via Digitalmars-d-announce wrote: > On Tuesday, 6 March 2018 at 18:17:58 UTC, Jonathan M Davis wrote: > > I'm not actually convinced that killing auto-decoding is really > > much better. > > I don't think the problem is auto-decoding in string range > adapters, but repeated validation. > https://issues.dlang.org/show_bug.cgi?id=14519#c32 > If you know that sth. works on code units just use > .representation. > > There is the related annoyance when the user of a function > presumably knows to only deal with ASCII strings but algorithms > fail, e.g. splitter.popBack or binary search. This one is tricky > because broken unicode support is often rooted in ignoring it's > existence. Yes, using stuff like representation or byCodeUnit helps to work around the auto-decoding, but as long as it's there, you have to constantly work around it if you care about efficiency with strings and/or want to be able to retain the original string type where possible. At this point, I think that it's pretty clear that we wouldn't have it if we could do stuff from scratch, but of course, we can't do stuff from scratch, because that would break everything. - Jonathan M Davis
Re: Article: Why Const Sucks
On Tuesday, 6 March 2018 at 18:17:58 UTC, Jonathan M Davis wrote: I'm not actually convinced that killing auto-decoding is really much better. I don't think the problem is auto-decoding in string range adapters, but repeated validation. https://issues.dlang.org/show_bug.cgi?id=14519#c32 If you know that sth. works on code units just use .representation. There is the related annoyance when the user of a function presumably knows to only deal with ASCII strings but algorithms fail, e.g. splitter.popBack or binary search. This one is tricky because broken unicode support is often rooted in ignoring it's existence.
Re: Article: Why Const Sucks
On Tuesday, March 06, 2018 10:47:36 H. S. Teoh via Digitalmars-d-announce wrote: > On Tue, Mar 06, 2018 at 01:31:39PM -0500, Steven Schveighoffer via Digitalmars-d-announce wrote: > > On 3/6/18 10:39 AM, Jonathan M Davis wrote: > > > Yeah. If you're dealing with generic code rather than a specific > > > range type that you know is implicitly saved when copied, you have > > > to use save so often that it's painful, and almost no one does it. > > > e.g. > > > > > > equal(lhs.save, rhs.save) > > > > > > or > > > > > > immutable result = range.save.startsWith(needle.save); > > > > Yep. The most frustrating thing about .save to me is that .save is > > nearly always implemented as: > > > > auto save() { return this; } > > > > This just screams "I really meant just copying". > > Yeah, and also: > > auto save() { > auto copy = this; > copy.blah = blah.dup; > return this; > } > > Which just screams "I'm really just a postblit in disguise". That's exactly what it is. It's a postblit constructor that you have to call manually and which works for classes and dynamic arrays in addition to structs. - Jonathan M Davis
Re: Article: Why Const Sucks
On Tue, Mar 06, 2018 at 01:31:39PM -0500, Steven Schveighoffer via Digitalmars-d-announce wrote: > On 3/6/18 10:39 AM, Jonathan M Davis wrote: > > Yeah. If you're dealing with generic code rather than a specific > > range type that you know is implicitly saved when copied, you have > > to use save so often that it's painful, and almost no one does it. > > e.g. > > > > equal(lhs.save, rhs.save) > > > > or > > > > immutable result = range.save.startsWith(needle.save); > > Yep. The most frustrating thing about .save to me is that .save is > nearly always implemented as: > > auto save() { return this; } > > This just screams "I really meant just copying". Yeah, and also: auto save() { auto copy = this; copy.blah = blah.dup; return this; } Which just screams "I'm really just a postblit in disguise". T -- This is not a sentence.
Re: Article: Why Const Sucks
On 3/6/18 10:39 AM, Jonathan M Davis wrote: Yeah. If you're dealing with generic code rather than a specific range type that you know is implicitly saved when copied, you have to use save so often that it's painful, and almost no one does it. e.g. equal(lhs.save, rhs.save) or immutable result = range.save.startsWith(needle.save); Yep. The most frustrating thing about .save to me is that .save is nearly always implemented as: auto save() { return this; } This just screams "I really meant just copying". -Steve
Re: Article: Why Const Sucks
On Tue, Mar 06, 2018 at 11:20:56AM -0700, Jonathan M Davis via Digitalmars-d-announce wrote: > On Tuesday, March 06, 2018 09:41:42 H. S. Teoh via Digitalmars-d-announce > wrote: > > As they say, hindsight is always 20/20. But it wasn't so easy to > > foresee the consequences at the time when the very concept of ranges > > was still brand new. > > Except that even worse, I'd argue that hindsight really isn't 20/20. > We can see a lot of the mistakes that were made, and if we were > starting from scratch or otherwise willing to break a lot of code, we > could change stuff like the range API based on the lessons learned. > But we'd probably still screw it up, because we wouldn't have the > experience with the new API to know where it was wrong. [...] Well, that means *hind*sight is still 20/20: we see where we went wrong, but *fore*sight is still blurry, because what we think is the solution to that wrong may not turn out to be a good solution later. :-D T -- Question authority. Don't ask why, just do it.
Re: Article: Why Const Sucks
On Tuesday, March 06, 2018 09:41:42 H. S. Teoh via Digitalmars-d-announce wrote: > As they say, hindsight is always 20/20. But it wasn't so easy to > foresee the consequences at the time when the very concept of ranges was > still brand new. Except that even worse, I'd argue that hindsight really isn't 20/20. We can see a lot of the mistakes that were made, and if we were starting from scratch or otherwise willing to break a lot of code, we could change stuff like the range API based on the lessons learned. But we'd probably still screw it up, because we wouldn't have the experience with the new API to know where it was wrong. Consider all of the stuff that was improved in D over C++ but which still has problems in D (like const). We build on experience to make the new stuff better and frequently lament that we didn't know better in the past, but we still make mistakes when we do new stuff or redesign old stuff. Frequently, the end result is better, but it's rarely perfect. - Jonathan M Davis
Re: Article: Why Const Sucks
On Tuesday, March 06, 2018 09:36:43 H. S. Teoh via Digitalmars-d-announce wrote: > Andrei has said before, and probably on more than one occasion, that if > he were to redesign ranges today, one of the things he would do > differently was to change the definition of forward range so that .save > is basically implicit on copying the range object, and non-forward input > ranges would just be reference / non-copyable types. > > But that boat has long sailed, and we just have to make do with what we > have today. Changing this now will literally break just about *every* D > program that uses ranges, which is breakage of an ecosystem-killing > magnitude that I can't even contemplate. I would much rather go with a > less intrusive breakage like killing autodecoding with fire, than with > something that will basically require me to rewrite practically every D > program I ever wrote. I'm not actually convinced that killing auto-decoding is really much better. As it stands, changing it would break a large percentage of string-based code, and the functions in question sit in std.range.primitives along with all of the other core range stuff such that I don't see how we can change them any more than we can change the basic range API. I would love to be proven wrong, but I don't know how we could change it at this point without code breakage that comes pretty close to the breakage that changing the range API would cause. - Jonathan M Davis
Re: Article: Why Const Sucks
On Mon, Mar 05, 2018 at 10:21:47PM -0500, Nick Sabalausky (Abscissa) via Digitalmars-d-announce wrote: > On 03/05/2018 12:38 PM, H. S. Teoh wrote: > > > > This broke the by-value assumption inherent in much of Phobos code, > > Wait, seriously? Phobos frequently passes ranges by value? I sincerely > hope that's only true for class-based ranges and forward-ranges (and > more specifically, only forward ranges where copying the range and > calling .save are designed to do the exact same thing). Otherwise, > that's really, *REALLY* bad since non-forward ranges *by definition* > cannot be duplicated. I think you misunderstood. :-D Passing ranges by value means passing the range itself, usually a struct, which is a value type. I did *not* say the *content* of ranges are *copied* -- that would be so horribly wrong that I would be thinking twice about using D for my projects. :-D [...] > The definition of "what is a forward/non-forward range" for > struct-based ranges should have been "is this() @disabled (non-forward > range), or is this() enabled *and* does the same thing as .save > (forward range)?" [...] Yeah, Andrei has admitted before that this is probably what he would do today, if he were given a second chance to design ranges. But at the time, the landscape of D was rather different, and certain language features didn't exist yet (sorry, can't recall exactly which off the top of my head), so he settled with the compromise that we have today. As they say, hindsight is always 20/20. But it wasn't so easy to foresee the consequences at the time when the very concept of ranges was still brand new. T -- What do you get if you drop a piano down a mineshaft? A flat minor.
Re: Article: Why Const Sucks
On Tue, Mar 06, 2018 at 08:39:41AM -0700, Jonathan M Davis via Digitalmars-d-announce wrote: [...] > Yeah. If you're dealing with generic code rather than a specific range > type that you know is implicitly saved when copied, you have to use > save so often that it's painful, and almost no one does it. e.g. > > equal(lhs.save, rhs.save) > > or > > immutable result = range.save.startsWith(needle.save); > > How well Phobos has done with this has improved over time as more and > better testing has been added (testing for reference type ranges is > probably the most critical to finding this particular problem), but I > doubt that Phobos has it right everywhere, and I'm sure that the > average programmer's code has tons of these problems. In my own code, I often run into subtle bugs that arise from ranges being unintentionally consumed because I forgot to call .save. So I tend to be extra careful about this. But yeah, it's so easy to miss unless your code actually uses ranges where it would make a difference. [...] > Ranges are wonderfully powerful, but they become a royal pain to get > right with truly generic code. And that's without getting into all of > the arguments about whether stuff like whether transitive fronts > should be allowed... I know we have disagreed on this before, but in my mind, it's very simple. Generic code should basically be written in such a way that it does the most making the least assumptions. Meaning, don't assume the return value of .front persists beyond the next .popFront, don't assume iterating the range won't consume it, etc.. It's just basic defensive programming. If the algorithm won't work without some of these assumptions, then make them explicit, either a part of the API, or clearly documented. IMNSHO, code that isn't written this way is just sloppy and a haven for hidden bugs. > Ranges are definitely one area where we could really use some redesign > to iron out some of the issues that we've found over time, but their > success makes them almost impossible to fix, because changing them > would break tons of code. But annoyingly, that's often what happens > when you implement a new idea. You simply don't have enough knowledge > about it ahead of time to avoid mistakes; those are easy enough to > make when you really know what you're doing, let alone with something > new. [...] Andrei has said before, and probably on more than one occasion, that if he were to redesign ranges today, one of the things he would do differently was to change the definition of forward range so that .save is basically implicit on copying the range object, and non-forward input ranges would just be reference / non-copyable types. But that boat has long sailed, and we just have to make do with what we have today. Changing this now will literally break just about *every* D program that uses ranges, which is breakage of an ecosystem-killing magnitude that I can't even contemplate. I would much rather go with a less intrusive breakage like killing autodecoding with fire, than with something that will basically require me to rewrite practically every D program I ever wrote. T -- "You know, maybe we don't *need* enemies." "Yeah, best friends are about all I can take." -- Calvin & Hobbes
Re: Article: Why Const Sucks
On Tuesday, 6 March 2018 at 15:39:41 UTC, Jonathan M Davis wrote: [snip] How well Phobos has done with this has improved over time as more and better testing has been added (testing for reference type ranges is probably the most critical to finding this particular problem), but I doubt that Phobos has it right everywhere, and I'm sure that the average programmer's code has tons of these problems. Code mostly just works because most code really isn't used with arbitrary ranges, and a large percentage of ranges implicitly save on copy. It's not that hard to get a piece of code working with a particular range or just a few similar range types. It's when it needs to work with _any_ range type that matches the template constraint that things start getting hairy, and without thorough testing, it simply doesn't happen unless the code is very simple and the programmer in question is very mindful of stuff like save (which most programmers aren't). And of course, since most testing is done with dynamic arrays, issues with other range types simply aren't found unless the programmer is really putting in the effort to do their due diligence. Someone could make a range testing library of sorts. Really just a variant packed with a bunch of different range types covering a variety of use cases. The user could import it in a version(unittest) block and then just loop through it and assert that the function works for all of them (would need some kind of variant full of what to compare it to for each type). Ranges are wonderfully powerful, but they become a royal pain to get right with truly generic code. And that's without getting into all of the arguments about whether stuff like whether transitive fronts should be allowed... Ranges are definitely one area where we could really use some redesign to iron out some of the issues that we've found over time, but their success makes them almost impossible to fix, because changing them would break tons of code. But annoyingly, that's often what happens when you implement a new idea. You simply don't have enough knowledge about it ahead of time to avoid mistakes; those are easy enough to make when you really know what you're doing, let alone with something new. - Jonathan M Davis There's probably value in writing up a retrospective of sorts on D's ranges. What works about the design, what are the limitations, and what could be improved (depending on the level of breakage considered acceptable). Even if D takes a long time to fix these issues, making the information more easily available to others outside the D community might be valuable.
Re: Article: Why Const Sucks
On Tuesday, March 06, 2018 15:23:52 Adam D. Ruppe via Digitalmars-d-announce wrote: > On Tuesday, 6 March 2018 at 03:21:47 UTC, Nick Sabalausky > > (Abscissa) wrote: > > Wait, seriously? Phobos frequently passes ranges by value? > > You *should* pass most ranges by value, just like how you should > rarely use `ref T[]` or `T[]*`. Ranges, like slices, are > typically already small references to some other container. > > Where Phobos effs it up is not follow its own rules on range.save > in most cases... Jonathan talked about this at dconf IIRC in 2015. Yeah. If you're dealing with generic code rather than a specific range type that you know is implicitly saved when copied, you have to use save so often that it's painful, and almost no one does it. e.g. equal(lhs.save, rhs.save) or immutable result = range.save.startsWith(needle.save); How well Phobos has done with this has improved over time as more and better testing has been added (testing for reference type ranges is probably the most critical to finding this particular problem), but I doubt that Phobos has it right everywhere, and I'm sure that the average programmer's code has tons of these problems. Code mostly just works because most code really isn't used with arbitrary ranges, and a large percentage of ranges implicitly save on copy. It's not that hard to get a piece of code working with a particular range or just a few similar range types. It's when it needs to work with _any_ range type that matches the template constraint that things start getting hairy, and without thorough testing, it simply doesn't happen unless the code is very simple and the programmer in question is very mindful of stuff like save (which most programmers aren't). And of course, since most testing is done with dynamic arrays, issues with other range types simply aren't found unless the programmer is really putting in the effort to do their due diligence. Ranges are wonderfully powerful, but they become a royal pain to get right with truly generic code. And that's without getting into all of the arguments about whether stuff like whether transitive fronts should be allowed... Ranges are definitely one area where we could really use some redesign to iron out some of the issues that we've found over time, but their success makes them almost impossible to fix, because changing them would break tons of code. But annoyingly, that's often what happens when you implement a new idea. You simply don't have enough knowledge about it ahead of time to avoid mistakes; those are easy enough to make when you really know what you're doing, let alone with something new. - Jonathan M Davis
Re: Article: Why Const Sucks
On Tuesday, 6 March 2018 at 03:21:47 UTC, Nick Sabalausky (Abscissa) wrote: Wait, seriously? Phobos frequently passes ranges by value? You *should* pass most ranges by value, just like how you should rarely use `ref T[]` or `T[]*`. Ranges, like slices, are typically already small references to some other container. Where Phobos effs it up is not follow its own rules on range.save in most cases... Jonathan talked about this at dconf IIRC in 2015. The definition of "what is a forward/non-forward range" for struct-based ranges should have been "is this() @disabled (non-forward range), or is this() enabled *and* does the same thing as .save (forward range)?" yeah.
Re: Article: Why Const Sucks
On Tuesday, 6 March 2018 at 10:02:10 UTC, Radu wrote: On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: http://jmdavisprog.com/articles/why-const-sucks.html I suppose that it's not exactly the most positive article, but I feel that it's accurate. - Jonathan M Davis Spot on article, and touches some of my pain points when working with const/immutable structs. Recently I tried to create a ref-counted immutable struct, oh boi... This later use case is of tremendous value for safe concurrent code that's @nogc. Unfortunately I couldn't find a way to make it work efficiently and in the same time not look like a disgusting hack. I suspect a possible solution is to allow immutable(const) postblit overloads as well as immutable dtors that will act as an escape hatch for unsafe work and in the same time provide hints that you are operating on an immutable(const) this. AFAIK this is a solved problem. Dicebot (not sure if he's still around) had initially proposed it: store the reference count just before the immutable piece of memory. | ref count |object/struct| In fact, Andrei added a new allocator to std.experimental.allocator to support this: https://dlang.org/library/std/experimental/allocator/building_blocks/affix_allocator/affix_allocator.prefix.html
Re: Article: Why Const Sucks
On Tuesday, 6 March 2018 at 12:05:26 UTC, Dukc wrote: I think we have a bug here. I believe postblits should behave like constructors in both events. https://issues.dlang.org/show_bug.cgi?id=18561
Re: Article: Why Const Sucks
On 3/6/18 1:49 AM, Jonathan M Davis wrote: Regardless, I doubt that the design of ranges is going to be changed at this point given the amount of code that would break as a result, and these sort of changes are not backwards compatible. I sometimes think we would be better off to drop InputRange, and base everything on the assumption that it can be copied (where isInputRange is renamed to isForwardRange, and `save` goes away). Then you could use @disable postblit to mimic what InputRange would have been. I've never seen the point of having classes be ranges. -Steve
Re: Article: Why Const Sucks
On Tuesday, 6 March 2018 at 11:03:24 UTC, Nemanja Boric wrote: title = title.dup;// doesn't work anymore Strange! You're right it does not when the type declares a member as const, yet: On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote (in the article): And if an object is const or immutable, then that's all of the members. ...does not prevent the postblit from working whenthe object is declared const by user. I think we have a bug here. I believe postblits should behave like constructors in both events.
Re: Article: Why Const Sucks
On Tuesday, 6 March 2018 at 10:49:48 UTC, Dukc wrote: On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote (in the article): The problem is that the entire object must be fully initialized before the body of the postblit constructor is run. That means that any member variables which are const or immutable are stuck at whatever they were in the original object, because it would violate the type system to mutate them. And if an object is const or immutable, then that's all of the members. I think we have a misunderstanding here. According to that, this would not compile (imports left out): struct placeAtWorldMap { char[] title; int[2] coordsMicroDeg; this(this) { title = title.dup; } } void main() { char[] title = "London bridge".dup; const place = placeAtWorldMap(title, [51_508_038, -87_693]); const samePlace = place; "falling down ".copy(title); place.title.writeln; // falling down samePlace.title.writeln; // London bridge readln; } ...but it compiles and correctly runs, and I'm happy about that. Quote from the article: . That means that any member variables which are const or immutable are stuck at whatever they were in the original object, because it would violate the type system to mutate them Meaning that if you declare your `title` member `const char[] title`, you can't change it in the postblit, but you could set it in the constructor. ``` import std.array; import std.stdio; import std.algorithm; struct placeAtWorldMap { const char[] title; int[2] coordsMicroDeg; this(char[] name) { this.title = name.idup; // you can assign const members here } this(char[] name, int[2] coords) { this.title = name; this.coordsMicroDeg = coords; } this(this) {// title = title.dup;// doesn't work anymore } } void main() { char[] title = "London bridge".dup; const place = placeAtWorldMap(title, [51_508_038, -87_693]); const newPlace = placeAtWorldMap("Big Ben".dup); const samePlace = place; "falling down ".copy(title); place.title.writeln; // falling down samePlace.title.writeln; // London bridge newPlace.title.writeln; // Big Ben } ```
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote (in the article): The problem is that the entire object must be fully initialized before the body of the postblit constructor is run. That means that any member variables which are const or immutable are stuck at whatever they were in the original object, because it would violate the type system to mutate them. And if an object is const or immutable, then that's all of the members. I think we have a misunderstanding here. According to that, this would not compile (imports left out): struct placeAtWorldMap { char[] title; int[2] coordsMicroDeg; this(this) { title = title.dup; } } void main() { char[] title = "London bridge".dup; const place = placeAtWorldMap(title, [51_508_038, -87_693]); const samePlace = place; "falling down ".copy(title); place.title.writeln; // falling down samePlace.title.writeln; // London bridge readln; } ...but it compiles and correctly runs, and I'm happy about that.
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: http://jmdavisprog.com/articles/why-const-sucks.html I suppose that it's not exactly the most positive article, but I feel that it's accurate. - Jonathan M Davis Spot on article, and touches some of my pain points when working with const/immutable structs. Recently I tried to create a ref-counted immutable struct, oh boi... This later use case is of tremendous value for safe concurrent code that's @nogc. Unfortunately I couldn't find a way to make it work efficiently and in the same time not look like a disgusting hack. I suspect a possible solution is to allow immutable(const) postblit overloads as well as immutable dtors that will act as an escape hatch for unsafe work and in the same time provide hints that you are operating on an immutable(const) this.
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 13:59:02 UTC, Adam D. Ruppe wrote: On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: And then, of course, like you said "don't use it" is the solution to most of the const system's flaws anyway which seems to be the case with a lot of D's add-on qualifiers. +1 The worst part of those add-on qualifiers is that the official gospel is that they are useful when in reality you HAVE to avoid them to be productive.
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: http://jmdavisprog.com/articles/why-const-sucks.html I suppose that it's not exactly the most positive article, but I feel that it's accurate. - Jonathan M Davis Its amazing how typed languages, in this case const, forces you to think in mutation but with an exception. I wonder the real world SOLE benefit of using const compared to mutable-immutable style. Its either mutable or immutable. Exception could only available by casting immutable to mutable. I feels this const thing is a theoretical problem. One can overcome practically with code style discipline. At a point, const becomes just a decoration; something people use prematurely (YAGNI). It must feel like sleeping on your bed with sharp knives hanging at the top such that they can fall on you anything: when comparing code in a typed language to an untyped one. That's a potential cause of premature use of certain type attributes. By the way, Jonathan, you should really consider writing a book with such deep insights your have. Especially the way you explain things.
Re: Article: Why Const Sucks
On Monday, March 05, 2018 22:21:47 Nick Sabalausky via Digitalmars-d- announce wrote: > On 03/05/2018 12:38 PM, H. S. Teoh wrote: > > This broke the by-value > > assumption inherent in much of Phobos code, > > Wait, seriously? Phobos frequently passes ranges by value? I sincerely > hope that's only true for class-based ranges and forward-ranges (and > more specifically, only forward ranges where copying the range and > calling .save are designed to do the exact same thing). Otherwise, > that's really, *REALLY* bad since non-forward ranges *by definition* > cannot be duplicated. > > Honestly, I think this is the one big flaw in the otherwise really nice > design of ranges. > > The definition of "what is a forward/non-forward range" for struct-based > ranges should have been "is this() @disabled (non-forward range), or is > this() enabled *and* does the same thing as .save (forward range)?" > > Without that, this is a serious hole in non-forward ranges. Passing ranges around by value is fine so long as you don't use the original after the copy is made. Where you get screwed is when you then use the original after the copy has been made. Almost nothing in Phobos passed ranges around by ref, and doing so would actually make it a royal pain to iteract with forward ranges, because save obviously isn't an lvalue, and range-based functions that return new ranges aren't returning lvalues. So, if range-based functions took their arguments by ref, then you couldn't chain them. And using auto ref wouldn't fix the problem, since you could still pass by value. It would just introduce all kinds of inconsistent behavior as to whether a range was copied or not depending on how exactly a range-based function were called, causing more bugs. Honestly, I think that the correct way to implement forward ranges would have been to disallowing ranges that weren't dynamic arrays or structs and then use postblit constructors instead of save (classes could then be used as ranges by wrapping them in structs, though even then, it would be better to avoid classes as ranges, because all of those calls to new get to be inefficent). With that, you wouldn't have all of these problems with accidentally saving or not. Any time a forward range was copied, it would be saved automatically, unless it could be moved, in which case, saving wasn't necessary. Unfortunately, that still leaves the problem of basic input ranges, since they wouldn't have postblit constructors, and they could still be defined as pseudo-reference types. Maybe we could require them to be defined as classes to force them to be full-on reference types (they obviously can't be value types, or they could be forward ranges), but then that would force allocations for basic input ranges. _Most_ ranges can be at least forward ranges, but some stuff can't reasonably be, and you wouldn't want to have to allocate all of those on the heap. So, I don't have a clean solution for how to deal with basic input ranges and copying, though I haven't sat down recently and tried to work through the problem. In principle though, they're reference types and ideally would be treated as such. Regardless, I doubt that the design of ranges is going to be changed at this point given the amount of code that would break as a result, and these sort of changes are not backwards compatible. - Jonathan M Davis
Re: Article: Why Const Sucks
On 03/05/2018 12:38 PM, H. S. Teoh wrote: This broke the by-value assumption inherent in much of Phobos code, Wait, seriously? Phobos frequently passes ranges by value? I sincerely hope that's only true for class-based ranges and forward-ranges (and more specifically, only forward ranges where copying the range and calling .save are designed to do the exact same thing). Otherwise, that's really, *REALLY* bad since non-forward ranges *by definition* cannot be duplicated. Honestly, I think this is the one big flaw in the otherwise really nice design of ranges. The definition of "what is a forward/non-forward range" for struct-based ranges should have been "is this() @disabled (non-forward range), or is this() enabled *and* does the same thing as .save (forward range)?" Without that, this is a serious hole in non-forward ranges.
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 13:48:23 UTC, Adam D. Ruppe wrote: Just a semantic note, it is "straitjacket". "straight" is like a non-wiggly line. "strait" means narrow or constricted. Thus, the straitjacket is a jacket that constricts your movement. Of course, using "straight" is such a common mistake it has become generally accepted... but still, I like being precise with my words. Programmers like precision, don't we! From Simon Tatham article about how to report bugs effectively: https://www.chiark.greenend.org.uk/~sgtatham/bugs.html "Above all, *be precise*. Programmers like precision."
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 13:48:23 UTC, Adam D. Ruppe wrote: Just a semantic note, it is "straitjacket". "straight" is like a non-wiggly line. "strait" means narrow or constricted. Thus, the straitjacket is a jacket that constricts your movement. Of course, using "straight" is such a common mistake it has become generally accepted... but still, I like being precise with my words. You guys are a bunch of nerds . . . . - Me
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 13:48:23 UTC, Adam D. Ruppe wrote: Just a semantic note, it is "straitjacket". "straight" is like a non-wiggly line. "strait" means narrow or constricted. Thus, the straitjacket is a jacket that constricts your movement. Of course, using "straight" is such a common mistake it has become generally accepted... but still, I like being precise with my words. You guys are a bunch of nerds . . . .
Re: Article: Why Const Sucks
On Mon, Mar 05, 2018 at 11:04:49AM -0700, Jonathan M Davis via Digitalmars-d-announce wrote: > On Monday, March 05, 2018 09:38:52 H. S. Teoh via Digitalmars-d-announce > wrote: > > Eventually, I discovered that the underlying problem was that > > Result, as defined above, was a struct with a const member, and > > therefore it was illegal to assign it to a variable of the same type > > outside of initialization (since doing do meant you were overwriting > > a const field with something else, which violates the constness of > > the field). This broke the by-value assumption inherent in much of > > Phobos code, so the resulting range ended being unusable with most > > Phobos algorithms. Which defeated the whole purpose in the first > > place. > > Honestly, I've come to the conclusion that structs should never have > const or immutable members. It just causes too many problems. Treating > them as read-only from the outside by having them be private and have > member functions be const is fine (assuming that const works in that > case), and having them work when the entire object gets marked as > const is great (assuming that const works in that case), but I think > that it's pretty much always a mistake to make individual member > variables of a struct const or immutable. [...] Yeah, but in this case, since `this` is const, there's simply no way to get around the fact that there must be `const` somewhere in the Result struct. The D compiler will not accept a mutable member referencing `this` that has a const access method, since that in theory breaks the const guarantee. I suppose replacing `const(Container)` with tail-const references to Container's innards would fix the problem, but it would uglify the code too much and would be far too much effort just to be able to say "we support const", that it's simply not worth it. Also, structs with const/immutable members are a rare case allowed by the language but almost never tested for in Phobos, so you can pretty much expect random things to break left, right, and center if you ever attempt to use such a struct with Phobos functions. In fact, I vaguely remember that even the compiler may have bugs / strange behaviours if you try to use such structs in non-trivial ways. T -- Never step over a puddle, always step around it. Chances are that whatever made it is still dripping.
Re: Article: Why Const Sucks
On Monday, March 05, 2018 09:38:52 H. S. Teoh via Digitalmars-d-announce wrote: > Eventually, I discovered > that the underlying problem was that Result, as defined above, was a > struct with a const member, and therefore it was illegal to assign it to > a variable of the same type outside of initialization (since doing do > meant you were overwriting a const field with something else, which > violates the constness of the field). This broke the by-value > assumption inherent in much of Phobos code, so the resulting range ended > being unusable with most Phobos algorithms. Which defeated the whole > purpose in the first place. Honestly, I've come to the conclusion that structs should never have const or immutable members. It just causes too many problems. Treating them as read-only from the outside by having them be private and have member functions be const is fine (assuming that const works in that case), and having them work when the entire object gets marked as const is great (assuming that const works in that case), but I think that it's pretty much always a mistake to make individual member variables of a struct const or immutable. Classes don't have the same problem, because they're on the heap and don't get copied, but with structs being on the stack and very much being designed with copying in mind, members that can't be copied becomes a definite problem. Tail-const and tail-immutable would work fine with member variables in structs, but that basically means that the data for those members has to be on the heap, which isn't always a reasonable option. > So yeah, while D's const provides actual guarantees unlike C++'s > laughable const-by-documentation, that also limits its scope so much > that in practice, it's rarely ever used outside of built-in types like > string. Which also limits the usefulness of its guarantees so much that > it's questionable whether it's actually worth the effort. Exactly. - Jonathan M Davis
Re: Article: Why Const Sucks
On Monday, March 05, 2018 17:35:28 ShadoLight via Digitalmars-d-announce wrote: > Very interesting and well written! Jonathan, your experiences > with const in C++/Java just about matches my experiences with it. > I also feel that the situation in D is less than ideal in this > regard. > > First, a small (for sure copy-pasta) typo in your article: > > const(int[]) arr1 = getArray(); > const(int)[] arr2 = p1; //Sure you meant arr2 = arr1; Thanks. Fixed. > Regarding the proposed solution to the issues with Object namely > "the solution to this problem which was agreed to a few years ago > was to remove opEquals, opCmp, toHash, and toString from Object > so that derived classes can define them with whatever attributes > are desired" which, you say, will never happen because of too > much code breakage... > > What about the following? Currently the compiler is smart enough, > so you can define class Foo and explicitly inherit from Object if > you prefer: ... > Why was something like this not considered (you don't give a > link, so I cannot investigate), rather than simply removing them > from Object? Can this be exploited to, in effect, create the same > opportunity but minus the breakage? Or am I missing something? As I mentioned, there was recently some talk about creating a DIP to add a new root object below Object. If that's done, presumably, Object will still be the default to avoid code breakage, but it would be provide essentially what you're talking about. However, such a DIP still has to be written, so anything at this point is speculation as to what it's going to look like. At the time that it was decided to remove the functions from Object, it was more reasonable than it would be now (since D is definitely older now with a larger user base), and the details of how it would be done were never fully decided, which is part of why it's never come to fruition in spite of it being clear that we needed a root object without those functions. - Jonathan M Davis
Re: Article: Why Const Sucks
Very interesting and well written! Jonathan, your experiences with const in C++/Java just about matches my experiences with it. I also feel that the situation in D is less than ideal in this regard. First, a small (for sure copy-pasta) typo in your article: const(int[]) arr1 = getArray(); const(int)[] arr2 = p1; //Sure you meant arr2 = arr1; Regarding the proposed solution to the issues with Object namely "the solution to this problem which was agreed to a few years ago was to remove opEquals, opCmp, toHash, and toString from Object so that derived classes can define them with whatever attributes are desired" which, you say, will never happen because of too much code breakage... What about the following? Currently the compiler is smart enough, so you can define class Foo and explicitly inherit from Object if you prefer: class Foo : Object {..} //OK, no recursive Object inherits Object ad-infinitum. Would it not be easier to exploit this and go the "opposite" way i.e. rather than remove it, just extend the hierarchy by adding a base class to Object itself (in object.d no less), something like this: class ObjectBase { interface Monitor{..} static ObjectBase factory(string classname){..} } class Object : ObjectBase { string toString(){..} size_t toHash() @trusted nothrow{..} int opCmp(Object o){..} bool opEquals(Object o){..} static Object factory(string classname){..} } Now, if you do: class Foo{..} //OK. Still inherits from Object i.e. identical to 'class Foo : Object {..} class Foo : Object {..} //The same, just explicit //On the other hand: class Bar : ObjectBase //Deliberately bypass inheritance from Object. Inheritance from ObjectBase will have to be explicit (the price to pay for non-breakage!), but now class Bar is free to implement opEquals, opCmp, toHash, etc as it sees fit. This still guarantees back-wards compatibility since all classes currently inherited from Object have exactly the same semantics as they do today. No upgrades to any current projects/code required! Why was something like this not considered (you don't give a link, so I cannot investigate), rather than simply removing them from Object? Can this be exploited to, in effect, create the same opportunity but minus the breakage? Or am I missing something? Actually, in a more general sense I have begun to wonder if the common point of departure in D-land, that the concept of tail-const is actually a "part of" the concept of const, is not maybe wrong. We typically describe the concepts of tail/head-const "in terms of" const i.e. const in D 'equals' head-const 'plus' tail-const [1]. Since the concepts of tail-const and head-const are not that well known (outside of mostly C++-land), these 2 concepts are often utilized to differentiate D's concept of const, and explain it relative to the const (or final/sealed/readonly/etc) found in other languages. Maybe that muddles the water, and the 3 concepts, even though related, can semantically exist separately as well! I am of the opinion that we really need something like tail-const in D - particularly to semantically mark that the "payload" in a range is constant, but that the range itself can mutate. But it invariably falls apart when you try to shoehorn it into the general concept of const-ness in D with its transitivity properties, etc. The 2 concepts just don't gel, and I think the only way this can be accomplished is to have a separate semantic construct and syntax for tail-const. I think the 2 use cases are mostly orthogonal, and only overlapping in a narrow sense - but it this narrow sense that is used to explain it!. And, yes, the tail-const version will have less guarantees and none of the optimization opportunities that the current const version can promise. Of course I realize the changes of this being added to D is practically zero; So, yes, i have to concur with your conclusions as well! PS. BTW, your blog's title "The Long-Winded D Guy" is quite brilliant. You are indeed a bit long-winded, but you also take the time to explain things very thoroughly. It is a trade-off and I think in your case the cost-benefit ratio is positive. Thank you for that! Also, if you ever write a book (it might have to come in multiple volumes ;-), you can already count on having 1 buyer! [1] https://dlang.org/articles/const-faq.html#const
Re: Article: Why Const Sucks
On Mon, Mar 05, 2018 at 03:57:35AM -0700, Jonathan M Davis via Digitalmars-d-announce wrote: > Here's something I wrote up on const: > > http://jmdavisprog.com/articles/why-const-sucks.html > > I suppose that it's not exactly the most positive article, but I feel > that it's accurate. [...] Yeah, I found myself in the same boat recently. As you often say, const and ranges simply don't mix. And modern idiomatic D is 90% about ranges, so that alone instantly reduces the scope of const's usefulness by a lot. A case in point: I was implementing a container recently, and wrote an opSlice() method to return a range over its elements. My initial thought was that it could be made const, since the range wouldn't mutate the underlying elements, but only represent a mutable slice over a const container, i.e., the slice can mutate, but the elements cannot, just like iterating over string (== immutable(char)[]). Should be easy, right? struct Container { auto opSlice() const { static struct Result { private Container impl; private int n; // internal mutable state @property bool empty() { ... } ... // rest of range API } return Result(this); } } Well, that didn't work, because in opSlice, `this` is const, and I can't initialize Result.impl which is a mutable Container. Well, no biggie, just make it const: struct Container { auto opSlice() const { static struct Result { private const(Container) impl; private int n; // internal mutable state @property bool empty() { ... } ... // rest of range API } return Result(this); } } At first, this worked, and it seems that I could have my const cake and eat it too. Until I decided at some point that I needed to make it a forward range, which requires a .save method. So I tried: ... static struct Result { private const(Container) impl; ... @property Result save() { Result copy = this; return this; } } That seemed to do the trick, everything compiles and works. But I soon ran into an unfixable problem: the resulting range, while it works in the simplest cases, started causing mysterious compile errors when I tried to use it with Phobos range algorithms. Eventually, I discovered that the underlying problem was that Result, as defined above, was a struct with a const member, and therefore it was illegal to assign it to a variable of the same type outside of initialization (since doing do meant you were overwriting a const field with something else, which violates the constness of the field). This broke the by-value assumption inherent in much of Phobos code, so the resulting range ended being unusable with most Phobos algorithms. Which defeated the whole purpose in the first place. While I'm sure with enough time and patience Phobos could be fixed to support this kind of range, the decision I was faced with was: should I (1) persist in using const, and thereby stall my project while I work on huge chunks of Phobos range algorithms to make them usable with ranges that have const members (not to mention spending how much time waiting in the Phobos PR queue and potentially getting things rejected because it might cause breakage of unknown amounts of existing code), or (2) just remove `const` from my code, and be able to continue with my project *right now*? The choice was a no-brainer, sad to say. So yeah, while D's const provides actual guarantees unlike C++'s laughable const-by-documentation, that also limits its scope so much that in practice, it's rarely ever used outside of built-in types like string. Which also limits the usefulness of its guarantees so much that it's questionable whether it's actually worth the effort. T -- It won't be covered in the book. The source code has to be useful for something, after all. -- Larry Wall
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: http://jmdavisprog.com/articles/why-const-sucks.html I suppose that it's not exactly the most positive article, but I feel that it's accurate. - Jonathan M Davis Interesting read and it explains some of the incomprehensible error messages. Basically you're saying we've got a strong const in D but essentially the situation is much like in Java because the general approach is don't use it ? Kind of ironic. I made a Logger module a couple years back and I used const very generously. Then came the moment where the LogWriters actually had to do the writing. One would assume that a function called 'stdio.write' mutates some state, but it sure as hell doesn't affect the state of my logger. DMD said: So swy! writeln not const-y, no can do, swy! So I remove const. One function at a time and end up with a module where basically no const remains. But it doesn't stop there. Since logger isn't const anymore, it now can't be used in any const functions. So more de-const-ification happened... Also, it's frustrating to have to get rid of const because Object.toString uses some Outputwriter thing which is not const and therefore transitively you can't have const anywhere else. Kind of a disappointing experience. As far as I'm concerned, it's not so much "don't use const; or, it's not worth it" but more like "I would if I could but I can't so I shan't". I bother with const as much as possible but it's incredible frustrating and I would argue the time lost to de-const-ify APIs is easily equivalent to a code breaking change that would make const more applicable. Something possimpible. Like a compiler that doesn't purely make decisions upon a const keyword, but one that can understand that interacting with some random function with only "in" parameters or objects, which aren't even interacting with,or are part of, the object's internal state and are just passed through, can't violate the const-ness of the object. But there's likely much more to consider than that and I couldn't ever dream of implementing such a thing..unfortunately
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: http://jmdavisprog.com/articles/why-const-sucks.html I suppose that it's not exactly the most positive article, but I feel that it's accurate. - Jonathan M Davis Great read for a Monday.
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: So as to the main thrust, I generally agree. In fact, I think const is almost useless even if you want to use it fully: you said immutable is better in many places, and yes, but in addition to that, inout is better than const in most the remaining cases. Constructing a const variable? Almost completely useless - immutable is better in almost all (if not actually all) cases. Referencing function parameters? inout is better in all cases except just plain input. If there's any part of it being returned - as is the case on most member methods to me at least - inout is flat-out better. And then, of course, like you said "don't use it" is the solution to most of the const system's flaws anyway which seems to be the case with a lot of D's add-on qualifiers.
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: Excellent read! I enjoyed the history of proposed solutions. I've run into the trouble with annotated opEquals in classes several times. I believe an empty/nonexistant Object class is the correct solution, together with templates. If it breaks tricky opEquals-overriding usercode, then so be it. The const trouble is merely a symptom here, the real problem is Object. In structs, all const members feel alien, they preventing all copying. This seems far harder to solve. -- Simon
Re: Article: Why Const Sucks
On Monday, March 05, 2018 11:38:05 Atila Neves via Digitalmars-d-announce wrote: > I used to use `immutable`, but gradually came around to only > using it if I have to send data to another thread, otherwise it's > too much of a hassle. Aside from the whole std.concurrency situation, I generally use immutable in the places where the semantics are the same as const, so it doesn't generally cause a problem for me. I just prefer immutable in the cases where it and const are the same, because immutable clearly indicates that the value never changes, so immutable more closely fits the idea even if const happens to mean that in that particular case. I agree that using immutable where you have to do stuff like design objects in a particular way in order to make immutable work is generally too much of a pain to be worth it, but at least in those cases, the data can then be shared across threads, and it's data that doesn't need mutable backdoors, because it only makes sense to use immutable in that fashion when the object is really designed to never change state. I think that the only time that I've programmed heavily in a way that involves immutability everywhere is when I programmed in Haskell, and that combined with the fact that Haskell is lazy and completely functional such that you can't use imperative idioms anywhere made it so that each line of Haskell code took me more time to write than is the case with any other language I've used (except maybe batch, because of how insanely error-prone it is). I think that I'm a better programmer for it, but I'd hate to program that way normally, and using immutable all over the place would head too heavily in that direction - though at least D, unlike Haskell, is a multi-paradigm language and allows you to choose to use a particular idiom when you think it makes the most sense instead of forcing it everywhere. - Jonathan M Davis
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: http://jmdavisprog.com/articles/why-const-sucks.html I suppose that it's not exactly the most positive article, but I feel that it's accurate. - Jonathan M Davis Brilliant article, Johathan! I feel the same...
Re: Article: Why Const Sucks
On Monday, 5 March 2018 at 10:57:35 UTC, Jonathan M Davis wrote: Here's something I wrote up on const: http://jmdavisprog.com/articles/why-const-sucks.html I suppose that it's not exactly the most positive article, but I feel that it's accurate. - Jonathan M Davis My biggest issues with const are, as you wrote, ranges and postBlit. I still use `const` everywhere unless I can't. I used to use `immutable`, but gradually came around to only using it if I have to send data to another thread, otherwise it's too much of a hassle. Atila
Article: Why Const Sucks
Here's something I wrote up on const: http://jmdavisprog.com/articles/why-const-sucks.html I suppose that it's not exactly the most positive article, but I feel that it's accurate. - Jonathan M Davis