DIP74 - where is at?
So I have another upcoming opportunity to introduce D in my workplace, this time as a front-end/plugin language to our C++ infrastructure, which is promising since I already have considerable experience in this area (my work at Remedy with Quantum Break), and there is a lot of recent work to interact better with C++, which we will stress-test extensively. You only get so many shots at this; but this is a particularly promising opportunity, since the C++ code is a nightmare, and the contrast against D will allow a lot of coders to see the advantage. There is however one critical missing feature, DIP74... where is it at currently? How is it going? Is it likely to be accepted in the near-term? Some sort of approximate timeline? I think it would be a mistake for me to introduce this without DIP74, since we will rely on it VERY heavily, and the machinery to work-around it will start to look just as heavy-weight as the C++ code I'm trying to deprecate... but then waiting on it starts to look like missing the window of opportunity. Thoughts?
Re: DIP74 - where is at?
On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: So I have another upcoming opportunity to introduce D in my workplace, this time as a front-end/plugin language to our C++ infrastructure, which is promising since I already have considerable experience in this area (my work at Remedy with Quantum Break), and there is a lot of recent work to interact better with C++, which we will stress-test extensively. You only get so many shots at this; but this is a particularly promising opportunity, since the C++ code is a nightmare, and the contrast against D will allow a lot of coders to see the advantage. There is however one critical missing feature, DIP74... where is it at currently? How is it going? Is it likely to be accepted in the near-term? Some sort of approximate timeline? I think it would be a mistake for me to introduce this without DIP74, since we will rely on it VERY heavily, and the machinery to work-around it will start to look just as heavy-weight as the C++ code I'm trying to deprecate... but then waiting on it starts to look like missing the window of opportunity. Thoughts? It doesn't looks like it is getting implemented. And, to be honest, I'd rather go a principle approach + library support rather than a pie of hacks. The pile of hacks approach is what made C++ C++.
Re: DIP74 - where is at?
I'm rather in favour of DIP74... what's unprincipled about it? What would you do instead? On 11 October 2015 at 10:20, deadalnix via Digitalmars-d wrote: > On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: >> >> So I have another upcoming opportunity to introduce D in my workplace, >> this time as a front-end/plugin language to our C++ infrastructure, which is >> promising since I already have considerable experience in this area (my work >> at Remedy with Quantum Break), and there is a lot of recent work to interact >> better with C++, which we will stress-test extensively. >> >> You only get so many shots at this; but this is a particularly promising >> opportunity, since the C++ code is a nightmare, and the contrast against D >> will allow a lot of coders to see the advantage. >> >> There is however one critical missing feature, DIP74... where is it at >> currently? How is it going? Is it likely to be accepted in the near-term? >> Some sort of approximate timeline? >> >> I think it would be a mistake for me to introduce this without DIP74, >> since we will rely on it VERY heavily, and the machinery to work-around it >> will start to look just as heavy-weight as the C++ code I'm trying to >> deprecate... but then waiting on it starts to look like missing the window >> of opportunity. >> >> Thoughts? > > > It doesn't looks like it is getting implemented. And, to be honest, I'd > rather go a principle approach + library support rather than a pie of hacks. > > The pile of hacks approach is what made C++ C++. >
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 00:20:08 UTC, deadalnix wrote: It doesn't looks like it is getting implemented. And, to be honest, I'd rather go a principle approach + library support rather than a pie of hacks. The pile of hacks approach is what made C++ C++. AFAIK, Walter and Andrei are still in favor of something that's at least similar to DIP 74. Andrei made a comment in a thread just the other day that indicated that he was in favor of having a way to build reference counting into classes. So, I don't know why you think that it's not going to be implemented other than the fact that it hasn't been implemented yet. It wouldn't surprise me if the DIP needed some tweaking though. Regardless, now is not the best time to ask this sort of question, because Walter and Andrei are on their trip to Romania, and they're going to have a limited presence in the newsgroup at the moment. It wouldn't surprise me though if something like the possibility of getting D into another company relied on something like DIP 74 helped push it along and got it sorted out faster. Clearly, Walter and Andrei think that it's an issue and have done some work on it at the theoretical level, but I don't know where it sits on their priority list. And even if DIP 74 were fully sorted out tomorrow, it would still require Walter or someone else actually implementing it, and that's probably not a small undertaking. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 01:48:05 UTC, Manu wrote: I'm rather in favour of DIP74... what's unprincipled about it? What would you do instead? Well, DIP25 and DIP74 are ad hoc adding to the language to support specific use cases. I think the whole thing is wrong headed. Ideally when designing the language you want to have a set of principled tools added to the language that enable the uses cases you want. You don't bake the uses cases into the language. Inevitably, new use cases crop up, they get poor support and/or ends up as new addition into the language, with more and more complexity along the road. The problem at hand is fairly well know at this stage: it is ownership. Everything else can be done as library. The DIP25/DIP74 combo is quite limited. It doesn't solve ownership transfer, it forces an ownership model on the user, it doesn't reduce friction with the GC, it doesn't allow to delegate memory management to the user (the only kind that make sense). and worse, as these issue aren't orthogonal to these DIP, they actively make these problem harder to solve. Solution have been proposed to solve these. It is not hardly new. I mean C++ obviously got the hint: https://github.com/isocpp/CppCoreGuidelines/blob/09aef9bd86d933bc1e1ffe344eb2e73d2de15685/docs/Lifetimes%20I%20and%20II%20-%20v0.9.1.pdf If we go these DIP road, there is no coming back and this will get in the way of a principled approach.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 02:01:09 UTC, Jonathan M Davis wrote: AFAIK, Walter and Andrei are still in favor of something that's at least similar to DIP 74. Andrei made a comment in a thread just the other day that indicated that he was in favor of having a way to build reference counting into classes. So, I don't know why you think that it's not going to be implemented other than the fact that it hasn't been implemented yet. It wouldn't surprise me if the DIP needed some tweaking though. Yes, and that's quite ridiculous. I mean this is getting into ridiculous ego battle. Remind of that concept vs static if grand debate, the peak of ridiculousness (everybody know you don't need type system when you have if statement and vice versa, so the same must be true at compile time as well). When a direction obviously showed to be the wrong one, the rational thing to do is not to double down in order to not admit one is wrong. DIP25 implementation showed a ton of limitations and pitfalls. It isn't even possible to do a slice type with eager deallocation, just one with deferred deallocation, with complex strategies to make it all safe. It is a sign of a poorly thought out language addition. It wouldn't surprise me though if something like the possibility of getting D into another company relied on something like DIP 74 helped push it along and got it sorted out faster. Clearly, Walter and Andrei think that it's an issue and have done some work on it at the theoretical level, but I don't know where it sits on their priority list. And even if DIP 74 were fully sorted out tomorrow, it would still require Walter or someone else actually implementing it, and that's probably not a small undertaking. - Jonathan M Davis Yeah, we saw what happens with attributes. Don't get me wrong, attribute are a very useful addition to the language and all, but the current implementation has some serious flaws, none of them could be addressed as it was pushed out of the door in an inconsequent manner. The fact that dlang.org is littered of antipaterns usage of them is quite telling. I'm all for pushing useful feature, especially if that can drive adoption in a company. But using it as an excuse for releasing half backed feature is VERY short sighted.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote: If we go these DIP road, there is no coming back and this will get in the way of a principled approach. Then come up with an alternative DIP which shows a better way to solve this. As it stands, it looks likely that we'll end up with some form of DIP 74, and if you have a better proposal, then now is probably the time to do it. Personally, I obviously haven't been following this closely enough, because I don't understand why something like RefCounted can't be made to do what we need with regards to reference counting and classes. It does get a bit nasty when inheritance and whatnot get involved, but C++ was able to solve that easily enough, and we should be able to do the same. The one place in-language where I'm sure that something like RefCounted doesn't do it is exceptions, since we really should have a way to make those reference counted, but you can only throw something derived from Throwable - which means a class and not a wrapper around one. So, we need a tweak of some kind to the language there, but that's pretty specific, whereas it _should_ be possible to get something like RefCounted to at least solve the normal cases. Clearly though, Walter and Andrei have come to the conclusion that it's not. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 04:35:03 UTC, Jonathan M Davis wrote: On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote: If we go these DIP road, there is no coming back and this will get in the way of a principled approach. Then come up with an alternative DIP which shows a better way to solve this. As it stands, it looks likely that we'll end up with some form of DIP 74, and if you have a better proposal, then now is probably the time to do it. I have, other have as well, in fact there was a lengthy discussion in private between Walter, Andrei, myself and some other where very precise proposal has been made. I'm sorry but that's bullshit. I'm tired of reexplaining the same thing again and again while is it clear that nobody cares about facts here. If people would care about facts, the DIP25 fiasco would have been enough to put the ideas back on the table. The one place in-language where I'm sure that something like RefCounted doesn't do it is exceptions, since we really should have a way to make those reference counted, but you can only throw something derived from Throwable - which means a class and not a wrapper around one. So, we need a tweak of some kind to the language there, but that's pretty specific, whereas it _should_ be possible to get something like RefCounted to at least solve the normal cases. Clearly though, Walter and Andrei have come to the conclusion that it's not. Proposal included solution for this problem. Exception ownerhip is transferred to the runtime when thrown, and the runtime transfer it back to the catch block that effectively catches it.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 04:56:25 UTC, deadalnix wrote: On Sunday, 11 October 2015 at 04:35:03 UTC, Jonathan M Davis wrote: On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote: If we go these DIP road, there is no coming back and this will get in the way of a principled approach. Then come up with an alternative DIP which shows a better way to solve this. As it stands, it looks likely that we'll end up with some form of DIP 74, and if you have a better proposal, then now is probably the time to do it. I have, other have as well, in fact there was a lengthy discussion in private between Walter, Andrei, myself and some other where very precise proposal has been made. I'm sorry but that's bullshit. I'm tired of reexplaining the same thing again and again while is it clear that nobody cares about facts here. If people would care about facts, the DIP25 fiasco would have been enough to put the ideas back on the table. Well, if they won't listen, they won't listen. And if they're wrong, we'll be worse off for it. Unfortunately, I wasn't involved in those discussions and haven't looked into DIP 25 much (I was too busy at the time of the major discussion for it IIRC). So, I'm not familiar enough with it to have a properly informed opinion. But convincing Walter and Andrei is typically pretty difficult. They do come around eventually at least some of the time though. So, as frustrating as such discussions can be, they do bear fruit at least some of the time (whether it's by them convincing you or by you convincing them). And since DIP 25 has only been implemented with a flag rather than adding it to the language proper yet, there's still time to convince them before we're fully committed to it - as difficult as convincing them may be. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote: The problem at hand is fairly well know at this stage: it is ownership. Everything else can be done as library. This.
Re: DIP74 - where is at?
On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: [...] Speaking of DIP74 can't we just wrap a class in a struct with use reference counting with and use alias this?
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 05:18:26 UTC, Jonathan M Davis wrote: Well, if they won't listen, they won't listen. And if they're wrong, we'll be worse off for it. Unfortunately, I wasn't involved in those discussions and haven't looked into DIP 25 much (I was too busy at the time of the major discussion for it IIRC). So, I'm not familiar enough with it to have a properly informed opinion. But convincing Walter and Andrei is typically pretty difficult. They do come around eventually at least some of the time though. So, as frustrating as such discussions can be, they do bear fruit at least some of the time (whether it's by them convincing you or by you convincing them). And since DIP 25 has only been implemented with a flag rather than adding it to the language proper yet, there's still time to convince them before we're fully committed to it - as difficult as convincing them may be. - Jonathan M Davis Truth be told at the time I was doubtful a DIP25 like approach, but willing to give it a try. They had me convinced of that much. Maybe that wasn't that powerful, but maybe that was powerful enough for basics uses cases and would cut it. Seeing what comes out of it confirmed that no, it won't, and once you get DIP25 + DIP74 you are in the same ballpark as other proposals in term of language additions, but barley measure in terms of capabilities.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote: On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: [...] Speaking of DIP74 can't we just wrap a class in a struct with use reference counting with and use alias this? You can. It is not safe, but it will do. Using type qualifier, one can get better than C++'s shared_ptr already.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:
Speaking of DIP74 can't we just wrap a class in a struct with
use reference counting with and use alias this?
Also how will DIP74 work with incremental compilation?
---
extern (D) class RcClass;
void func(RcClass a)
{
//opps
auto b = a;
return;
}
---
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote: On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: [...] Speaking of DIP74 can't we just wrap a class in a struct with use reference counting with and use alias this? alias is problematic, because it allows the class reference to escape. opDispatch doesn't have that problem, though there may be other complications that it introduces (I don't know). It does get kind of complicated though when you consider member functions which return the a reference to the object and things like that. So, while it's generally feasible, it's not that hard for it to become unsafe. How much that matters is debatable, but it could make it so that reference counting classes is infeasible in @safe code. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 06:10:32 UTC, Jonathan M Davis wrote: On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote: On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: [...] Speaking of DIP74 can't we just wrap a class in a struct with use reference counting with and use alias this? alias is problematic, because it allows the class reference to escape. opDispatch doesn't have that problem, though there may be other complications that it introduces (I don't know). It does get kind of complicated though when you consider member functions which return the a reference to the object and things like that. So, while it's generally feasible, it's not that hard for it to become unsafe. How much that matters is debatable, but it could make it so that reference counting classes is infeasible in @safe code. - Jonathan M Davis Ok dispatch has that problem. You can escape the this pointer from within member functions. Always. There is no @safe reference counting without language support. Either the RC is backed into the language (DIP74) or ownership is baked into the language, so that RC can be baked into the library.
Re: DIP74 - where is at?
On 11 October 2015 at 14:16, deadalnix via Digitalmars-d wrote: > On Sunday, 11 October 2015 at 01:48:05 UTC, Manu wrote: >> >> I'm rather in favour of DIP74... what's unprincipled about it? What would >> you do instead? >> > > Well, DIP25 and DIP74 are ad hoc adding to the language to support specific > use cases. I think the whole thing is wrong headed. I don't really see that DIP25 and DIP74 are particularly closely related. DIP25 is a lame cop-out with respect to scope. I was a major backer of a proper scope implementation, and given that, I don't/didn't support DIP25 at all. DIP74 on the other hand introduces 2 magic functions that the compiler calls as the appropriate moments to implement ref counting. The implementation is somewhat detail, theoretically changeable in the future, but I think language support for ref-counting primitives is critical for ref counting to be efficient and simple. Look at C++, and we see a disaster. C++ basically implemented rval-references to improve (not solve) the RC problem... we don't have rval-ref's in D, so we're screwed from the library angle. > Ideally when designing the language you want to have a set of principled > tools added to the language that enable the uses cases you want. You don't > bake the uses cases into the language. Inevitably, new use cases crop up, > they get poor support and/or ends up as new addition into the language, with > more and more complexity along the road. > > The problem at hand is fairly well know at this stage: it is ownership. > Everything else can be done as library. Yup, I was firmly behind scope (ie, borrowing), but that was destroyed, and DIP25 kinda cements the death of that :( That said, this has nothing to do with ref-counting. Scheduling (and optimising) calls to inc/dec is not something I've seen language tools implement efficiently. The compiler needs to understand that it has the authority to elide pairs appropriately, even when the optimiser is unable to see the code that implements the inc/dec pair. I can't think of any way to do this without a foundation like DIP74...? > The DIP25/DIP74 combo is quite limited. It doesn't solve ownership transfer, > it forces an ownership model on the user, it doesn't reduce friction with > the GC, it doesn't allow to delegate memory management to the user (the only > kind that make sense). and worse, as these issue aren't orthogonal to these > DIP, they actively make these problem harder to solve. How? I don't see how opInc/opDec really changes the underlying type system or allocations mechanisms much, it just gives the compiler the opportunity to understand the optimise reference counting, which is a well understood problem. > Solution have been proposed to solve these. It is not hardly new. I mean C++ > obviously got the hint: > https://github.com/isocpp/CppCoreGuidelines/blob/09aef9bd86d933bc1e1ffe344eb2e73d2de15685/docs/Lifetimes%20I%20and%20II%20-%20v0.9.1.pdf > > If we go these DIP road, there is no coming back and this will get in the > way of a principled approach. DIP25 is done, I rejected it but it's done... I don't really see how DIP74 ruins anything?
Re: DIP74 - where is at?
On 11 October 2015 at 14:25, deadalnix via Digitalmars-d wrote: > On Sunday, 11 October 2015 at 02:01:09 UTC, Jonathan M Davis wrote: >> >> AFAIK, Walter and Andrei are still in favor of something that's at least >> similar to DIP 74. Andrei made a comment in a thread just the other day that >> indicated that he was in favor of having a way to build reference counting >> into classes. So, I don't know why you think that it's not going to be >> implemented other than the fact that it hasn't been implemented yet. It >> wouldn't surprise me if the DIP needed some tweaking though. >> > > Yes, and that's quite ridiculous. I mean this is getting into ridiculous ego > battle. Remind of that concept vs static if grand debate, the peak of > ridiculousness (everybody know you don't need type system when you have if > statement and vice versa, so the same must be true at compile time as well). > > When a direction obviously showed to be the wrong one, the rational thing to > do is not to double down in order to not admit one is wrong. > > DIP25 implementation showed a ton of limitations and pitfalls. It isn't even > possible to do a slice type with eager deallocation, just one with deferred > deallocation, with complex strategies to make it all safe. > > It is a sign of a poorly thought out language addition. > >> It wouldn't surprise me though if something like the possibility of >> getting D into another company relied on something like DIP 74 helped push >> it along and got it sorted out faster. Clearly, Walter and Andrei think that >> it's an issue and have done some work on it at the theoretical level, but I >> don't know where it sits on their priority list. And even if DIP 74 were >> fully sorted out tomorrow, it would still require Walter or someone else >> actually implementing it, and that's probably not a small undertaking. >> >> - Jonathan M Davis > > > Yeah, we saw what happens with attributes. Don't get me wrong, attribute are > a very useful addition to the language and all, but the current > implementation has some serious flaws, none of them could be addressed as it > was pushed out of the door in an inconsequent manner. The fact that > dlang.org is littered of antipaterns usage of them is quite telling. What's wrong with attributes? I can think of some needed additions to finish the job, but what would you say is fundamentally wrong with them as is? > I'm all for pushing useful feature, especially if that can drive adoption in > a company. But using it as an excuse for releasing half backed feature is > VERY short sighted. What would you do instead? I'm happy with DIP74, and I haven't followed threads where people have objected and why...?
Re: DIP74 - where is at?
On 11 October 2015 at 14:35, Jonathan M Davis via Digitalmars-d wrote: > On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote: >> >> If we go these DIP road, there is no coming back and this will get in the >> way of a principled approach. > > > Then come up with an alternative DIP which shows a better way to solve this. > As it stands, it looks likely that we'll end up with some form of DIP 74, > and if you have a better proposal, then now is probably the time to do it. > > Personally, I obviously haven't been following this closely enough, because > I don't understand why something like RefCounted can't be made to do what we > need with regards to reference counting and classes. It does get a bit nasty > when inheritance and whatnot get involved, but C++ was able to solve that > easily enough, and we should be able to do the same. C++ didn't solve anything(?). C++ doesn't support ref-counting at all! shared_ptr is not a part of the language, or a proper ref counting mechanism. It's just a hack; it's awkward, and really inefficient (the compiler can't optimise it). ARC requires language knowledge. I don't know what language primitives can possibly allow the compiler to do proper ref fiddling optimisation with a lib?
Re: DIP74 - where is at?
On 11 October 2015 at 15:57, deadalnix via Digitalmars-d wrote: > On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote: >> >> On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: >>> >>> [...] >> >> >> Speaking of DIP74 can't we just wrap a class in a struct with use >> reference counting with and use alias this? > > > You can. It is not safe, but it will do. Using type qualifier, one can get > better than C++'s shared_ptr already. How?
Re: DIP74 - where is at?
On 11 October 2015 at 16:06, Freddy via Digitalmars-d
wrote:
> On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:
>>
>> Speaking of DIP74 can't we just wrap a class in a struct with use
>> reference counting with and use alias this?
>
>
> Also how will DIP74 work with incremental compilation?
> ---
> extern (D) class RcClass;
>
> void func(RcClass a)
> {
> //opps
> auto b = a;
> return;
> }
> ---
extern (D) class RcClass
{
void opInc();
void opDec(); // or whatever they're called
}
?
Re: DIP74 - where is at?
On 11 October 2015 at 15:52, Freddy via Digitalmars-d wrote: > On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: >> >> [...] > > > Speaking of DIP74 can't we just wrap a class in a struct with use reference > counting with and use alias this? Because the language doesn't have ref counting. And... no. If the 'solution' D offers is more complicated than C++, why would I leave C++? I certainly won't win over any new D users when I show them how awesome D is at making their already horribly complex code even worse.
Re: DIP74 - where is at?
On 11 October 2015 at 16:17, deadalnix via Digitalmars-d wrote: > On Sunday, 11 October 2015 at 06:10:32 UTC, Jonathan M Davis wrote: >> >> On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote: >>> >>> On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: [...] >>> >>> >>> Speaking of DIP74 can't we just wrap a class in a struct with use >>> reference counting with and use alias this? >> >> >> alias is problematic, because it allows the class reference to escape. >> opDispatch doesn't have that problem, though there may be other >> complications that it introduces (I don't know). It does get kind of >> complicated though when you consider member functions which return the a >> reference to the object and things like that. So, while it's generally >> feasible, it's not that hard for it to become unsafe. How much that matters >> is debatable, but it could make it so that reference counting classes is >> infeasible in @safe code. >> >> - Jonathan M Davis > > > Ok dispatch has that problem. You can escape the this pointer from within > member functions. Always. There is no @safe reference counting without > language support. Either the RC is backed into the language (DIP74) or > ownership is baked into the language, so that RC can be baked into the > library. Ownership is only part of the problem (and I support your ideas about ownership, I think D is completely going the wrong way on ownership). There is still the problem that the inc/dec primitives need to be known for the compiler to optimise. Constructors/destructors don't do it semantically. That seems to be what DIP74 is introducing, and even if the ownership problems were addressed/improved, DIP74 would just remap to those concepts. We still need the inc/dec primitives either way.
Re: DIP74 - where is at?
On 10/11/15 7:25 AM, deadalnix wrote: On Sunday, 11 October 2015 at 02:01:09 UTC, Jonathan M Davis wrote: AFAIK, Walter and Andrei are still in favor of something that's at least similar to DIP 74. Andrei made a comment in a thread just the other day that indicated that he was in favor of having a way to build reference counting into classes. So, I don't know why you think that it's not going to be implemented other than the fact that it hasn't been implemented yet. It wouldn't surprise me if the DIP needed some tweaking though. Yes, and that's quite ridiculous. I mean this is getting into ridiculous ego battle. It's unlikely that adding inflammation to this would help too much - sticking to the technical points is more helpful. You're saying that as far as you can tell you have made the perfect argument in favor of your proposed approach (have you written it up as a DIP?) and that essentially there is no reasonable way your idea could not be accepted. If you failed to raise the response you think that proposal deserves, getting emotional about it can't possibly be more productive than reiterating the technical point. We're all on the same boat. We want to make D better. Let's. Andrei
Re: DIP74 - where is at?
On 10/11/15 9:48 AM, Manu via Digitalmars-d wrote: C++ basically implemented rval-references to improve (not solve) the RC problem... Interesting, haven't heard of this viewpoint. Could you please give detail on that? -- Andrei
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 06:55:50 UTC, Manu wrote: On 11 October 2015 at 14:35, Jonathan M Davis via Digitalmars-d wrote: On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote: If we go these DIP road, there is no coming back and this will get in the way of a principled approach. Then come up with an alternative DIP which shows a better way to solve this. As it stands, it looks likely that we'll end up with some form of DIP 74, and if you have a better proposal, then now is probably the time to do it. Personally, I obviously haven't been following this closely enough, because I don't understand why something like RefCounted can't be made to do what we need with regards to reference counting and classes. It does get a bit nasty when inheritance and whatnot get involved, but C++ was able to solve that easily enough, and we should be able to do the same. C++ didn't solve anything(?). C++ doesn't support ref-counting at all! shared_ptr is not a part of the language, or a proper ref counting mechanism. It's just a hack; it's awkward, and really inefficient (the compiler can't optimise it). ARC requires language knowledge. I don't know what language primitives can possibly allow the compiler to do proper ref fiddling optimisation with a lib? Ref-counting with shared_ptr works just fine. It just doesn't optimize out any of the incrementing or decrementing. And while having those optimized out would be nice, I don't think that in and of itself that makes it worth having ref-counting in the language rather than in the library. Maybe C++ ref-counting hasn't work well for your use cases, but I've used it for years, and it's worked great for what I've needed. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 06:48:54 UTC, Manu wrote: I don't really see that DIP25 and DIP74 are particularly closely related. Ho they are. It is all about ownership. DIP25 is a lame cop-out with respect to scope. I was a major backer of a proper scope implementation, and given that, I don't/didn't support DIP25 at all. DIP25 being a sorry excuse for ownership is the very reason DIP74 is needed. Rust can do reference counting just fine all in library without need for core language support. DIP74 on the other hand introduces 2 magic functions that the compiler calls as the appropriate moments to implement ref counting. The implementation is somewhat detail, theoretically changeable in the future, but I think language support for ref-counting primitives is critical for ref counting to be efficient and simple. Look at C++, and we see a disaster. C++ basically implemented rval-references to improve (not solve) the RC problem... we don't have rval-ref's in D, so we're screwed from the library angle. C++ ref counting is not slow for the reasons you think it is. ARC is overly complex and also not as fast as you think it is. First, they are slow because in C++ (or ObjC), you don't know what is shared and what isn't. As a result, you got to go the pessimistic road and use thread safe ref counting. This can be a solved problem in D purely using the type qualifier. Second, exceptions. Their unwinding makes pair of inc/dec hard to recognize, plus now cleanup is need at pretty much every frames, meaning incredibly slow exceptions + various control flow optimization that aren't possible anymore (tail call being one of them). One that one, nobody has a solution. Not ARC (that's why swift does not have exception), not C++ and not us. IMO the obvious solution here is to allow ourselves to leak on unwind and rely on the GC. Yes, sometime the GC is best. That being said, it is up to the application to decide if leaking is acceptable or not, and so this can't be baked in the language (this also means that the user must be in charge the of deallocation policy, which puts a major dent into DIP74). The 3rd problem is safety. This is where ownership shines. The 4th problem is elision of refcount operations when not needed. Some thing, ownership can solve this. No need to do refcount operations when not necessary. The compiler is perfectly able to optimize pairs of inc/dec granted it has the rights infos. And a proper ownership system provide the right infos. Yup, I was firmly behind scope (ie, borrowing), but that was destroyed, and DIP25 kinda cements the death of that :( It is implemented with a flag. That allowed us to try it and get real life experience. That is a good thing. We aren't committed to it at this stage. And now we have actual experimentation that shows that it is too underpowered to be really useful. The smart move here is to admit defeat and reconsider.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 07:01:35 UTC, Manu wrote: On 11 October 2015 at 15:57, deadalnix via Digitalmars-d wrote: On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote: On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: [...] Speaking of DIP74 can't we just wrap a class in a struct with use reference counting with and use alias this? You can. It is not safe, but it will do. Using type qualifier, one can get better than C++'s shared_ptr already. How? In C++, you need to assume things are shared, and, as such, use thread safe inc/dec . That means compiler won't be able to optimize them. D can do better as sharedness is part of the type system.
Re: DIP74 - where is at?
On 11 October 2015 at 17:10, Andrei Alexandrescu via Digitalmars-d wrote: > On 10/11/15 9:48 AM, Manu via Digitalmars-d wrote: >> >> C++ basically implemented rval-references to >> improve (not solve) the RC problem... > > > Interesting, haven't heard of this viewpoint. Could you please give detail > on that? -- Andrei Yeah, that is a terrible claim, but that do offer some advantage (in this situation as with others). The language doesn't support ref counting; constructors/destructors are a pretty crude mechanism to implement them, since they don't model the ref-counting pattern but a superset, which technically allows it to work, but it's pretty lame and inefficient to do ref-counting this way, and it's impossible for code (or the compiler) to distinguish between actual ref-counting events and must behave conservatively. rvalue-references allow the language to elide some unnecessary assignment logic, which can be further used to perform some inc/dec elision, but it starts to get extremely complex at this point, and it still doesn't do the trick. It's a sort of band-aid, but certainly not just for RC.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 07:08:26 UTC, Andrei Alexandrescu wrote: On 10/11/15 7:25 AM, deadalnix wrote: On Sunday, 11 October 2015 at 02:01:09 UTC, Jonathan M Davis wrote: AFAIK, Walter and Andrei are still in favor of something that's at least similar to DIP 74. Andrei made a comment in a thread just the other day that indicated that he was in favor of having a way to build reference counting into classes. So, I don't know why you think that it's not going to be implemented other than the fact that it hasn't been implemented yet. It wouldn't surprise me if the DIP needed some tweaking though. Yes, and that's quite ridiculous. I mean this is getting into ridiculous ego battle. It's unlikely that adding inflammation to this would help too much - sticking to the technical points is more helpful. You're saying that as far as you can tell you have made the perfect argument in favor of your proposed approach (have you written it up as a DIP?) and that essentially there is no reasonable way your idea could not be accepted. If you failed to raise the response you think that proposal deserves, getting emotional about it can't possibly be more productive than reiterating the technical point. We're all on the same boat. We want to make D better. Let's. Andrei No I'm not saying I've made the perfect argument. I'm saying that DIP 25 was implemented and showed that it was too limited. The various experiment with it showed that is wasn't enough (ref counted objects) or required to jump though a lot of hoops for somewhat disappointing result (ref counted arrays). DIP74 is a direct result of DIP25 limitations. I think we should start by acknowledging that the DIP25 experiment was not a success before taking it for granted and proceed. I do think a buffed up version of DIP25 would make DIP74 obsolete (or implementable as a library).
Re: DIP74 - where is at?
On 11 October 2015 at 17:16, Jonathan M Davis via Digitalmars-d wrote: > On Sunday, 11 October 2015 at 06:55:50 UTC, Manu wrote: >> >> On 11 October 2015 at 14:35, Jonathan M Davis via Digitalmars-d >> wrote: >>> >>> On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote: If we go these DIP road, there is no coming back and this will get in the way of a principled approach. >>> >>> >>> >>> Then come up with an alternative DIP which shows a better way to solve >>> this. As it stands, it looks likely that we'll end up with some form of DIP >>> 74, and if you have a better proposal, then now is probably the time to do >>> it. >>> >>> Personally, I obviously haven't been following this closely enough, >>> because I don't understand why something like RefCounted can't be made to do >>> what we need with regards to reference counting and classes. It does get a >>> bit nasty when inheritance and whatnot get involved, but C++ was able to >>> solve that easily enough, and we should be able to do the same. >> >> >> C++ didn't solve anything(?). C++ doesn't support ref-counting at all! >> shared_ptr is not a part of the language, or a proper ref counting >> mechanism. It's just a hack; it's awkward, and really inefficient (the >> compiler can't optimise it). >> ARC requires language knowledge. I don't know what language primitives >> can possibly allow the compiler to do proper ref fiddling optimisation >> with a lib? > > > Ref-counting with shared_ptr works just fine. It just doesn't optimize out > any of the incrementing or decrementing. I wouldn't call that 'working', I'd call that 'grossly over-working' at best. I'd personally diagnose signals firing at improper times to be a bug, and that's effectively what we have ;) I think it's absolutely worthwhile that a language should know that refcounting is a thing, and what the proper handling is. I don't know any way to model it accurately without language support. > And while having those optimized > out would be nice, I don't think that in and of itself that makes it worth > having ref-counting in the language rather than in the library. For real? If you're handling ref-counted objects, ref-fiddling often completely overwhelms your code at the interface level, and god forbid you have some sort of adaptation layers, shims, small temporaries, call-throughs, or iterations. Problem is amplified big time if your ref-counting functions are foreign library functions, which is pretty common in C api's. These functions need to be wrapped with a primitive that properly knows when the functions should be called. I'm seeing objects with RC==8+ or so across one event handling call-tree. > Maybe C++ ref-counting hasn't work well for your use cases, but I've used it > for years, and it's worked great for what I've needed. Sure. If you don't care, I'm sure it's fine. But I don't feel it's reasonable to say C++ has ref counting. You might as well say "C++ has garbage collection" (which is probably actually more true than this) ;) ... Objective-C supports ref counting.
Re: DIP74 - where is at?
On 11 October 2015 at 17:30, deadalnix via Digitalmars-d wrote: > On Sunday, 11 October 2015 at 06:48:54 UTC, Manu wrote: >> >> I don't really see that DIP25 and DIP74 are particularly closely related. > > > Ho they are. It is all about ownership. > >> DIP25 is a lame cop-out with respect to scope. I was a major backer of >> a proper scope implementation, and given that, I don't/didn't support >> DIP25 at all. >> > > DIP25 being a sorry excuse for ownership is the very reason DIP74 is needed. > Rust can do reference counting just fine all in library without need for > core language support. > >> DIP74 on the other hand introduces 2 magic functions that the compiler >> calls as the appropriate moments to implement ref counting. The >> implementation is somewhat detail, theoretically changeable in the future, >> but I think language support for ref-counting primitives is critical for ref >> counting to be efficient and simple. Look at C++, and we see a disaster. C++ >> basically implemented rval-references to improve (not solve) the RC >> problem... we don't have rval-ref's in D, so we're screwed from the library >> angle. >> > > C++ ref counting is not slow for the reasons you think it is. ARC is overly > complex and also not as fast as you think it is. > > First, they are slow because in C++ (or ObjC), you don't know what is shared > and what isn't. As a result, you got to go the pessimistic road and use > thread safe ref counting. This can be a solved problem in D purely using the > type qualifier. > > Second, exceptions. Their unwinding makes pair of inc/dec hard to recognize, > plus now cleanup is need at pretty much every frames, meaning incredibly > slow exceptions + various control flow optimization that aren't possible > anymore (tail call being one of them). > > One that one, nobody has a solution. Not ARC (that's why swift does not have > exception), not C++ and not us. IMO the obvious solution here is to allow > ourselves to leak on unwind and rely on the GC. Yes, sometime the GC is > best. That being said, it is up to the application to decide if leaking is > acceptable or not, and so this can't be baked in the language (this also > means that the user must be in charge the of deallocation policy, which puts > a major dent into DIP74). > > The 3rd problem is safety. This is where ownership shines. > > The 4th problem is elision of refcount operations when not needed. Some > thing, ownership can solve this. No need to do refcount operations when not > necessary. > > The compiler is perfectly able to optimize pairs of inc/dec granted it has > the rights infos. And a proper ownership system provide the right infos. > >> Yup, I was firmly behind scope (ie, borrowing), but that was >> destroyed, and DIP25 kinda cements the death of that :( >> > > It is implemented with a flag. That allowed us to try it and get real life > experience. That is a good thing. We aren't committed to it at this stage. > And now we have actual experimentation that shows that it is too > underpowered to be really useful. The smart move here is to admit defeat and > reconsider. Nothing here explains to me how it is that the compiler can do without RC primitives that the compiler knows it is allowed to optimise/schedule as it likes? How does it look in rust that it is able to optimise calls to inc/dec primitives? You also make the points that I expect to bank on in D; we are thread-local by default (big advantage!), nothrow is pervasive (big advantage!), but the language still doesn't know the primitives that it can use to optimise. The ownership proposal solving the problem isn't entirely correct. Sure, borrowed pointers imply RC elision, and it's particularly useful for writing API's that are allocation/ownership agnostic, but it doesn't solve the problem as you suggest. Once a pointer is borrowed, it can't be assigned to another RC pointer anymore. If you must use borrowing to implement RC elision across calls, then you lose the ability to give someone else a reference down the call-tree. Passing an RC object proper to a function also elides RC fiddling, but the compiler can't distinguish between RC fiddling and copying logic unless the RC primitives are explicit. I find that this is a very common case.
Re: DIP74 - where is at?
On 10/11/15 10:16 AM, Jonathan M Davis wrote: On Sunday, 11 October 2015 at 06:55:50 UTC, Manu wrote: On 11 October 2015 at 14:35, Jonathan M Davis via Digitalmars-d wrote: On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote: If we go these DIP road, there is no coming back and this will get in the way of a principled approach. Then come up with an alternative DIP which shows a better way to solve this. As it stands, it looks likely that we'll end up with some form of DIP 74, and if you have a better proposal, then now is probably the time to do it. Personally, I obviously haven't been following this closely enough, because I don't understand why something like RefCounted can't be made to do what we need with regards to reference counting and classes. It does get a bit nasty when inheritance and whatnot get involved, but C++ was able to solve that easily enough, and we should be able to do the same. C++ didn't solve anything(?). C++ doesn't support ref-counting at all! shared_ptr is not a part of the language, or a proper ref counting mechanism. It's just a hack; it's awkward, and really inefficient (the compiler can't optimise it). ARC requires language knowledge. I don't know what language primitives can possibly allow the compiler to do proper ref fiddling optimisation with a lib? Ref-counting with shared_ptr works just fine. It just doesn't optimize out any of the incrementing or decrementing. And while having those optimized out would be nice, I don't think that in and of itself that makes it worth having ref-counting in the language rather than in the library. Maybe C++ ref-counting hasn't work well for your use cases, but I've used it for years, and it's worked great for what I've needed. Yah, it's a success story and a good baseline. Two things that are suboptimal are the necessity of two allocations unless make_shared is used, and compulsive interlocked inc/dec. We ought to address both issues, but by and large we're looking at a successful experience that we want to have D users benefit from as well. -- Andrei
Re: DIP74 - where is at?
On 10/11/15 10:53 AM, deadalnix wrote: I'm saying that DIP 25 was implemented and showed that it was too limited. The various experiment with it showed that is wasn't enough (ref counted objects) or required to jump though a lot of hoops for somewhat disappointing result (ref counted arrays). DIP74 is a direct result of DIP25 limitations. I think we should start by acknowledging that the DIP25 experiment was not a success before taking it for granted and proceed. I do think a buffed up version of DIP25 would make DIP74 obsolete (or implementable as a library). Walter and I are happy with DIP25, and the fact of the matter is we weren't surprised more complementary work is needed. So no, I won't acknowledge what I don't believe. I'd say the one way to get things looked at seriously is to create a DIP. That's no guarantee it will be accepted but there is a guarantee that our chat at DConf is not sufficient even as a basis for further study. Andrei
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 06:10:32 UTC, Jonathan M Davis wrote: alias is problematic, because it allows the class reference to escape. opDispatch doesn't have that problem, though there may be other complications that it introduces (I don't know). It does get kind of complicated though when you consider member functions which return the a reference to the object and things like that. So, while it's generally feasible, it's not that hard for it to become unsafe. How much that matters is debatable, but it could make it so that reference counting classes is infeasible in @safe code. - Jonathan M Davis Can't we make opAssign and this(this) work with classes add @disable to them? Also rec counted classes shouldn't have members of that have their type, they should have to have members of the aliased struct containing their type.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 13:51:18 UTC, Andrei Alexandrescu wrote: Walter and I are happy with DIP25, and the fact of the matter is we weren't surprised more complementary work is needed. So no, I won't acknowledge what I don't believe. That is an empty statement. What is there to be happy about ? Also the complementary argument pretty much destroy the best argument you and Walter made for DIP25 : it is simple. I mean, one need to look at the big picture. DIP25 + complementary addition is not simple anymore. I'd say the one way to get things looked at seriously is to create a DIP. That's no guarantee it will be accepted but there is a guarantee that our chat at DConf is not sufficient even as a basis for further study. Yeah there are IRL discussion, there are many posts in the forum, there are by mail discussions at DIP25 creation time, there are at least one DIP already. The only rebuttal to all of this is "Walter and I are happy with DIP25, and the fact of the matter", while everybody else is wondering what there is to be happy about.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote: On Sunday, 11 October 2015 at 13:51:18 UTC, Andrei Alexandrescu wrote: Walter and I are happy with DIP25, and the fact of the matter is we weren't surprised more complementary work is needed. So no, I won't acknowledge what I don't believe. That is an empty statement. What is there to be happy about ? Also the complementary argument pretty much destroy the best argument you and Walter made for DIP25 : it is simple. I mean, one need to look at the big picture. DIP25 + complementary addition is not simple anymore. I'd say the one way to get things looked at seriously is to create a DIP. That's no guarantee it will be accepted but there is a guarantee that our chat at DConf is not sufficient even as a basis for further study. Yeah there are IRL discussion, there are many posts in the forum, there are by mail discussions at DIP25 creation time, there are at least one DIP already. The only rebuttal to all of this is "Walter and I are happy with DIP25, and the fact of the matter", while everybody else is wondering what there is to be happy about. Also, I'm sorry but there is no me writing once again a document about what alternative are possible. Spending hours to write documents so that one is answered something along the line of "we are happy with the other thing, but we can't give any reason why" is something I've engaged in several time in already, and has no desire to indulge into this if I have reason to think the same will happen. Your answer here are telling me one thing: it won't be taken seriously.
Re: DIP74 - where is at?
Manu via Digitalmars-d wrote: > Nothing here explains to me how it is that the compiler can do without > RC primitives that the compiler knows it is allowed to > optimise/schedule as it likes? > How does it look in rust that it is able to optimise calls to inc/dec > primitives? > > You also make the points that I expect to bank on in D; we are > thread-local by default (big advantage!), nothrow is pervasive (big > advantage!), but the language still doesn't know the primitives that > it can use to optimise. > > The ownership proposal solving the problem isn't entirely correct. > Sure, borrowed pointers imply RC elision, and it's particularly useful > for writing API's that are allocation/ownership agnostic, but it > doesn't solve the problem as you suggest. Once a pointer is borrowed, > it can't be assigned to another RC pointer anymore. If you must use > borrowing to implement RC elision across calls, then you lose the > ability to give someone else a reference down the call-tree. > Passing an RC object proper to a function also elides RC fiddling, but > the compiler can't distinguish between RC fiddling and copying logic > unless the RC primitives are explicit. I find that this is a very > common case. The key is, that RC in Rust is a move-by-default type. That means if you are passing RCs around, you are actually passing ownership. Only if you explicitly call 'clone', ownership is split up. That means in the default case, no inc/dec is necessary. Even more important, RC isn't used very often. True shared ownership is actually quite rare. References (borrowed pointers) and 'Box'es (unique pointers) are much more common. shared_ptr is much overused in C++ IMO. Tobi
Re: DIP74 - where is at?
On 10/11/15 9:57 PM, deadalnix wrote: On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote: On Sunday, 11 October 2015 at 13:51:18 UTC, Andrei Alexandrescu wrote: Walter and I are happy with DIP25, and the fact of the matter is we weren't surprised more complementary work is needed. So no, I won't acknowledge what I don't believe. That is an empty statement. What is there to be happy about ? Also the complementary argument pretty much destroy the best argument you and Walter made for DIP25 : it is simple. I mean, one need to look at the big picture. DIP25 + complementary addition is not simple anymore. I'd say the one way to get things looked at seriously is to create a DIP. That's no guarantee it will be accepted but there is a guarantee that our chat at DConf is not sufficient even as a basis for further study. Yeah there are IRL discussion, there are many posts in the forum, there are by mail discussions at DIP25 creation time, there are at least one DIP already. The only rebuttal to all of this is "Walter and I are happy with DIP25, and the fact of the matter", while everybody else is wondering what there is to be happy about. Also, I'm sorry but there is no me writing once again a document about what alternative are possible. Could you please point to the document you have already written? Spending hours to write documents so that one is answered something along the line of "we are happy with the other thing, but we can't give any reason why" is something I've engaged in several time in already, and has no desire to indulge into this if I have reason to think the same will happen. Your answer here are telling me one thing: it won't be taken seriously. There's a bit of a stalemate here. So we have: 1. You say that DIP25 is a failure. More so, you demand that is admitted without evidence. What I see is a feature that solves one problem, and solves it well: annotating a function that returns a reference to its argument. The syntactic cost is low, the impact on existing code is small, and the impact on safety is positive. Walter and I think it is the simplest solution of all considered. 2. You refuse to write a DIP under the assumption it will not be taken seriously. Conversely if you do write a DIP there is an expectation it will be approved just because you put work in it. I don't think rewarding work is the right way to go. We need to reward good work. The "work" part (i.e. a DIP) is a prerequisite; you can't demand to implement a complex feature based on posts and discussions. So I'm not sure how we can move forward from here. If you want to discuss DIP74, great, it can be discussed because it exists. My personal opinion on DIP74 is it has holes and corner cases so it seems it doesn't quite hit the spot. One option is to make it work, another is to take a different attack on the problem. But we need the appropriate DIP. Andrei
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote: The only rebuttal to all of this is "Walter and I are happy with DIP25, and the fact of the matter", while everybody else is wondering what there is to be happy about. To be fair, you haven't really said much better. You're claiming that it's clear that it's a failure, whereas Andrei is saying that he doesn't see a problem with it. No concrete arguments are being given. The closest is that you think that the fact that we need something like DIP 74 means that DIP 25's advantage of being simple is lost and that that somehow shows that DIP 25 is a failure. I don't think that it's at all clear that DIP 25 and DIP 74 are even particularly related (and Manu has stated the same). One deals with escaping ref parameters (which doesn't involved reference counting), whereas the other has to do with reference counting. It may very well be that there's a way to cover both issues with some sort of ownership model, but that's not clear from simply looking at the DIPs or the current discussion. All we really have is your assertion that there's a better way. I'm completely open to there being a better solution than DIP 25 or DIP 74, but I have yet to see one explained. The only alternatives to DIP 74 that I've really seen are suggestions that we should be able to do it with library types (like C++ does) and not require built-in ref-counting. If there have been any in-depth explanations or discussions on how an ownership model could solve this, I haven't seen them (though there may very well have been one in a previous thread that I missed), so I really don't know what that would even entail. The only place that it's clear to me that an ownership model of some kind would be nice would be one related to threads so that it we could safely pass mutable data between threads, but honestly, I'm okay with it being unsafe to pass mutable data across threads. D's threading model makes it pretty easy to get that right, and most programs don't need it anyway. But I have no idea how an ownership model for memory in general would work. I'd probably have a much better idea if I had read up on Rust (which I really haven't), but every discussion I've seen on Rust's model seems to indicate that it gets pretty complicated - complicated enough that while it might be more correct, it risks being unusable for the average programmer. I don't know how true that is, but I do think that if we're going to seriously discuss an ownership model in D, we need something concrete discuss and debate. I know that writing a DIP takes time (even a small one; it surprised me how long it took to write DIP 82 given how simple it is, and I've written DIPs before), but if we don't have something like that to discuss, then we're just going to be going in circles here. If you have a concrete proposal that you think would really benefit D, then please write up a DIP for it. We could all be way better off for it. Maybe it'll be rejected, but without it, we're definitely not going to end up with some kind of ownership model like you want. And if you're right, then we will be worse off. Concrete proposals are required to have concrete discussions and move forward. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 14:27:36 UTC, Freddy wrote:
On Sunday, 11 October 2015 at 06:10:32 UTC, Jonathan M Davis
wrote:
alias is problematic, because it allows the class reference to
escape. opDispatch doesn't have that problem, though there may
be other complications that it introduces (I don't know). It
does get kind of complicated though when you consider member
functions which return the a reference to the object and
things like that. So, while it's generally feasible, it's not
that hard for it to become unsafe. How much that matters is
debatable, but it could make it so that reference counting
classes is infeasible in @safe code.
- Jonathan M Davis
Can't we make opAssign and this(this) work with classes add
@disable to them? Also rec counted classes shouldn't have
members of that have their type, they should have to have
members of the aliased struct containing their type.
I don't know if opAssign can be overloaded for a class or not.
That gets a bit wonky IMHO given that you never assign classes -
only references to them. Similarly, class objects never get
postblitted. And something as simple as a parent-child
relationship tends to cause serious problems restricting
references to smart pointers. e.g. take this C++ code
class Parent
{
friend class Child;
public
Parent() _child(make_shared(this)) {}
private:
shared_ptr _child;
}
class Child
{
public:
Child(Parent* parent) : _parent(parent) {}
private:
Parent* _parent;
}
auto parent = make_shared();
In this case in C++, because the ref-counting is not built-in to
the type, there is no way for the Child to have access to its
parent via a shared_ptr. It has to be done via a normal pointer.
D has exactly this same problem. If the ref-counting isn't
built-in, then there are cases where you have to let a
non-ref-counted reference escape.
The problem can be reduced by making it so that RefCounted (or
whatever the wrapper struct is) doesn't allow accidental, direct
access to the wrapped class object. But some stuff will simply be
impossible without providing direct access.
By having the class reference contain the reference count, the
problem can significantly reduced, because then it's possible to
create a shared pointer from the class object without ending up
with a separate reference count. But you still have the problem
of that class object being passed around and not necessarily
always being put in a wrapper object like it's supposed to be.
So, you still have a safety issue.
The only way for this to be completely safe is for the
ref-counting to be built into the language such that there is no
way to have a reference being passed around which isn't
ref-counted. Now, we can certainly take the approach of saying
that we don't care about that being completely safe (in which
case, ref-counting classes will be limited if not outright banned
in @safe code), but if we want @safe ref-counting, we have no
choice.
- Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 07:55:08 UTC, Manu wrote: Sure. If you don't care, I'm sure it's fine. But I don't feel it's reasonable to say C++ has ref counting. You might as well say "C++ has garbage collection" (which is probably actually more true than this) ;) ... Objective-C supports ref counting. Reference counting is now part of C++'s standard library. They may not have the kind of reference counting that you want, but they support reference counting as part of the C++ standard. And certainly for my use cases, having wrapper classes like smart_ptr has worked fantastically. Any performance hit that might be incurred be extraneous increments and decrements is dwarfed by everything else that the program is doing. And I think that it's pretty clear that the same is true for a _lot_ of programs. I can certainly believe that it's not true for the kind of programs that you write, and if we can better support those environments, great. But most programmers don't program in a world where it normally matters whether a couple of extra math operations are done or whether a function call is virtual or not. It matters in certain areas of the program but not in general. We don't want to be doing unnecessary operations, and efficiency in other areas can have a huge impact on our programs, but I think that it's quite clear that you live in a world that cares way more about every little ounce of efficiency than most programmers do. So, I think that it's a huge stretch to say that C++ doesn't have ref-counting or that it doesn't work - or even that a similar solution in D wouldn't work. It would work for many people. If we can come up with one that better supports your use cases, then great. As long as the cost elsewhere isn't too high, why not? But that doesn't mean that a solution like smart_ptr isn't a fantastic one for many of us. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 20:56:28 UTC, Jonathan M Davis wrote: On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote: The only rebuttal to all of this is "Walter and I are happy with DIP25, and the fact of the matter", while everybody else is wondering what there is to be happy about. To be fair, you haven't really said much better. You're claiming that it's clear that it's a failure, whereas Andrei is saying that he doesn't see a problem with it. No concrete arguments are being given. The closest is that you think that the fact that we need something like DIP 74 means that DIP 25's advantage of being simple is lost and that that somehow shows that DIP 25 is a failure. That's not true. I explained in this thread why it is too limited, most notably : 1/ It was barely able to provide a non GC managed array type. More details here: http://forum.dlang.org/thread/mcg8qq$1mbr$1...@digitalmars.com Notably, it is impossible to reclaim the memory eagerly, but only to clean all memory at once when all reference to one of the memory location is gone. This in turn lead to the need to pretty much have a memory allocator in the array. And jump through many hoops. 2/ It is completely unable to handle reference types (hence DIP74). Note that if you don't have indirection, there is no memory allocation problem to begin with as value type will be either on stack or involve some reference at some point. But this whole thing is really a shift of the burden of proof. What do DIP25 enabled that is really worth it ?
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 21:15:38 UTC, Jonathan M Davis wrote: In this case in C++, because the ref-counting is not built-in to the type, there is no way for the Child to have access to its parent via a shared_ptr. It has to be done via a normal pointer. D has exactly this same problem. If the ref-counting isn't built-in, then there are cases where you have to let a non-ref-counted reference escape. In this case one should use unique_ptr, so the D discussion about ref counting is irrelevant. If you have truely shared_ptr semantics and back pointers, then one shoud use weak_ptr for this since the back pointers don't own the resource. Keep in mind that shared_ptr only denote ownership, not resource usage. C++ is now moving towards Rust and static analysis, but as a step seperate from compilation. And that makes sense since it is time consuming and doesn't have to be done for codegen.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 22:07:44 UTC, deadalnix wrote: That's not true. I explained in this thread why it is too limited, most notably : 1/ It was barely able to provide a non GC managed array type. More details here: http://forum.dlang.org/thread/mcg8qq$1mbr$1...@digitalmars.com Notably, it is impossible to reclaim the memory eagerly, but only to clean all memory at once when all reference to one of the memory location is gone. This in turn lead to the need to pretty much have a memory allocator in the array. And jump through many hoops. 2/ It is completely unable to handle reference types (hence DIP74). Note that if you don't have indirection, there is no memory allocation problem to begin with as value type will be either on stack or involve some reference at some point. But this whole thing is really a shift of the burden of proof. What do DIP25 enabled that is really worth it ? DIP 25 fixes a serious flaw in ref. It makes it possible to guarantee that ref is @safe. Without it, ref is not @safe. As I understand it, that's the whole point of DIP 25, and it has nothing to do with reference counting at all. It doesn't even really have anything to do with the heap beyond the fact that a ref parameter might refer to an object on the heap - the safety problem is with the stack, not the heap. Walter's proposal in that link does take advantage of the fact that DIP 25 restricts what can be returned by ref, but that's as close as DIP 25 gets to ref-counting, whereas all of the rest of the stuff you're talking about here has to do with ref-counting. DIP 25 is intended to plug the holes in ref to solve the ref safety problem, whereas DIP 74 is intended to build ref-counting into the language so that it can be done safely. They both relate to @safe code, but as far as I can tell, they're pretty much orthogonal, and arguing against flaws in DIP 74 doesn't really show flaws in DIP 25. When you're arguing about them together rather than separately, it seems like you're seeing some other part of the puzzle that I'm not. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 22:12:50 UTC, Ola Fosheim Grøstad wrote: On Sunday, 11 October 2015 at 21:15:38 UTC, Jonathan M Davis wrote: In this case in C++, because the ref-counting is not built-in to the type, there is no way for the Child to have access to its parent via a shared_ptr. It has to be done via a normal pointer. D has exactly this same problem. If the ref-counting isn't built-in, then there are cases where you have to let a non-ref-counted reference escape. In this case one should use unique_ptr, so the D discussion about ref counting is irrelevant. If you have truely shared_ptr semantics and back pointers, then one shoud use weak_ptr for this since the back pointers don't own the resource. Keep in mind that shared_ptr only denote ownership, not resource usage. I don't want get into arguments about smart_ptr and unique_ptr. It's completely irrelevant to my point. weak_ptr doesn't work when the child is constructed by the parent in the parent's constructor. The fact that the parent does not have access to the smart pointer type that it's about to be put into means that it cannot give that to the child. It has to give it a normal pointer. And that means that you can't make it so that all accesses to the parent object are via a smart pointer. And in the case of D, that means that it doesn't work to restrict all access to a class object to a wrapper struct like RefCounted, and it becomes quite easy for a reference to the class object to escape the wrapper struct. And that means that it's not @safe, because the wrapper struct might destroy the class object when it's destroyed (because its ref-count reached 0) while a reference to that class object is still floating around somewhere. Obviously, it can work (it works in C++). It just means that it can't be @safe and that the programmer has to worry about ensuring that memory doesn't get used incorrectly. But part of the whole point of DIP 74 is so that we can have @safe ref-counting in D. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 20:35:05 UTC, Andrei Alexandrescu wrote: Could you please point to the document you have already written? For instance, we had a discussion with Walter and Mark that eventually yielded DIP25. In there, I made the following proposal : http://pastebin.com/LMkuTbgN I made several other very detailed proposal. Other did. It's not about me here. Others simply abandoned as far as I can tell. I'm just a stubborn idiot. There's a bit of a stalemate here. So we have: 1. You say that DIP25 is a failure. More so, you demand that is admitted without evidence. What I see is a feature that solves one problem, and solves it well: annotating a function that returns a reference to its argument. The syntactic cost is low, the impact on existing code is small, and the impact on safety is positive. Walter and I think it is the simplest solution of all considered. It is indeed the simplest. However, experiences that have been made and discussed in the forum showed it was often too simple to be really useful. I cited example of this, namely the RCArray thing and the existence of DIP74. I don't think the simplicity argument holds water in general as long as we don't have the whole thing. DIP25 + DIP74 + ... must be measured against the alternative. 2. You refuse to write a DIP under the assumption it will not be taken seriously. Conversely if you do write a DIP there is an expectation it will be approved just because you put work in it. I don't think rewarding work is the right way to go. We need to reward good work. The "work" part (i.e. a DIP) is a prerequisite; you can't demand to implement a complex feature based on posts and discussions. No that is inaccurate. I think I have evidence that it won't be taken seriously. To start with, there are already several DIP on the subject and they are not discussed at ALL. Namely : http://wiki.dlang.org/DIP35 http://wiki.dlang.org/DIP36 http://wiki.dlang.org/DIP69 http://wiki.dlang.org/DIP71 These do not even register as a blip on the radar. I don't see how adding my to the pile would change anything. There are not considered because DIP25 is "simpler" and you and Walter "like it". As long as nothing changes here, there is really no point in wasting my time. So I'm not sure how we can move forward from here. If you want to discuss DIP74, great, it can be discussed because it exists. My personal opinion on DIP74 is it has holes and corner cases so it seems it doesn't quite hit the spot. One option is to make it work, another is to take a different attack on the problem. But we need the appropriate DIP. Let's start by the beginning: what good design was enabled by DIP25 ? As long as none is presented, we can't consider it a success.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 20:56:28 UTC, Jonathan M Davis wrote: To be fair, you haven't really said much better. You're claiming that it's clear that it's a failure, whereas Andrei is saying that he doesn't see a problem with it. No concrete arguments are being given. The closest is that you think that the fact that we need something like DIP 74 means that DIP 25's advantage of being simple is lost and that that somehow shows that DIP 25 is a failure. OK first thing first, that is not how that works. DIP25 has been out for a while, so one should have something to show for it. What good thing came out of DIP25 ? That should be what is discussed here rather than having to argue why it is not good enough. See my other messages here to see what isn't good with it. It simply isn't allowing enough to pay for itself. I don't think that it's at all clear that DIP 25 and DIP 74 are even particularly related (and Manu has stated the same). They are the same thing. If you can guarantee that some reference does not escape in uncontrollable manner, you can have ref counting as library. The need for DIP74 only arise because DIP25 cannot ensure that the reference does not escape. But I have no idea how an ownership model for memory in general would work. I'd probably have a much better idea if I had read up on Rust (which I really haven't), but every discussion I've seen on Rust's model seems to indicate that it gets pretty complicated - complicated enough that while it might be more correct, it risks being unusable for the average programmer. I don't know how true that is, but I do think that if we're going to seriously discuss an ownership model in D, we need something concrete discuss and debate. I know that writing a DIP takes time (even a small one; it surprised me how long it took to write DIP 82 given how simple it is, and I've written DIPs before), but if we don't have something like that to discuss, then we're just going to be going in circles here. If you have a concrete proposal that you think would really benefit D, then please write up a DIP for it. We could all be way better off for it. Maybe it'll be rejected, but without it, we're definitely not going to end up with some kind of ownership model like you want. And if you're right, then we will be worse off. Concrete proposals are required to have concrete discussions and move forward. - Jonathan M Davis Yes, you should definitively read on Rust, or read this : https://github.com/isocpp/CppCoreGuidelines/blob/09aef9bd86d933bc1e1ffe344eb2e73d2de15685/docs/Lifetimes%20I%20and%20II%20-%20v0.9.1.pdf Which is basically C++ adopting the idea, but no, it doesn't come from Rut, no, I tell you, C++ invented a long time ago, C++ invented everything, stop doubting us. Things is, Rust has that one mechanism to do everything. As a result it is fairly complex, because it has to handle all common cases. We can do something fairly similar, yet simpler, if we accept to rely on the GC or unsafe constructs for things that do not fit well in the model.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 22:33:44 UTC, Jonathan M Davis wrote: I don't want get into arguments about smart_ptr and unique_ptr. It's completely irrelevant to my point. Ok, but keep in mind that since unique_ptr is rc which max out at 1 and therefore don't need extra support beyond having an owning pointer. So whatever is done for reasoning about rc also has to work for unique. If not thenD is doomed to fail, because nobody wants rc when unique is sufficient. weak_ptr doesn't work when the child is constructed by the parent in the parent's constructor. The fact that the parent does not have access to the smart pointer type that it's about to be put into means that it cannot give that to the child. It has to give it a normal pointer. And that means that you can't make it so that all accesses to the parent object are via a smart pointer. And in the case of D, that means that it doesn't work to restrict all access to a class object to a wrapper struct like RefCounted, and it becomes quite easy for a reference to the class object to escape the wrapper struct. Well, the wrapper approach is no good (is it part of dip74?) since it messes up alignment etc, so the refcount interface should be part of the parent object. The child can be given the Refcount interface from dip74, it won't use it for a weak reference until it dereference (borrow). A weak reference just delays destruction until the borrowing is completed. So it works(?) if you dont allow dereferencing of the weak pointer in the constructor of the child. get used incorrectly. But part of the whole point of DIP 74 is so that we can have @safe ref-counting in D. Well, @safe in D is broken and should just be dropped in favour of something sane like real pointer analysis instead of piling up bad designs like dip25. You only have to run it for release... Who cares if it is slow?
Re: DIP74 - where is at?
On 12 Oct 2015 7:31 am, "Jonathan M Davis via Digitalmars-d" < digitalmars-d@puremagic.com> wrote: > > On Sunday, 11 October 2015 at 07:55:08 UTC, Manu wrote: >> >> Sure. If you don't care, I'm sure it's fine. But I don't feel it's reasonable to say C++ has ref counting. You might as well say "C++ has garbage collection" (which is probably actually more true than this) ;) ... Objective-C supports ref counting. > > > Reference counting is now part of C++'s standard library. They may not have the kind of reference counting that you want, but they support reference counting as part of the C++ standard. > > And certainly for my use cases, having wrapper classes like smart_ptr has worked fantastically. Any performance hit that might be incurred be extraneous increments and decrements is dwarfed by everything else that the program is doing. And I think that it's pretty clear that the same is true for a _lot_ of programs. > > I can certainly believe that it's not true for the kind of programs that you write, and if we can better support those environments, great. But most programmers don't program in a world where it normally matters whether a couple of extra math operations are done or whether a function call is virtual or not. It matters in certain areas of the program but not in general. We don't want to be doing unnecessary operations, and efficiency in other areas can have a huge impact on our programs, but I think that it's quite clear that you live in a world that cares way more about every little ounce of efficiency than most programmers do. > > So, I think that it's a huge stretch to say that C++ doesn't have ref-counting or that it doesn't work - or even that a similar solution in D wouldn't work. It would work for many people. If we can come up with one that better supports your use cases, then great. As long as the cost elsewhere isn't too high, why not? But that doesn't mean that a solution like smart_ptr isn't a fantastic one for many of us. > > - Jonathan M Davis Incidentally, I tried to use shared_ptr initially, but it took about 20 minutes before I realised I had to pass an rc pointer from a method... Since rc is a wrapper, like you said above, you lose it as soon as you're within a method. I then had to write an invasive rc implementation and hard create a new rc instance from 'this', all over the place. Seriously, I don't understand how anyone can say shared_ptr is a success. It's a massive kludge, and it alone has me firmly convinced that rc is weak, inconvenient, and highly complicated without language support.
Re: DIP74 - where is at?
On 10/11/2015 10:35 PM, Andrei Alexandrescu wrote:
...
1. You say that DIP25 is a failure. More so, you demand that is admitted
without evidence.
FWIW, DIP25 is insufficiently formal and/or incorrect.
I have been able to find those holes in the implementation pretty
quickly (I'll also put them to bugzilla):
---
enum N=100;
ref int[N] foo(return ref int[N] s)@safe{ return s; }
int[N]* bar(return ref int[N] s)@safe{ return &foo(s); }
ref int[N] baz()@safe{ int[N] s; return *bar(s); }
// for illustration:
void bang(ref int[N] x)@safe{ x[]=0x25BAD; }
void main()@safe{ bang(baz()); }
---
enum N=100;
ref int[N] fun(ref int[N] x)@safe{
ref int[N] bar(){ return x; }
return bar();
}
ref int[N] hun()@safe{
int[N] k;
return fun(k);
}
// for illustration:
void bang(ref int[N] x)@safe{ x[]=0x25BAD; }
void main()@safe{ bang(hun); }
---
ref int foo()@safe{
struct S{
int x;
ref int bar()return{ return x; }
}
return S().bar();
}
// fun fact: paste it twice into a D file to get an ICE :-)
---
ref int foo()@safe{
int x;
struct S{
ref int bar(){ return x; }
}
return S().bar();
}
// (the DIP quite explicitly allows this,
// as x lives longer than 'this'.)
---
Basically every technique I have tried to use for escaping references to
locals and non-return parameters directly after reading the DIP has been
fruitful. This seems like a pretty obvious failure to achieve "with the
proposed rules, it is impossible in safe code to have ref refer to a
destroyed object" to me.
Had the DIP explicitly mentioned all necessary rules in more detail,
there would probably be fewer cases like this. Its true complexity would
then also be more apparent. (Be it low or high, but at least it would be
a leveled playing ground.)
What I see is a feature that solves one problem, and
solves it well: annotating a function that returns a reference to its
argument.
When you say "well", what do you mean?
My personal definition of "well" for such cases is that all the
contextual information needed to type check an expression can be encoded
into the type signature of a function. (Unfortunately, D violates this
rule quite often.) I don't think there is a syntax-free way of fixing
the holes which satisfies this, as the return attribute inherently
throws away e.g. the information about which nested scope a reference
returned after being accessed through a context pointer refers to.
The syntactic cost is low,
I have noticed you tend to attach a large weight to "syntactic cost".
How to evaluate syntactic cost of a feature? I think concepts that exist
but have no syntax have a larger cost.
and the impact on safety is positive.
Walter and I think it is the simplest solution of all considered.
I'm not entirely sure. It has the lowest amount of syntax though.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 23:08:58 UTC, Manu wrote: Incidentally, I tried to use shared_ptr initially, but it took about 20 minutes before I realised I had to pass an rc pointer from a method... Since rc is a wrapper, like you said above, you lose it as soon as you're within a method. And here you have it. You asked earlier how DIP25 and DIP74 connect, and this is where. If you want to ensure reference counting safety, you must make sure that reference do not escape in uncontrolled manner. I then had to write an invasive rc implementation and hard create a new rc instance from 'this', all over the place. Seriously, I don't understand how anyone can say shared_ptr is a success. It's a massive kludge, and it alone has me firmly convinced that rc is weak, inconvenient, and highly complicated without language support. By C++'s standards, this is not that messy.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 23:07:18 UTC, Ola Fosheim Gr wrote: On Sunday, 11 October 2015 at 22:33:44 UTC, Jonathan M Davis wrote: Well, the wrapper approach is no good (is it part of dip74?) since it messes up alignment etc, so the refcount interface should be part of the parent object. The child can be given the Refcount interface from dip74, it won't use it for a weak reference until it dereference (borrow). A weak reference just delays destruction until the borrowing is completed. DIP 74 proposes built-in reference counting. There is no wrapper type, and the compiler is able to elide increments and decrements of the ref count if it determines that they're unnecessary. If a type is ref-counted with DIP 74, it's always ref-counted, and it doesn't have anything for weak references (presumably leaving that up to the GC to take care of). get used incorrectly. But part of the whole point of DIP 74 is so that we can have @safe ref-counting in D. Well, @safe in D is broken and should just be dropped in favour of something sane like real pointer analysis instead of piling up bad designs like dip25. You only have to run it for release... Who cares if it is slow? @safe isn't going anywhere. And it mostly works just fine. It's primary flaw is that it's been done via blacklisting operations rather than whitelisting them, but that doesn't stop it from working. It just makes the implementation more error-prone. The only thing preventing the use of @safe in most cases is functions that haven't been updated to use it yet (in which case @trusted can be used). Walter and Andrei are completely behind @safe and are going to be in favor of fixing any holes in it that are found, not getting rid of it. And I don't see any reason why that can't work or that it's a bad idea. - Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 23:08:58 UTC, Manu wrote: Incidentally, I tried to use shared_ptr initially, but it took about 20 minutes before I realised I had to pass an rc pointer from a method... Since rc is a wrapper, like you said above, you lose it as soon as you're within a method. I then had to write an invasive rc implementation and hard create a new rc instance from 'this', all over the place. Seriously, I don't understand how anyone can say shared_ptr is a success. It's a massive kludge, and it alone has me firmly convinced that rc is weak, inconvenient, and highly complicated without language support. If you use shared_ptr, what typically happens is that you use shared_ptr everywhere and naked pointers pretty much disappear from your code. The main problem is when a type needs to return a pointer to itself, but in my own code, I've found that need to be rare (if anything, I'm more likely to pass an object out of its own member function by &, not by pointer). The primary counter-example is when a class that it owns needs to have access to it, but then in most cases, that class doesn't need to pass that pointer on to anyone else, and since it lives for as long as its parent does, it doesn't have to worry about ref-counting. I suppose that using & would work just as well in those cases, though I don't tend to use & much outside of passing into a function and returning out of it (generally when having a copy is okay but I want it to be possible to avoid it), because it's too easy to make inadvertent copies. Having ref-counting built into the type does solve some of those problems, but it can cause others. I doubt that there was ever any chance that the C++ standards committee would have ever gone with built-in ref-counting though given how invasive that is in the class hierarchy and how it would not work very well with existing code. In our case, with classes, built-in ref-counting like DIP 74 proposes is probably better. - Jonathan M Davis
Re: DIP74 - where is at?
On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote: So I have another upcoming opportunity to introduce D in my workplace, this time as a front-end/plugin language to our C++ infrastructure, which is promising since I already have considerable experience in this area (my work at Remedy with Quantum Break), and there is a lot of recent work to interact better with C++, which we will stress-test extensively. You only get so many shots at this; but this is a particularly promising opportunity, since the C++ code is a nightmare, and the contrast against D will allow a lot of coders to see the advantage. There is however one critical missing feature, DIP74... where is it at currently? How is it going? Is it likely to be accepted in the near-term? Some sort of approximate timeline? I think it would be a mistake for me to introduce this without DIP74, since we will rely on it VERY heavily, and the machinery to work-around it will start to look just as heavy-weight as the C++ code I'm trying to deprecate... but then waiting on it starts to look like missing the window of opportunity. Thoughts? I recently brought this up here: http://forum.dlang.org/thread/aetzsanbypaimljrg...@forum.dlang.org We discussed pros and cons, etc.. I restated my main concern here, and Walter seemed to agree: http://forum.dlang.org/post/mv29l2$1rln$1...@digitalmars.com So it seems there is hope for DIP74(or something similar), but it may take time. I asked Andrei when we may have another "D Vision" or similar document, and his response was here: http://forum.dlang.org/post/mv7is9$m41$1...@digitalmars.com Bit
Re: DIP74 - where is at?
Am Sun, 11 Oct 2015 07:32:26 + schrieb deadalnix : > In C++, you need to assume things are shared, and, as such, use > thread safe inc/dec . That means compiler won't be able to > optimize them. D can do better as sharedness is part of the type > system. With the little nag that `shared` itself is not fleshed out. -- Marco
Re: DIP74 - where is at?
On Monday, 12 October 2015 at 03:59:04 UTC, Marco Leise wrote:
Am Sun, 11 Oct 2015 07:32:26 +
schrieb deadalnix :
In C++, you need to assume things are shared, and, as such,
use thread safe inc/dec . That means compiler won't be able to
optimize them. D can do better as sharedness is part of the
type system.
With the little nag that `shared` itself is not fleshed out.
Well, it really should be better fleshed out, but the reality of
the matter is that it actually works pretty well as it is. The
problem is primarily that it's a pain to use - and to a certain
extent, that's actually a good thing, but it does make it harder
to use correctly. Better support for detecting when shared can be
safely cast away would be nice. e.g. being able to do something
like
synchronized(mutex_for_foo)
{
// foo is now implicitly treated as thread-local
}
// foo is now treated as shared again
would be nice. However, figuring out how to do that safely is
very tricky.
Ultimately though, I think that the problem with shared generally
comes down to folks not liking the fact that you pretty much
can't do anything with a shared variable until you cast away
shared, but the fact that it won't let you do much is actually
protecting you.
We do need to take another look at shared and see what we can do
to improve it, but I'm sure that it's ultimately going to be much
different from how it is now.
- Jonathan M Davis
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 23:46:42 UTC, Jonathan M Davis wrote: @safe isn't going anywhere. And it mostly works just fine. It's primary flaw is that it's been done via blacklisting operations rather than whitelisting them, but that doesn't stop it from working. It just makes the implementation more error-prone. Has nothing with whitelisting or blacklisting, that's basically the same thing for a finite set of features. The real problem is that it cannot be assumed to work until proven correct, formally. So it does not give you anything more than a weak sanitizer would have done until such proofs have been verified. The fact that it is difficult to be convince oneself that you don't have holes reinforces this viewpoint. If reasoning is difficult, you need proofs. Walter and Andrei are completely behind @safe and are going to be in favor of fixing any holes in it that are found, not getting rid of it. And I don't see any reason why that can't work or that it's a bad idea. This has been discussed before. @trusted regions can be correct in isolation, but it will make assumptions about what goes into @safe regions, so when the @safe region changes @trusted regions cannot assumed to be correct anymore. Therefore you need a prover to prove the @trusted region to remain safe whenever you change the surrounding @safe region. That means @trusted has to be truly exceptional and rare and impose verfiied restrictions on the assumptions it makes, which is not likely to happen. This comes in conflict with the very nature of system level programming.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 23:08:58 UTC, Manu wrote: Incidentally, I tried to use shared_ptr initially, but it took about 20 minutes before I realised I had to pass an rc pointer from a method... Since rc is a wrapper, like you said above, you lose it as soon as you're within a method. I then had to write an invasive rc implementation and hard create a new rc instance from 'this', all over the place. Seriously, I don't understand how anyone can say shared_ptr is a success. It's a massive kludge, and it alone has me firmly convinced that rc is weak, inconvenient, and highly complicated without language support. Shared_ptr and unique_ptr represent ownership, not references. So you should not pass them around unless you transfer ownership. When the owner allows someone to borrow a reference, that borrowing point has to ensure that the reference does not outlive the lifespan of the object. This is not so hard to get right. shared_ptr has several advantages: 1. It works with existing classes (from foreign libraries) without any notion of RC. 2. It allows weak references and cycles without keeping the whole object allocated. 3. It supports concurrency. 5. The reference counting mechanism can be done in a way that fits with polymorphism since it does not make any assumptions about the structure of object itself. 6. The object itself can remain fully immutable. Disadvantages: 1. double indirection 2. two allocations
Re: DIP74 - where is at?
On 10/12/15 1:44 AM, deadalnix wrote: On Sunday, 11 October 2015 at 20:35:05 UTC, Andrei Alexandrescu wrote: Could you please point to the document you have already written? For instance, we had a discussion with Walter and Mark that eventually yielded DIP25. In there, I made the following proposal : http://pastebin.com/LMkuTbgN This is an unstructured text. Could you please use it as a basis for a formal proposal? I made several other very detailed proposal. Where are they? Other did. It's not about me here. Others simply abandoned as far as I can tell. I'm just a stubborn idiot. At this point it would be great to just keep it calm and reduce inflammation. It won't achieve anything. There's a bit of a stalemate here. So we have: 1. You say that DIP25 is a failure. More so, you demand that is admitted without evidence. What I see is a feature that solves one problem, and solves it well: annotating a function that returns a reference to its argument. The syntactic cost is low, the impact on existing code is small, and the impact on safety is positive. Walter and I think it is the simplest solution of all considered. It is indeed the simplest. However, experiences that have been made and discussed in the forum showed it was often too simple to be really useful. I cited example of this, namely the RCArray thing and the existence of DIP74. I don't think the simplicity argument holds water in general as long as we don't have the whole thing. DIP25 + DIP74 + ... must be measured against the alternative. What is the alternative? Some handwaving asking to do ownership a la Rust cannot be analyzed. 2. You refuse to write a DIP under the assumption it will not be taken seriously. Conversely if you do write a DIP there is an expectation it will be approved just because you put work in it. I don't think rewarding work is the right way to go. We need to reward good work. The "work" part (i.e. a DIP) is a prerequisite; you can't demand to implement a complex feature based on posts and discussions. No that is inaccurate. I think I have evidence that it won't be taken seriously. To start with, there are already several DIP on the subject and they are not discussed at ALL. Namely : http://wiki.dlang.org/DIP35 http://wiki.dlang.org/DIP36 http://wiki.dlang.org/DIP69 http://wiki.dlang.org/DIP71 These do not even register as a blip on the radar. I don't see how adding my to the pile would change anything. Creating a DIP is no guarantee it will be approved, or discussed immediately. These in particular - I've been over most. I think DIP35 is not good. DIP36 I didn't look at yet, but was aware of it and will definitely do. DIP69 is obviously known to me because my name is on it. DIP71 is very sketchy and is not in reviewable form. There are not considered because DIP25 is "simpler" and you and Walter "like it". As long as nothing changes here, there is really no point in wasting my time. That is a fair assessment. Basically I believe DIP25 is good language design, and I have evidence for it. The evidence you showed failed to convince me the design is a hack, and yelling at me is unlikely to help. Please decide as you find fit. At some point it is clear that several language designers will disagree on estimating the quality of something. So I'm not sure how we can move forward from here. If you want to discuss DIP74, great, it can be discussed because it exists. My personal opinion on DIP74 is it has holes and corner cases so it seems it doesn't quite hit the spot. One option is to make it work, another is to take a different attack on the problem. But we need the appropriate DIP. Let's start by the beginning: what good design was enabled by DIP25 ? As long as none is presented, we can't consider it a success. Probably git grep in phobos may be a good starting point. Andrei
Re: DIP74 - where is at?
On 10/12/15 2:39 AM, Timon Gehr wrote: On 10/11/2015 10:35 PM, Andrei Alexandrescu wrote: ... 1. You say that DIP25 is a failure. More so, you demand that is admitted without evidence. FWIW, DIP25 is insufficiently formal and/or incorrect. That I agree with. We need to get a lot better at making precise DIPs. I have been able to find those holes in the implementation pretty quickly (I'll also put them to bugzilla): Thanks. That's great work! Andrei
Re: DIP74 - where is at?
On Monday, 12 October 2015 at 06:02:47 UTC, Andrei Alexandrescu wrote: There are not considered because DIP25 is "simpler" and you and Walter "like it". As long as nothing changes here, there is really no point in wasting my time. That is a fair assessment. Basically I believe DIP25 is good language design, and I have evidence for it. The evidence you showed failed to convince me the design is a hack, and yelling at me is unlikely to help. Please decide as you find fit. At some point it is clear that several language designers will disagree on estimating the quality of something. If you are wondering why I'm inflammatory, here you go. You are pulling me the old prove a negative trick. You have good evidence that DIP25 is good design ? Good, because I have none. And that's my proof. As long as I have no evidence that DIP25 is good, DIP25 is bad. Probably git grep in phobos may be a good starting point. I don't think grepping for return will have a good noise to signal ratio. You also mentioned several time that you have good evidence that DIP25 rox. Yet, every time you post that without any evidence, I'm a bit more convinced that none exists.
Re: DIP74 - where is at?
On 10/12/15 7:19 AM, Jonathan M Davis wrote: On Monday, 12 October 2015 at 03:59:04 UTC, Marco Leise wrote: Am Sun, 11 Oct 2015 07:32:26 + schrieb deadalnix : In C++, you need to assume things are shared, and, as such, use thread safe inc/dec . That means compiler won't be able to optimize them. D can do better as sharedness is part of the type system. With the little nag that `shared` itself is not fleshed out. Well, it really should be better fleshed out, but the reality of the matter is that it actually works pretty well as it is. The problem is primarily that it's a pain to use - and to a certain extent, that's actually a good thing, but it does make it harder to use correctly. Yah, I'd like to make "finalizing the language" a priority going forward, and finalizing shared is a big topic. It's hard to present to the world a language with fuzzy corners. -- Andrei
Re: DIP74 - where is at?
On 10/12/15 10:21 AM, deadalnix wrote: On Monday, 12 October 2015 at 06:02:47 UTC, Andrei Alexandrescu wrote: There are not considered because DIP25 is "simpler" and you and Walter "like it". As long as nothing changes here, there is really no point in wasting my time. That is a fair assessment. Basically I believe DIP25 is good language design, and I have evidence for it. The evidence you showed failed to convince me the design is a hack, and yelling at me is unlikely to help. Please decide as you find fit. At some point it is clear that several language designers will disagree on estimating the quality of something. If you are wondering why I'm inflammatory, here you go. You are pulling me the old prove a negative trick. You have good evidence that DIP25 is good design ? Good, because I have none. And that's my proof. As long as I have no evidence that DIP25 is good, DIP25 is bad. Instead of assuming my purpose here is to pull tricks on you and manipulate the dialog politically, it's more productive to just stick to the technical discussion. I only started the dialog to get more informed about technical things. Probably git grep in phobos may be a good starting point. I don't think grepping for return will have a good noise to signal ratio. Sorry, I meant to git grep for "return ref". You also mentioned several time that you have good evidence that DIP25 rox. Yet, every time you post that without any evidence, I'm a bit more convinced that none exists. The motivation is in the document and follows many discussions derived from it. It's all about functions and particularly member functions returning up references to data safely. Amaury, you and Timon are probably the most competent PL theorists in this forum. He did great work: found real problems with DIP25 that need to be looked at. In that light, spending time protesting and yelling figuratively at people is a distinctly unproductive way to spend your time as a very talented contributor. Do great work. It will be recognized. Don't point me at your past posts. They are not great work and you know it. Don't point me at those related DIPs. They are not great work and you know it. Don't find reasons to not do great work because it'll be wasted on my ego. Do great work and you will prevail. This tone of discussion has carried its course. I'm done arguing so if you want to continue arguing, great - last word is yours. In the recent times I've done my best to reduce my participation to unproductive discussions in forums, and the added perspective and time for real work have been very valuable. I suggest everyone to try it. Andrei
Re: DIP74 - where is at?
On Monday, 12 October 2015 at 07:44:47 UTC, Andrei Alexandrescu wrote: Instead of assuming my purpose here is to pull tricks on you and manipulate the dialog politically, it's more productive to just stick to the technical discussion. I only started the dialog to get more informed about technical things. I'm not assuming this. I'm fairly confident that you have the best intention for D in mind rather than winning internet debate points. So do I. Still you are pulling the above mentioned trick. i don't think you did it on purpose. I think you did it because you are convinced that DIP25 is great, in fact so convinced that you take thing for granted that aren't. It is a natural tendency we all have to lower the standard of evidence we require when we are already convinced. You, me, everybody. I don't think grepping for return will have a good noise to signal ratio. Sorry, I meant to git grep for "return ref". So I did. I found 2 uses of return ref in current master that aren't in unitests. These are both related to unicode decoding. These are legit use of DIP25, and I recognize their value. Still I don't think this is valuable enough to have its own syntax. I also, on the other hand presented cases of things that people thought could be done with DIP25, but ended up to be either not possible or to have a disappointingly low complexity to added value ratio. I can agree on something: DIP25 is the right direction. But it is too little to pay for itself and I haven't seen anything here that would suggest otherwize. You also mentioned several time that you have good evidence that DIP25 rox. Yet, every time you post that without any evidence, I'm a bit more convinced that none exists. The motivation is in the document and follows many discussions derived from it. It's all about functions and particularly member functions returning up references to data safely. Amaury, you and Timon are probably the most competent PL theorists in this forum. He did great work: found real problems with DIP25 that need to be looked at. In that light, spending time protesting and yelling figuratively at people is a distinctly unproductive way to spend your time as a very talented contributor. Do great work. It will be recognized. Don't point me at your past posts. They are not great work and you know it. Don't point me at those related DIPs. They are not great work and you know it. Don't find reasons to not do great work because it'll be wasted on my ego. Do great work and you will prevail. This tone of discussion has carried its course. I'm done arguing so if you want to continue arguing, great - last word is yours. In the recent times I've done my best to reduce my participation to unproductive discussions in forums, and the added perspective and time for real work have been very valuable. I suggest everyone to try it. Ok, I'll write a DIP.
Re: DIP74 - where is at?
Am Mon, 12 Oct 2015 10:44:47 +0300
schrieb Andrei Alexandrescu :
> >> Probably git grep in phobos may be a good starting point.
> >>
> >
> > I don't think grepping for return will have a good noise to signal
> > ratio.
>
> Sorry, I meant to git grep for "return ref".
AFAICS this returns only 6 results:
// false positive
std/functional.d:static ref int func_ref() { return refvar; }
// test functions for return ref
std/functional.d:ref int foo(return ref int a) { return a; }
std/traits.d:void test(scope int, ref int, out int, lazy
int, int, return ref int) { }
std/traits.d:ref const(Inner[string]) retfunc( return ref Inner
var1 );
// Real examples
// I don't see the advantage here. Simple example, private API anyway,
// not using @safe
std/stdio.d:private char[] takeFront(return ref char[4] buf)
// A good, but very simple example
std/utf.d:wchar[] toUTF16(return ref wchar[2] buf, dchar c) nothrow
@nogc @safe
So AFAICS it's not used much and the use cases are rather simple
compared to RCSlice / RC!T / Unique and all these complex cases that
have been discussed related to ownership.
Re: DIP74 - where is at?
On Sunday, 11 October 2015 at 20:35:05 UTC, Andrei Alexandrescu wrote: On 10/11/15 9:57 PM, deadalnix wrote: On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote: On Sunday, 11 October 2015 at 13:51:18 UTC, Andrei Alexandrescu wrote: Walter and I are happy with DIP25, and the fact of the matter is we weren't surprised more complementary work is needed. So no, I won't acknowledge what I don't believe. That is an empty statement. What is there to be happy about ? Also the complementary argument pretty much destroy the best argument you and Walter made for DIP25 : it is simple. I mean, one need to look at the big picture. DIP25 + complementary addition is not simple anymore. I'd say the one way to get things looked at seriously is to create a DIP. That's no guarantee it will be accepted but there is a guarantee that our chat at DConf is not sufficient even as a basis for further study. Yeah there are IRL discussion, there are many posts in the forum, there are by mail discussions at DIP25 creation time, there are at least one DIP already. The only rebuttal to all of this is "Walter and I are happy with DIP25, and the fact of the matter", while everybody else is wondering what there is to be happy about. Also, I'm sorry but there is no me writing once again a document about what alternative are possible. Could you please point to the document you have already written? Spending hours to write documents so that one is answered something along the line of "we are happy with the other thing, but we can't give any reason why" is something I've engaged in several time in already, and has no desire to indulge into this if I have reason to think the same will happen. Your answer here are telling me one thing: it won't be taken seriously. There's a bit of a stalemate here. So we have: 1. You say that DIP25 is a failure. More so, you demand that is admitted without evidence. What I see is a feature that solves one problem, and solves it well: annotating a function that returns a reference to its argument. The syntactic cost is low, the impact on existing code is small, and the impact on safety is positive. Walter and I think it is the simplest solution of all considered. Except that it isn't a solution to the problems it was claimed to solve. For example, Walter tried to build a safe RCArray implementation with it, and it turned out that it's still unsafe in the presence of aliasing. I still see DIP25 as going in the right direction, but it from my POV it must at least a) be usable with all kinds of references, not just `ref`, and without double indirections, b) solve the problems with global variables (probably easy, just make them unsafe), and c) keep track of aliases or otherwise handle them correctly to avoid the RCArray problems. (An additional nice-to-have feature would be a method to make a variable inaccessible that can be enforced at compile time, because that will enable various uniqueness-related things. This is closely related, but not a necessity.) 2. You refuse to write a DIP under the assumption it will not be taken seriously. Conversely if you do write a DIP there is an expectation it will be approved just because you put work in it. I don't think rewarding work is the right way to go. We need to reward good work. The "work" part (i.e. a DIP) is a prerequisite; you can't demand to implement a complex feature based on posts and discussions. The problem is the signals we get from you and Walter. From various posts (or lack of response to certain questions) and the way you've treated this entire topic so far, I got the impression that you both are opposed to anything similar to Rust's approach. Unfortunately, we know that Rust's approach (or other solutions involving linear type systems) is the only thing that can provide a compiler-verifiable free-at-runtime solution. That's the real stalemate as I see it. I hope you see how that's not particularly motivating.
Re: DIP74 - where is at?
On 10/12/15 4:38 PM, Marc Schütz wrote: The problem is the signals we get from you and Walter. From various posts (or lack of response to certain questions) and the way you've treated this entire topic so far, I got the impression that you both are opposed to anything similar to Rust's approach. Unfortunately, we know that Rust's approach (or other solutions involving linear type systems) is the only thing that can provide a compiler-verifiable free-at-runtime solution. That's the real stalemate as I see it. I hope you see how that's not particularly motivating. Yes, I agree. Experience with Rust is still young, but there seems to have already been a backlash; programmers try it and it's just too arcane to use in constant preoccupation about them ownership rules. Copying linear types and going whole-hog stealing the ownership system from Rust doesn't sound like the best strategy to Walter and myself. At least one PL researcher whose opinion I trust believes linear types don't have a future. D has its own context and its own approach to matters. I believe creative solutions are possible that achieve much of what we need without going the Rust way, which seems not appropriate for us. I agree that DIP74 has problems; it has had several corner cases that needed patching. That's a signal DIP74 doesn't quite cut with the grain. What we need here is good creative work that takes us where we want to be without breaking the complexity bank. BTW where we want to be is: expressive reference counting without loss of @safe-ty and hopefully good uniqueness control again with @safe-ty and possibly a bit of compiler help. Andrei
Re: DIP74 - where is at?
On Monday, 12 October 2015 at 07:21:58 UTC, deadalnix wrote: On Monday, 12 October 2015 at 06:02:47 UTC, Andrei Alexandrescu wrote: There are not considered because DIP25 is "simpler" and you and Walter "like it". As long as nothing changes here, there is really no point in wasting my time. That is a fair assessment. Basically I believe DIP25 is good language design, and I have evidence for it. The evidence you showed failed to convince me the design is a hack, and yelling at me is unlikely to help. Please decide as you find fit. At some point it is clear that several language designers will disagree on estimating the quality of something. If you are wondering why I'm inflammatory, here you go. You are pulling me the old prove a negative trick. You have good evidence that DIP25 is good design ? Good, because I have none. And that's my proof. As long as I have no evidence that DIP25 is good, DIP25 is bad. I've noticed that you seem to be quite arrogant. Usually it is a result of ignorance. Your statement basically proves that. Maybe you should take a break from programming for a while and work on your attitude? While you have no proof of this, If you do a little soul searching you'll find that the world doesn't revolve around you. Put down your toys and get out of the sandbox and you might learn something!
Re: DIP74 - where is at?
On Monday, 12 October 2015 at 16:56:27 UTC, Jacob wrote: I've noticed that you seem to be quite arrogant. Usually it is a result of ignorance. Your statement basically proves that. Maybe you should take a break from programming for a while and work on your attitude? While you have no proof of this, If you do a little soul searching you'll find that the world doesn't revolve around you. Put down your toys and get out of the sandbox and you might learn something! Dude you are kind of being a jerk. He's just arguing against, rather passionately, a design decision he thinks is poor. It values no one if individuals remain quiet and conform. Certainly not the leadership. What you see as arrogance is really just passion.
Re: DIP74 - where is at?
Am Mon, 12 Oct 2015 10:28:55 +0300 schrieb Andrei Alexandrescu : > On 10/12/15 7:19 AM, Jonathan M Davis wrote: > > On Monday, 12 October 2015 at 03:59:04 UTC, Marco Leise wrote: > >> Am Sun, 11 Oct 2015 07:32:26 + > >> schrieb deadalnix : > >> > >>> In C++, you need to assume things are shared, and, as such, use > >>> thread safe inc/dec . That means compiler won't be able to optimize > >>> them. D can do better as sharedness is part of the type system. > >> > >> With the little nag that `shared` itself is not fleshed out. > > > > Well, it really should be better fleshed out, but the reality of the > > matter is that it actually works pretty well as it is. The problem is > > primarily that it's a pain to use - and to a certain extent, that's > > actually a good thing, but it does make it harder to use correctly. > > Yah, I'd like to make "finalizing the language" a priority going > forward, and finalizing shared is a big topic. It's hard to present to > the world a language with fuzzy corners. -- Andrei Wouldn't it be great if everyone took notes of the currently perceived shortcomings of shared so that there is a pile of use- and corner-cases to look at for a redesign? Then you would filter by valid use case and undesired usage and practically had everyone's input already under consideration when it hits the discussion forums. My own experience is what Jonathan describes. It is annoying in a good way. It makes you wait and think twice and more often than not you really attempted an illegal access on shared data. This went well up to a point when I had a component that held references to shared things. There is no such thing as a shared struct with shared fields in the type system. Once you share the outer struct, the inner stuff's shared status is merged with the outer and you can't cast it back to what it was before. So it works for plain integral types, but aggregates and what others mentioned about thread local GC heaps are my big question marks. -- Marco
Re: DIP74 - where is at?
On 10/12/2015 06:49 PM, Andrei Alexandrescu wrote: Experience with Rust is still young, but there seems to have already been a backlash; programmers try it and it's just too arcane to use in constant preoccupation about them ownership rules. ... Copying linear types and going whole-hog stealing the ownership system from Rust doesn't sound like the best strategy to Walter and myself. D can do ownership without copying the details of the Rust way. I think that Marc's point was that basic similarities should be allowed, because the design space isn't that vast. At least one PL researcher whose opinion I trust believes linear types don't have a future. We already have linear types. (@disable this(this)). D has its own context and its own approach to matters. I believe creative solutions are possible that achieve much of what we need without going the Rust way, which seems not appropriate for us. What do we need?
Re: DIP74 - where is at?
On 10/13/15 3:09 AM, Timon Gehr wrote: On 10/12/2015 06:49 PM, Andrei Alexandrescu wrote: Experience with Rust is still young, but there seems to have already been a backlash; programmers try it and it's just too arcane to use in constant preoccupation about them ownership rules. ... Copying linear types and going whole-hog stealing the ownership system from Rust doesn't sound like the best strategy to Walter and myself. D can do ownership without copying the details of the Rust way. I think that Marc's point was that basic similarities should be allowed, because the design space isn't that vast. At least one PL researcher whose opinion I trust believes linear types don't have a future. We already have linear types. (@disable this(this)). That much is not too complicated. D has its own context and its own approach to matters. I believe creative solutions are possible that achieve much of what we need without going the Rust way, which seems not appropriate for us. What do we need? Next paragraph after the one you quoted: BTW where we want to be is: expressive reference counting without loss of @safe-ty and hopefully good uniqueness control again with @safe-ty and possibly a bit of compiler help. Andrei
Re: DIP74 - where is at?
On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload wrote: On Monday, 12 October 2015 at 16:56:27 UTC, Jacob wrote: I've noticed that you seem to be quite arrogant. Usually it is a result of ignorance. Your statement basically proves that. Maybe you should take a break from programming for a while and work on your attitude? While you have no proof of this, If you do a little soul searching you'll find that the world doesn't revolve around you. Put down your toys and get out of the sandbox and you might learn something! Dude you are kind of being a jerk. He's just arguing against, rather passionately, a design decision he thinks is poor. It values no one if individuals remain quiet and conform. Certainly not the leadership. What you see as arrogance is really just passion. Passion or not, "If you are wondering why I'm inflammatory, here you go. You are pulling me the old prove a negative trick. You have good evidence that DIP25 is good design ? Good, because I have none. And that's my proof. As long as I have no evidence that DIP25 is good, DIP25 is bad." That statement shows a lot of arrogance. It he not really just saying "I have no clue if X is true, but since I don't know, I'll just assume it's false and assume you are wrong.". That's not very logical. Why wouldn't he just as well assume X is true? The fact is, he can't and shouldn't make such statements about X if he has no "evidence" about it. Instead, wouldn't the proper approach be to discuss, learn, and share what one things in a positive way so everyone can learn about X and reach a more intelligent understanding of it? Hilter was very passionate too, are you saying he was right?
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote: On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload wrote: On Monday, 12 October 2015 at 16:56:27 UTC, Jacob wrote: I've noticed that you seem to be quite arrogant. Usually it is a result of ignorance. Your statement basically proves that. Maybe you should take a break from programming for a while and work on your attitude? While you have no proof of this, If you do a little soul searching you'll find that the world doesn't revolve around you. Put down your toys and get out of the sandbox and you might learn something! Dude you are kind of being a jerk. He's just arguing against, rather passionately, a design decision he thinks is poor. It values no one if individuals remain quiet and conform. Certainly not the leadership. What you see as arrogance is really just passion. Passion or not, "If you are wondering why I'm inflammatory, here you go. You are pulling me the old prove a negative trick. You have good evidence that DIP25 is good design ? Good, because I have none. And that's my proof. As long as I have no evidence that DIP25 is good, DIP25 is bad." That statement shows a lot of arrogance. It he not really just saying "I have no clue if X is true, but since I don't know, I'll just assume it's false and assume you are wrong.". That's not very logical. Why wouldn't he just as well assume X is true? The context is that all additions have an intrinsic cost because they increase complexity and restrict future design choices, therefore the onus is on any change/addition to justify its value in order to overcome that cost. I think you are completely misreading the situation, or trolling. This is just
Re: DIP74 - where is at?
On Monday, 12 October 2015 at 19:35:34 UTC, Marco Leise wrote: Wouldn't it be great if everyone took notes of the currently perceived shortcomings of shared so that there is a pile of use- and corner-cases to look at for a redesign? The problem with shared is that shared should not be constant over time, it should be related to behavioural typing/type state. Synchronization is a temporal factor not a static typing factor.
Re: DIP74 - where is at?
On Monday, 12 October 2015 at 08:10:44 UTC, deadalnix wrote: Ok, I'll write a DIP. That would be great to see. N.B. it's not only about Walter and Andrei: with a detailed, written-up proposal in place, the rest of us can examine it and, assuming we like it, lend our vocal support to the idea.
Re: DIP74 - where is at?
On 12 October 2015 at 16:02, Andrei Alexandrescu via Digitalmars-d
wrote:
>
> DIP69 is obviously known to me because my name is on it.
What is the problem with DIP69? It looks like a great direction to me.
It's almost exactly what I've been begging for for years!
As long as methods can overload on scope, then it can be used to
implement efficient RC. Also many other situations are improved in
general.
The only thing there that's not clear to me from DIP69 is:
scope ref T func(scope ref T x) { return x; }
This needs to work, or you can't chain... but it doesn't look like
it's addressed?
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 12:40:46 UTC, Ola Fosheim Grøstad wrote: On Monday, 12 October 2015 at 19:35:34 UTC, Marco Leise wrote: Wouldn't it be great if everyone took notes of the currently perceived shortcomings of shared so that there is a pile of use- and corner-cases to look at for a redesign? The problem with shared is that shared should not be constant over time, it should be related to behavioural typing/type state. Synchronization is a temporal factor not a static typing factor. That said, if you have: 1. writer-ownership as a feature 2. mark variables as "only writable by one owner" Then the compiler can drop some read locks for the _owning_ thread. But D does not have ownership as a feature beyond "thread local memory"?
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 13:02:43 UTC, Ola Fosheim Grøstad wrote: That said, if you have: 1. writer-ownership as a feature 2. mark variables as "only writable by one owner" Then the compiler can drop some read locks for the _owning_ thread. But D does not have ownership as a feature beyond "thread local memory"? I think the D designers should take a long and hard look at Pony: isolated: deny global read/write, deny local read/write transition: deny global read/write, deny local write reference: deny global read/write value: deny global write, deny local write box: deny global write aliases tag: allow all aliases _loosely_ translated: isolated: void* transition: const T* reference: T* value: immutable T* box: globally as shared const T*, locally as shared T* tag: shared T* So D lacks some way to express the "box" type?
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote: On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload wrote: On Monday, 12 October 2015 at 16:56:27 UTC, Jacob wrote: I've noticed that you seem to be quite arrogant. Usually it is a result of ignorance. Your statement basically proves that. Maybe you should take a break from programming for a while and work on your attitude? While you have no proof of this, If you do a little soul searching you'll find that the world doesn't revolve around you. Put down your toys and get out of the sandbox and you might learn something! Dude you are kind of being a jerk. He's just arguing against, rather passionately, a design decision he thinks is poor. It values no one if individuals remain quiet and conform. Certainly not the leadership. What you see as arrogance is really just passion. Passion or not, "If you are wondering why I'm inflammatory, here you go. You are pulling me the old prove a negative trick. You have good evidence that DIP25 is good design ? Good, because I have none. And that's my proof. As long as I have no evidence that DIP25 is good, DIP25 is bad." That statement shows a lot of arrogance. It he not really just saying "I have no clue if X is true, but since I don't know, I'll just assume it's false and assume you are wrong.". That's not very logical. Why wouldn't he just as well assume X is true? Because people with half a brain know that's not how it works. Proves me that unicorn do not exists. I'm waiting. And remember, having no evidence that they do exists doesn't mean they do not ! Hilter was very passionate too, are you saying he was right? ICH BIN EIN POLYNOMIAL!
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote: On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload wrote: [...] Passion or not, "If you are wondering why I'm inflammatory, here you go. You are pulling me the old prove a negative trick. You have good evidence that DIP25 is good design ? Good, because I have none. And that's my proof. As long as I have no evidence that DIP25 is good, DIP25 is bad." That statement shows a lot of arrogance. It he not really just saying "I have no clue if X is true, but since I don't know, I'll just assume it's false and assume you are wrong.". That's not very logical. Why wouldn't he just as well assume X is true? The fact is, he can't and shouldn't make such statements about X if he has no "evidence" about it. Instead, wouldn't the proper approach be to discuss, learn, and share what one things in a positive way so everyone can learn about X and reach a more intelligent understanding of it? Hilter was very passionate too, are you saying he was right? Since you brought up the Hilter reference, it's useful to mention that he was successful because people blindly followed him and his actions... But really...do we always have to resort to Hitler references when discussing disagreements? Seems very cliche.
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 17:59:28 UTC, deadalnix wrote: Hilter was very passionate too, are you saying he was right? ICH BIN EIN POLYNOMIAL! As this thread has run it course starting with the Hitler comparison, and is therefor OT. You have to explain to me, why your are a polynomial :-)
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 18:18:46 UTC, Robert burner Schadek wrote: On Tuesday, 13 October 2015 at 17:59:28 UTC, deadalnix wrote: Hilter was very passionate too, are you saying he was right? ICH BIN EIN POLYNOMIAL! As this thread has run it course starting with the Hitler comparison, and is therefor OT. You have to explain to me, why your are a polynomial :-) That's a reference to The Oatmeal : http://theoatmeal.com/comics/atheism
Re: DIP74 - where is at?
Am Tue, 13 Oct 2015 17:59:26 + schrieb deadalnix : > > It he not really just saying "I have no clue if X is true, but > > since I don't know, I'll just assume it's false and assume you > > are wrong.". > > > > That's not very logical. Why wouldn't he just as well assume X > > is true? > > > > Because people with half a brain know that's not how it works. > > Proves me that unicorn do not exists. I'm waiting. And remember, > having no evidence that they do exists doesn't mean they do not ! Ok, so here we arrived in d.religion. Today: "Agnostic vs. atheist, who is right." And: "Testimony: I tried to change the world but God didn't give me the source code." -- Marco
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 18:23:05 UTC, Marco Leise wrote: Ok, so here we arrived in d.religion. Today: "Agnostic vs. atheist, who is right." And: "Testimony: I tried to change the world but God didn't give me the source code." Well, I talk about D-ifying code sometimes, but saying that out loud is problematic... - Jonathan M Davis
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 18:24:13 UTC, deadalnix wrote: That's a reference to The Oatmeal : http://theoatmeal.com/comics/atheism thanks
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 16:04:07 UTC, Ola Fosheim Grøstad wrote: _loosely_ translated: isolated: void* transition: const T* reference: T* value: immutable T* box: globally as shared const T*, locally as shared T* tag: shared T* The above turned out rather allegorical (and possibly confusing), the following is a lockfree interpretation that is a little bit more useful and somewhat more accurate: ISOLATED: Unique!T referable as void*, shared void* transferrable to other threads TRANSITION: Unique!T referable as const T*, void*, shared void* convertible to immutable T* REFERENCE: T* referable as T*, const T*, void*, shared void* VALUE: immutable T* referable as immutable T*, const T*, shared const T*, void*, shared void* shareable with other threads BOX: const T* referable as const T*, void*, shared void* aliasing with T*, immutable T* TAG: void T* or shared void* referable as void T*, shared void* aliasing with all types as it is a pure identity Then we can do a new interpretation with locking/synchronization and get something like: LOCKED BOX: shared const T* referable as shared const T*, void*, shared void* aliasing with shared T*, immutable T* LOCKED TAG: shared T* referable as shared T*, shared const T*, void T*, shared void* More or less...
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 18:13:33 UTC, Ice Cream Overload wrote: On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote: On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload wrote: [...] Passion or not, "If you are wondering why I'm inflammatory, here you go. You are pulling me the old prove a negative trick. You have good evidence that DIP25 is good design ? Good, because I have none. And that's my proof. As long as I have no evidence that DIP25 is good, DIP25 is bad." That statement shows a lot of arrogance. It he not really just saying "I have no clue if X is true, but since I don't know, I'll just assume it's false and assume you are wrong.". That's not very logical. Why wouldn't he just as well assume X is true? The fact is, he can't and shouldn't make such statements about X if he has no "evidence" about it. Instead, wouldn't the proper approach be to discuss, learn, and share what one things in a positive way so everyone can learn about X and reach a more intelligent understanding of it? Hilter was very passionate too, are you saying he was right? Since you brought up the Hilter reference, it's useful to mention that he was successful because people blindly followed him and his actions... But really...do we always have to resort to Hitler references when discussing disagreements? Seems very cliche. It's only cliche if you aren't interested in the truth. It doesn't matter if I used Hilter or any other person that was "passionate" but wrong. Hilter is just the greatest example and most obvious example. If you actually understand the issue, you would realize it has nothing to do with Hilter.
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 21:17:43 UTC, Jacob wrote: On Tuesday, 13 October 2015 at 18:13:33 UTC, Ice Cream Overload wrote: On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote: On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload wrote: [...] Passion or not, "If you are wondering why I'm inflammatory, here you go. You are pulling me the old prove a negative trick. You have good evidence that DIP25 is good design ? Good, because I have none. And that's my proof. As long as I have no evidence that DIP25 is good, DIP25 is bad." That statement shows a lot of arrogance. It he not really just saying "I have no clue if X is true, but since I don't know, I'll just assume it's false and assume you are wrong.". That's not very logical. Why wouldn't he just as well assume X is true? The fact is, he can't and shouldn't make such statements about X if he has no "evidence" about it. Instead, wouldn't the proper approach be to discuss, learn, and share what one things in a positive way so everyone can learn about X and reach a more intelligent understanding of it? Hilter was very passionate too, are you saying he was right? Since you brought up the Hilter reference, it's useful to mention that he was successful because people blindly followed him and his actions... But really...do we always have to resort to Hitler references when discussing disagreements? Seems very cliche. It's only cliche if you aren't interested in the truth. It doesn't matter if I used Hilter or any other person that was "passionate" but wrong. Hilter is just the greatest example and most obvious example. If you actually understand the issue, you would realize it has nothing to do with Hilter. I don't know. Whenever someone runs out of arguments and is forced to go on the attack, Hitler seems to be the first insult people reach for. Thus once I see Hitler references pop up, I throw out that individual's credibility. Solves bandwidth problems of who is worth listening to. Some soul searching for you, is I'd refrain from such references as it only hurts your credibility. Not the one you sling it at.
Re: DIP74 - where is at?
On Tue, Oct 13, 2015 at 09:37:27PM +, I SCREAM for ICECream via Digitalmars-d wrote: > On Tuesday, 13 October 2015 at 21:17:43 UTC, Jacob wrote: > >On Tuesday, 13 October 2015 at 18:13:33 UTC, Ice Cream Overload wrote: > >>On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote: > >>>On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload wrote: > [...] > >>> > >>>Passion or not, > >>> > >>>"If you are wondering why I'm inflammatory, here you go. You are > >>>pulling me the old prove a negative trick. You have good evidence > >>>that DIP25 is good design ? Good, because I have none. And that's > >>>my proof. As long as I have no evidence that DIP25 is good, DIP25 > >>>is bad." > >>> > >>>That statement shows a lot of arrogance. > >>> > >>>It he not really just saying "I have no clue if X is true, but > >>>since I don't know, I'll just assume it's false and assume you are > >>>wrong.". > >>> > >>>That's not very logical. Why wouldn't he just as well assume X is > >>>true? > >>> > >>>The fact is, he can't and shouldn't make such statements about X if > >>>he has no "evidence" about it. > >>> > >>>Instead, wouldn't the proper approach be to discuss, learn, and > >>>share what one things in a positive way so everyone can learn about > >>>X and reach a more intelligent understanding of it? > >>> > >>>Hilter was very passionate too, are you saying he was right? > >> > >>Since you brought up the Hilter reference, it's useful to mention > >>that he was successful because people blindly followed him and his > >>actions... > >> > >>But really...do we always have to resort to Hitler references when > >>discussing disagreements? Seems very cliche. > > > >It's only cliche if you aren't interested in the truth. It doesn't > >matter if I used Hilter or any other person that was "passionate" but > >wrong. > > > >Hilter is just the greatest example and most obvious example. If you > >actually understand the issue, you would realize it has nothing to do > >with Hilter. > > I don't know. Whenever someone runs out of arguments and is forced to > go on the attack, Hitler seems to be the first insult people reach > for. Thus once I see Hitler references pop up, I throw out that > individual's credibility. Solves bandwidth problems of who is worth > listening to. > > Some soul searching for you, is I'd refrain from such references as it > only hurts your credibility. Not the one you sling it at. Godwin's Law[1] has been invoked, boys and girls. The game is now over. Thanks for playing. You may go home now. Have a nice day. [1] https://en.wikipedia.org/wiki/Godwin's_law T -- "I'm not childish; I'm just in touch with the child within!" - RL
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 21:46:22 UTC, H. S. Teoh wrote: On Tue, Oct 13, 2015 at 09:37:27PM +, I SCREAM for ICECream via Digitalmars-d wrote: On Tuesday, 13 October 2015 at 21:17:43 UTC, Jacob wrote: >[...] I don't know. Whenever someone runs out of arguments and is forced to go on the attack, Hitler seems to be the first insult people reach for. Thus once I see Hitler references pop up, I throw out that individual's credibility. Solves bandwidth problems of who is worth listening to. Some soul searching for you, is I'd refrain from such references as it only hurts your credibility. Not the one you sling it at. Godwin's Law[1] has been invoked, boys and girls. The game is now over. Thanks for playing. You may go home now. Have a nice day. [1] https://en.wikipedia.org/wiki/Godwin's_law T Amazingly true :D
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 21:17:43 UTC, Jacob wrote: It's only cliche if you aren't interested in the truth. It doesn't matter if I used Hilter or any other person that was "passionate" but wrong. You are right, but Andrei and Walter often go into "passionate but wrong" mode too... It's a curse of D and probably will keep it from reaching a mature state. DIP25 is no exception. In language design it is better to have a small set of features in the core language that are easy to reason about as a whole. D has many simple features, but the combinatorial explosion is quite high. For instance, how is DIP25 going to work with coroutines that yield? So you transfer a refererence by "return ref", then what you call yields and the object is destructed. When the coroutine is later resumed the object no longer exists, so you have a memory unsafe situation. So then you have to add the requirement that "return ref" functions cannot call anything that yields... After some time you realize that it is possible to pass in a lambda that can destroy the object. Then you forbid passing in lambdas... What one should have realized is that if reasoning about correctness isn't obvious then you need proofs. There is no obvious memory safety in D and there are no proofs. Meaning, you would be better off using a general static analyzer because you get more flexibility and the same level of memory safety.
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 21:46:22 UTC, H. S. Teoh wrote: Godwin's Law[1] has been invoked, boys and girls. The game is now over. Thanks for playing. You may go home now. Have a nice day. Neh, Godwin's law only states that as time progresses the probability of invoking "Hitler" as an example increases, but it says nothing about whether the comparison was useful or not.
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 22:00:28 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 13 October 2015 at 21:46:22 UTC, H. S. Teoh wrote: Godwin's Law[1] has been invoked, boys and girls. The game is now over. Thanks for playing. You may go home now. Have a nice day. Neh, Godwin's law only states that as time progresses the probability of invoking "Hitler" as an example increases, but it says nothing about whether the comparison was useful or not. The D language, does have a 'feature' creep problem.
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 22:20:02 UTC, Jonathan M Davis wrote: On Tuesday, 13 October 2015 at 22:10:30 UTC, Ice Cream Madness wrote: The D language, does have a 'feature' creep problem. Maybe, but at this point, I think that C++ is actually getting features faster than D is. We talk about adding features or tweaking existing features to fix problems, but we're long past the point where we're frequently adding features. If anything, the typical complaint now is that we're _not_ making changes, even when we've been discussing them (e.g. this very thread was started to ask what the deal with a proposed change is, because it hasn't gone anywhere yet, and it's a change proposed by Walter and Andrei no less). - Jonathan M Davis C++ also has a 'feature' creep problem.
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 22:10:30 UTC, Ice Cream Madness wrote: The D language, does have a 'feature' creep problem. Maybe, but at this point, I think that C++ is actually getting features faster than D is. We talk about adding features or tweaking existing features to fix problems, but we're long past the point where we're frequently adding features. If anything, the typical complaint now is that we're _not_ making changes, even when we've been discussing them (e.g. this very thread was started to ask what the deal with a proposed change is, because it hasn't gone anywhere yet, and it's a change proposed by Walter and Andrei no less). - Jonathan M Davis
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 22:20:02 UTC, Jonathan M Davis wrote: Maybe, but at this point, I think that C++ is actually getting features faster than D is. And as a result advanced c++ analyzers work on lowlevel IR, not at the language level. That has many consequences, one is time consumption, another is accuracy. We talk about adding features or tweaking existing features to fix problems, but we're long past the point where we're frequently adding features. More syntax sugar is not problematic. It is what you have left when you factor out everything that is expressible by other mechanisms that matters.
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 22:21:24 UTC, Ice Cream Desserter wrote: On Tuesday, 13 October 2015 at 22:20:02 UTC, Jonathan M Davis wrote: On Tuesday, 13 October 2015 at 22:10:30 UTC, Ice Cream Madness wrote: The D language, does have a 'feature' creep problem. Maybe, but at this point, I think that C++ is actually getting features faster than D is. We talk about adding features or tweaking existing features to fix problems, but we're long past the point where we're frequently adding features. If anything, the typical complaint now is that we're _not_ making changes, even when we've been discussing them (e.g. this very thread was started to ask what the deal with a proposed change is, because it hasn't gone anywhere yet, and it's a change proposed by Walter and Andrei no less). - Jonathan M Davis C++ also has a 'feature' creep problem. If I may the problem is not that much how feature rich the language is, but how orthogonal these feature are. The less orthogonal they are, the more special cases you get when they interact with each other.
Re: DIP74 - where is at?
On Tuesday, 13 October 2015 at 22:20:02 UTC, Jonathan M Davis wrote: On Tuesday, 13 October 2015 at 22:10:30 UTC, Ice Cream Madness wrote: The D language, does have a 'feature' creep problem. Maybe, but at this point, I think that C++ is actually getting features faster than D is. We talk about adding features or tweaking existing features to fix problems, but we're long past the point where we're frequently adding features. If anything, the typical complaint now is that we're _not_ making changes, even when we've been discussing them (e.g. this very thread was started to ask what the deal with a proposed change is, because it hasn't gone anywhere yet, and it's a change proposed by Walter and Andrei no less). - Jonathan M Davis DIP74 is also just an alternative solution to a problem that has a long history, and nothing is being done about it.
