Re: forcing @nogc on class destructors
On Wednesday, 28 January 2015 at 11:32:26 UTC, Ola Fosheim Grøstad wrote: But I think GC and destructors are not a good match... so I would personally go for performance and no destructors. Yes. That is the bottom line. A good match for GC is the phantom reference, which notifies you when an object gets collected with no way to resurrect the collected object or access its instance variables.
Re: forcing @nogc on class destructors
On Friday, 30 January 2015 at 00:28:18 UTC, Jerry Morrison wrote: On Wednesday, 28 January 2015 at 11:32:26 UTC, Ola Fosheim Grøstad wrote: But I think GC and destructors are not a good match... so I would personally go for performance and no destructors. Yes. That is the bottom line. A good match for GC is the phantom reference, which notifies you when an object gets collected with no way to resurrect the collected object or access its instance variables. That is more important than the nogc thing, as it has real impact of definition and implementation. This is the kind of thing we can solve with proper lifetime definition.
Re: forcing @nogc on class destructors
On Wednesday, 28 January 2015 at 02:33:11 UTC, Jerry Morrison wrote: Sorry, I don't understand the question. I expect LIFO for freeing structs on the stack. Yes, but if the class is designed for unique_ptr style usage... Anyway, a destructor is the END for the constructor BEGIN. Together they form a scope. The children of the scope ought to live until the END. The background for constructor/destructor is that they match a closure. Simula had the INNER keyword so that you could write: CLASS Transaction BEGIN ...open... INNER ...close... END Then specialize it by prefix style subclassing: Transaction BEGIN ...do stuff executed in the inner clause... END C++ offer similar semantics with constructor/destructor RAII... Some languages keep track of parent-child relationships, you can do it in the typing even. Nevertheless, children ought to be alive when the parent dies... If the language cannot provide this, then provide another mechanism such as finalize or just disallow GC allocating destructor based classes. Mish-mashing established programming concepts is Not a Good Idea. :)
Re: forcing @nogc on class destructors
On Wednesday, 28 January 2015 at 09:51:09 UTC, Ola Fosheim Grøstad wrote: Some languages keep track of parent-child relationships, you can do it in the typing even. Nevertheless, children ought to be alive when the parent dies... If the language cannot provide this, then provide another mechanism such as finalize or just disallow GC allocating destructor based classes. Mish-mashing established programming concepts is Not a Good Idea. :) But wouldn't enforcing strict parent-child relationships make cyclic references illegal ?
Re: forcing @nogc on class destructors
On Wednesday, 28 January 2015 at 10:36:11 UTC, Matthias Bentrup wrote: But wouldn't enforcing strict parent-child relationships make cyclic references illegal ? You would either need: - serial number/timestamping (just a sort is required) - dynamic information like a back pointer - static typing like an ownership reference type - behavioural typing (e.g. proper child references are inferred by looking at what is done in the constructor) But I think GC and destructors are not a good match... so I would personally go for performance and no destructors. But unions also have to be turned into strongly typed constructs for GC allocation if they can contain reference scanned memory. Behavioural typing could go a long way.
Re: forcing @nogc on class destructors
On Saturday, 24 January 2015 at 23:28:35 UTC, Jerry Morrison wrote: This is the first I've heard that allocating GC memory in a destructor will crash. That's an unexpected gotcha. I'd expect to be able to reliably do I/O or throw an exception. Strategy 1. Fix the GC's limitation. (One fewer pitfall to baby-sit.) I expect this to happen sooner rather than later. GC is currently a very hot topic, with several people working on different aspects of it (performance, preciseness...). Strategy 2. Have the compiler inform the programmer. (The compiler can't catch all bugs but it should do what it can. Arguably this is a GC bug, not mine.) Might be worthwhile if we don't manage to remove the GC limitation before, say, 2.068. But it will also trigger some false positives. It's unfortunate that currently the class/struct distinction is tightly coupled with the memory management strategy. Strategy 3. Put bold warnings in the reference http://dlang.org/class.html#destructors and all tutorials. That's useful but insufficient. Programmers will carry assumptions from other programming languages. Even those who read the D reference won't all remember that detail when it matters. Agreed. Strategy 4. Accept such crashes. (No good. The D home page promises safety.) Strongly agreed. IMNSHO we should try hard to detect errors already at compile time, even if it means disallowing potentially harmless things. It's just that in this case, the right thing to do is to fix the GC (and make writeln @nogc). This btw also applies to the other problems destructors have, like that the order of destruction of objects referring to each other is undefined. The GC should try to zero all references that it knows are GC managed, and are no longer life, before calling the destructor.
Re: forcing @nogc on class destructors
On Saturday, 24 January 2015 at 23:28:35 UTC, Jerry Morrison wrote: On Saturday, 24 January 2015 at 15:04:47 UTC, Ola Fosheim Grøstad wrote: If the classes are written for RAII then the destructors have to be called in reverse order of the constructors. IIRC D does not guarantee this when you use the GC. So to do it right there is a lot of GC overhead. Yes, but the usability question is what do programmers expect? How much do they assume before turning to the docs? Unfortunately, I think D is now entrenched in Java/C#ish expectations. Which is no good, since the main advantage D can have over those languages is to restrict the language semantics to a level where D has an inherent performance (timeliness) advantage. My expectations from a GC in a system level programming language would be to give max priority to fast collection at the expense of features (lean and mean). It's a big stretch to expect LIFO behavior from garbage collection. It's not a stretch to expect logging to work. What does logging in a destructor tell you? The destructor might not execute until the program terminates. You might not expect LIFO from the GC, but can you trust library authors to ensure that it does assume LIFO when manual memory management becomes commonplace? D needs to define what it means by safe and convenient. It is currently very much up in there air when it applies and when it does not.
Re: forcing @nogc on class destructors
From the forum thread http://forum.dlang.org/thread/ossuvfmqthllgdpgz...@forum.dlang.org?page=1 and the PR https://github.com/D-Programming-Language/dlang.org/pull/851 I learned that allocation in a destructor isn't the only crash case here. The GC calls a class instance's invariant() method during a collection cycle, before calling the object's destructor, and possibly after destructing objects it references. So if the invariant() touches any of its member references, it can crash! A solution to that would be to never call invariants during a collection cycle, but what other gotcha's exist during collection? Would @nogc on class destructors help them?
Re: forcing @nogc on class destructors
On 1/27/15 7:00 PM, Jerry Morrison wrote: A solution to that would be to never call invariants during a collection cycle, but what other gotcha's exist during collection? Would @nogc on class destructors help them? I, too, think no invariants should be called during GC-triggered destruction. Andrei P.S. Good to see you around here, Jerry!
Re: forcing @nogc on class destructors
On Tuesday, 27 January 2015 at 22:46:30 UTC, Ola Fosheim Grøstad wrote: On Saturday, 24 January 2015 at 23:28:35 UTC, Jerry Morrison wrote: On Saturday, 24 January 2015 at 15:04:47 UTC, Ola Fosheim Grøstad wrote: If the classes are written for RAII then the destructors have to be called in reverse order of the constructors. IIRC D does not guarantee this when you use the GC. So to do it right there is a lot of GC overhead. Yes, but the usability question is what do programmers expect? How much do they assume before turning to the docs? Unfortunately, I think D is now entrenched in Java/C#ish expectations. Which is no good, since the main advantage D can have over those languages is to restrict the language semantics to a level where D has an inherent performance (timeliness) advantage. My expectations from a GC in a system level programming language would be to give max priority to fast collection at the expense of features (lean and mean). It's a big stretch to expect LIFO behavior from garbage collection. It's not a stretch to expect logging to work. What does logging in a destructor tell you? The destructor might not execute until the program terminates. Logging might help with debugging, say, to log accumulated data before the object disappears. Perhaps it's not a great example, but since logging doesn't do anything dubious like forging a strong link to the object, it would seem safe at first expectation. Because of this unknown delay before GC collection, also because GC can collect a cycle of objects, we know it can't promise LIFO order. You might not expect LIFO from the GC, but can you trust library authors to ensure that it does assume LIFO when manual memory management becomes commonplace? Sorry, I don't understand the question. I expect LIFO for freeing structs on the stack. D needs to define what it means by safe and convenient. It is currently very much up in there air when it applies and when it does not. Yes. The forum discussion http://forum.dlang.org/thread/gvaacvczsrybguddo...@forum.dlang.org?page=1 deals with an InvalidMemoryOperationError. Vladimir Panteleev created a wiki page http://wiki.dlang.org/InvalidMemoryOperationError on how to debug such cases, starting with building a patched Druntime. Very helpful, but not easy. == It would be a big step forwards if the runtime printed a clear error message like: Attempt to allocate GC memory within the destructor for class Foo.
Re: forcing @nogc on class destructors
1. is the only viable plan.
Re: forcing @nogc on class destructors
On Monday, 26 January 2015 at 00:39:18 UTC, ketmar wrote: On Sun, 25 Jan 2015 22:09:04 +, deadalnix wrote: 1. is the only viable plan. ...which, in turn, equals to do nothing for another year or more. We are waiting for your pull request.
Re: forcing @nogc on class destructors
On Sun, 25 Jan 2015 22:09:04 +, deadalnix wrote: 1. is the only viable plan. ...which, in turn, equals to do nothing for another year or more. instead of issuing warning to inform people that what they doing NOW is bad. i'm pretty sure that people are very happy to know that their code will work somewhere in the future, but here and now is invalid, and compiler doesn't even try to warn 'em about that fact. reading all that regular questions about InvalidMemoryOperation is fun, after all. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Mon, 26 Jan 2015 02:13:26 +, deadalnix wrote: On Monday, 26 January 2015 at 00:39:18 UTC, ketmar wrote: On Sun, 25 Jan 2015 22:09:04 +, deadalnix wrote: 1. is the only viable plan. ...which, in turn, equals to do nothing for another year or more. We are waiting for your pull request. it was preliminary rejected, as i'm not interested in fixing GC (it's ok for me in it's current state), only in error/waning. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
This is the first I've heard that allocating GC memory in a destructor will crash. That's an unexpected gotcha. I'd expect to be able to reliably do I/O or throw an exception. Strategy 1. Fix the GC's limitation. (One fewer pitfall to baby-sit.) Strategy 2. Have the compiler inform the programmer. (The compiler can't catch all bugs but it should do what it can. Arguably this is a GC bug, not mine.) Strategy 3. Put bold warnings in the reference http://dlang.org/class.html#destructors and all tutorials. That's useful but insufficient. Programmers will carry assumptions from other programming languages. Even those who read the D reference won't all remember that detail when it matters. Strategy 4. Accept such crashes. (No good. The D home page promises safety.) On Saturday, 24 January 2015 at 15:04:47 UTC, Ola Fosheim Grøstad wrote: If the classes are written for RAII then the destructors have to be called in reverse order of the constructors. IIRC D does not guarantee this when you use the GC. So to do it right there is a lot of GC overhead. Yes, but the usability question is what do programmers expect? How much do they assume before turning to the docs? It's a big stretch to expect LIFO behavior from garbage collection. It's not a stretch to expect logging to work.
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 20:32:14 UTC, Steven Schveighoffer wrote: Actually there's nothing on the documentation about class destructors [1] that warns about that specific issue of the current (and default) GC. [1] http://dlang.org/class.html#destructors I think the docs are in need of updating there. It's not meant to be a secret that you cannot allocate inside a GC collection. I'll try to put together a PR. Please do! On Thursday, 22 January 2015 at 14:07:31 UTC, Steven Schveighoffer wrote: In the vast majority of cases, nobody adds a dtor, or they use dtors for their intended purpose (deallocating non-GC resources) and everything works fine. And the runtime catches any slips with an appropriate handling (abort instead of corrupt memory). While you're writing that PR, would you clarify that The destructor is expected to release any resources held by the object. means ***non-GC resources*** and that `new` will abort? Also would you be so kind as to make a bolder warning that member refs to GC objects may be invalid? That's nasty and unexpected! (Java finalizers don't work like that.) [Nulling them out would be nicer for debugging...] This means that destructors cannot reference sub objects. This rule does not apply to auto objects or objects deleted with the DeleteExpression, as the destructor is not being run by the garbage collector, meaning all references are valid. That wording contradicts the previous paragraph about the delete expression ... [calls] ... the garbage collector calls the destructor immediately. Maybe something like ...not being run in a garbage collection... would be better.
Re: forcing @nogc on class destructors
On Saturday, 24 January 2015 at 00:31:48 UTC, Paolo Invernizzi wrote: But my real point is that depending on dtors being called by the GC is not broken 90% of the time. -Steve You are right, on a literal point, but I think that what he was If the classes are written for RAII then the destructors have to be called in reverse order of the constructors. IIRC D does not guarantee this when you use the GC. So to do it right there is a lot of GC overhead. be appropriate: it's a very common pitfall, and a pitfall that simply cast a bad light on the language in newcomers' minds when it occurs at runtime. Yes: https://www.google.no/search?q=InvalidMemoryOperationError
Re: forcing @nogc on class destructors
On Friday, 23 January 2015 at 13:12:44 UTC, Steven Schveighoffer wrote: On 1/23/15 8:05 AM, Matthias Bentrup wrote: On Friday, 23 January 2015 at 10:53:54 UTC, aldanor wrote: On Friday, 23 January 2015 at 08:58:11 UTC, Ola Fosheim Grøstad wrote: How about banning GC-allocation of classes with destructors? Uh... what? ^__^ Maybe just ban classes altogether then? No, don't ban them, that will break to much code. Just don't execute them. Any application that depends on destructors being called by the GC is broken in 9 out of 10 cases anyway. This is very wrong. -Steve This is as per spec :)
Re: forcing @nogc on class destructors
On 1/23/15 3:40 PM, deadalnix wrote: On Friday, 23 January 2015 at 13:12:44 UTC, Steven Schveighoffer wrote: On 1/23/15 8:05 AM, Matthias Bentrup wrote: On Friday, 23 January 2015 at 10:53:54 UTC, aldanor wrote: On Friday, 23 January 2015 at 08:58:11 UTC, Ola Fosheim Grøstad wrote: How about banning GC-allocation of classes with destructors? Uh... what? ^__^ Maybe just ban classes altogether then? No, don't ban them, that will break to much code. Just don't execute them. Any application that depends on destructors being called by the GC is broken in 9 out of 10 cases anyway. This is very wrong. This is as per spec :) No, spec says your dtor is not guaranteed to run for unreferenced memory. It doesn't say your destructor may not be run even when the memory is collected. In fact, it says The garbage collector calls the destructor function when the object is deleted. But my real point is that depending on dtors being called by the GC is not broken 90% of the time. -Steve
Re: forcing @nogc on class destructors
On Friday, 23 January 2015 at 21:00:08 UTC, Steven Schveighoffer wrote: On 1/23/15 3:40 PM, deadalnix wrote: On Friday, 23 January 2015 at 13:12:44 UTC, Steven Schveighoffer wrote: On 1/23/15 8:05 AM, Matthias Bentrup wrote: On Friday, 23 January 2015 at 10:53:54 UTC, aldanor wrote: On Friday, 23 January 2015 at 08:58:11 UTC, Ola Fosheim Grøstad wrote: How about banning GC-allocation of classes with destructors? Uh... what? ^__^ Maybe just ban classes altogether then? No, don't ban them, that will break to much code. Just don't execute them. Any application that depends on destructors being called by the GC is broken in 9 out of 10 cases anyway. This is very wrong. This is as per spec :) No, spec says your dtor is not guaranteed to run for unreferenced memory. It doesn't say your destructor may not be run even when the memory is collected. In fact, it says The garbage collector calls the destructor function when the object is deleted. But my real point is that depending on dtors being called by the GC is not broken 90% of the time. -Steve If the program ends before the next GC the destructor is never called.
Re: forcing @nogc on class destructors
On Friday, 23 January 2015 at 06:16:21 UTC, Jakob Ovrum wrote: Classes don't have to be designed to be allocated on the GC heap. Class instances can be allocated on the stack, the C heap, or anywhere else. …and this is a badly typed design if you don't embed ownership in reference types. But, GC should not call destructors, because classes using RAII will then fail. D is a systems-programming language and should cater to a wide variety of use patterns, including users who wish to use classes in non-GC memory locations. There are way too many weaknesses in current D semantics to make it suitable for systems programming. By systems programming I mean programming that push the hardware to the limits and take advantage of hardware specific properties. The fact that you cannot gracefully recover from low-memory conditions is a bad sign. A good reason for not acquiring resources in destructors is that you can then throw exceptions and recover when you run out of resources. Which is important on high load and/or low memory devices. Perhaps splitting class destructors into two separate concepts, a destructor and a finalizer, with the latter being @nogc, could be a more satisfactory solution. How about banning GC-allocation of classes with destructors?
Re: forcing @nogc on class destructors
On Friday, 23 January 2015 at 21:00:08 UTC, Steven Schveighoffer wrote: On 1/23/15 3:40 PM, deadalnix wrote: On Friday, 23 January 2015 at 13:12:44 UTC, Steven Schveighoffer wrote: On 1/23/15 8:05 AM, Matthias Bentrup wrote: On Friday, 23 January 2015 at 10:53:54 UTC, aldanor wrote: On Friday, 23 January 2015 at 08:58:11 UTC, Ola Fosheim Grøstad wrote: How about banning GC-allocation of classes with destructors? Uh... what? ^__^ Maybe just ban classes altogether then? No, don't ban them, that will break to much code. Just don't execute them. Any application that depends on destructors being called by the GC is broken in 9 out of 10 cases anyway. This is very wrong. This is as per spec :) No, spec says your dtor is not guaranteed to run for unreferenced memory. It doesn't say your destructor may not be run even when the memory is collected. In fact, it says The garbage collector calls the destructor function when the object is deleted. But my real point is that depending on dtors being called by the GC is not broken 90% of the time. -Steve You are right, on a literal point, but I think that what he was saying was something like: Any application that depends on object being collected in a collect circle as unreferenced, so that depends on destructors are being called by the GC is broken in 9 out of 10 cases anyway I agree with aldanor about his argumentation that a warning would be appropriate: it's a very common pitfall, and a pitfall that simply cast a bad light on the language in newcomers' minds when it occurs at runtime. Also if a lint tool can catch it, we all are conscious of the big effort that the community leaders are putting on picturing a better language for newbies (please, no pun intended, really!) -- Paolo
Re: forcing @nogc on class destructors
On Fri, 23 Jan 2015 23:28:28 +, Matthias Bentrup wrote:
If the program ends before the next GC the destructor is never called.
nope.
class A {
~this () { import iv.writer; writeln(dtor); }
}
void main () {
auto n = new A;
assert(0);
}
this outputs dtor.
signature.asc
Description: PGP signature
Re: forcing @nogc on class destructors
On Friday, 23 January 2015 at 23:28:28 UTC, Matthias Bentrup wrote: On Friday, 23 January 2015 at 21:00:08 UTC, Steven Schveighoffer wrote: On 1/23/15 3:40 PM, deadalnix wrote: On Friday, 23 January 2015 at 13:12:44 UTC, Steven Schveighoffer wrote: On 1/23/15 8:05 AM, Matthias Bentrup wrote: On Friday, 23 January 2015 at 10:53:54 UTC, aldanor wrote: On Friday, 23 January 2015 at 08:58:11 UTC, Ola Fosheim Grøstad wrote: How about banning GC-allocation of classes with destructors? Uh... what? ^__^ Maybe just ban classes altogether then? No, don't ban them, that will break to much code. Just don't execute them. Any application that depends on destructors being called by the GC is broken in 9 out of 10 cases anyway. This is very wrong. This is as per spec :) No, spec says your dtor is not guaranteed to run for unreferenced memory. It doesn't say your destructor may not be run even when the memory is collected. In fact, it says The garbage collector calls the destructor function when the object is deleted. But my real point is that depending on dtors being called by the GC is not broken 90% of the time. -Steve If the program ends before the next GC the destructor is never called. when the runtime shuts down it calls a full collect on the GC. I'm not sure if this is actually required as per the spec.
Re: forcing @nogc on class destructors
On Friday, 23 January 2015 at 08:58:11 UTC, Ola Fosheim Grøstad wrote: How about banning GC-allocation of classes with destructors? Uh... what? ^__^ Maybe just ban classes altogether then?
Re: forcing @nogc on class destructors
On 1/23/15 8:05 AM, Matthias Bentrup wrote: On Friday, 23 January 2015 at 10:53:54 UTC, aldanor wrote: On Friday, 23 January 2015 at 08:58:11 UTC, Ola Fosheim Grøstad wrote: How about banning GC-allocation of classes with destructors? Uh... what? ^__^ Maybe just ban classes altogether then? No, don't ban them, that will break to much code. Just don't execute them. Any application that depends on destructors being called by the GC is broken in 9 out of 10 cases anyway. This is very wrong. -Steve
Re: forcing @nogc on class destructors
On Friday, 23 January 2015 at 13:12:44 UTC, Steven Schveighoffer wrote: On 1/23/15 8:05 AM, Matthias Bentrup wrote: On Friday, 23 January 2015 at 10:53:54 UTC, aldanor wrote: On Friday, 23 January 2015 at 08:58:11 UTC, Ola Fosheim Grøstad wrote: How about banning GC-allocation of classes with destructors? Uh... what? ^__^ Maybe just ban classes altogether then? No, don't ban them, that will break to much code. Just don't execute them. Any application that depends on destructors being called by the GC is broken in 9 out of 10 cases anyway. This is very wrong. -Steve Summing up the arguments in this thread, I think the reasonable middle ground here would be for a compiler to throw a warning if dtor doesn't satisfy @nogc. I.e., compiler knows at compile time that something MAY and probably WILL go wrong; the code as of current GC implementation is then unsafe which should be reported to the user. This is much better than just silent obscure memory exceptions at runtime.
Re: forcing @nogc on class destructors
On Friday, 23 January 2015 at 10:53:54 UTC, aldanor wrote: On Friday, 23 January 2015 at 08:58:11 UTC, Ola Fosheim Grøstad wrote: How about banning GC-allocation of classes with destructors? Uh... what? ^__^ Maybe just ban classes altogether then? No, don't ban them, that will break to much code. Just don't execute them. Any application that depends on destructors being called by the GC is broken in 9 out of 10 cases anyway.
Re: forcing @nogc on class destructors
On Thursday, 22 January 2015 at 11:50:55 UTC, Steven Schveighoffer wrote: On 1/21/15 6:03 PM, Paolo Invernizzi wrote: On Wednesday, 21 January 2015 at 20:32:14 UTC, Steven Schveighoffer wrote: On 1/21/15 3:37 AM, Paolo Invernizzi wrote: On Wednesday, 21 January 2015 at 03:02:53 UTC, Steven Schveighoffer wrote: On 1/20/15 9:04 PM, ketmar via Digitalmars-d wrote: If he does it wrong, it gives him a stack trace on where to look. What is different here than any other programming error? Are you suggesting that newcomers should learn D by discovering it day by day from stack traces? No, I was saying if something causes an exception/error, it is a programming error, and there just isn't any way for a compiler to prevent people from making *any* mistakes. But calling sometimes-allocating functions inside a dtor that don't allocate when you call them *that* time shouldn't be banned by the compiler. You can't ban them, either now with an annotated @nogc destructor: SetFunctionAttributes. Right, but the runtime will still catch it if it allocates. I think the correct place to check it is where it's checked now. -Steve I have troubles following your reasoning, so I must have lost something. - the proposal was to have a default @nogc in the dtor. - the rebuttal was that functions that don't allocate when you call them *THAT TIME* shouldn't be banned. - I just noticed that also with destructor marked @nogc, if you know and trust a @gc function you can use it with the help of traits and casts. - the reply was that the runtime will catch it if it allocate, that I don't understand what is related with the notice, or with the original proposal. I don't see nothing strange in turning into the default the CT check about avoiding GC allocations in destructors, that's the vast majority of uses cases, and force an explicit cast/traits if the programmer want to mark in the code hey, I know that using the GC in the DTOR is a bad, but trust this cast, in any case If I'm wrong the runtime will signal you. Sorry if I feel pedantic in the discussion, but being a not native english speaker (and a little in a hurry when writing in the forums) I just wanted to be clear in my exposition... ;-) -- Paolo
Re: forcing @nogc on class destructors
On 1/21/15 6:03 PM, Paolo Invernizzi wrote: On Wednesday, 21 January 2015 at 20:32:14 UTC, Steven Schveighoffer wrote: On 1/21/15 3:37 AM, Paolo Invernizzi wrote: On Wednesday, 21 January 2015 at 03:02:53 UTC, Steven Schveighoffer wrote: On 1/20/15 9:04 PM, ketmar via Digitalmars-d wrote: If he does it wrong, it gives him a stack trace on where to look. What is different here than any other programming error? Are you suggesting that newcomers should learn D by discovering it day by day from stack traces? No, I was saying if something causes an exception/error, it is a programming error, and there just isn't any way for a compiler to prevent people from making *any* mistakes. But calling sometimes-allocating functions inside a dtor that don't allocate when you call them *that* time shouldn't be banned by the compiler. You can't ban them, either now with an annotated @nogc destructor: SetFunctionAttributes. Right, but the runtime will still catch it if it allocates. I think the correct place to check it is where it's checked now. -Steve
Re: forcing @nogc on class destructors
On 1/22/15 8:42 AM, Paolo Invernizzi wrote: You can't ban them, either now with an annotated @nogc destructor: SetFunctionAttributes. Right, but the runtime will still catch it if it allocates. I think the correct place to check it is where it's checked now. I have troubles following your reasoning, so I must have lost something. - the proposal was to have a default @nogc in the dtor. - the rebuttal was that functions that don't allocate when you call them *THAT TIME* shouldn't be banned. - I just noticed that also with destructor marked @nogc, if you know and trust a @gc function you can use it with the help of traits and casts. - the reply was that the runtime will catch it if it allocate, that I don't understand what is related with the notice, or with the original proposal. Missing from this list is that -- if it isn't broke don't fix it :) I will note that I think if @nogc was the default, I would be in favor of keeping it that way. In order to justify a breaking change like this, there needs to be a good enough reason. IMO (and really, this is my opinion, I'm not the ultimate gatekeeper here) the bar has not been cleared. I don't see nothing strange in turning into the default the CT check about avoiding GC allocations in destructors, that's the vast majority of uses cases, and force an explicit cast/traits if the programmer want to mark in the code hey, I know that using the GC in the DTOR is a bad, but trust this cast, in any case If I'm wrong the runtime will signal you. In the vast majority of cases, nobody adds a dtor, or they use dtors for their intended purpose (deallocating non-GC resources) and everything works fine. And the runtime catches any slips with an appropriate handling (abort instead of corrupt memory). Sorry if I feel pedantic in the discussion, but being a not native english speaker (and a little in a hurry when writing in the forums) I just wanted to be clear in my exposition... ;-) I understand your opinion, I just don't think it's worth the trouble and breaking of code. -Steve
Re: forcing @nogc on class destructors
On 1/21/15 3:45 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Wednesday, 21 January 2015 at 03:03:50 UTC, Steven Schveighoffer wrote: It's neither spurious, nor a race condition. How can you say that a priori? When the GC collects, it doesn't allow memory allocation. This is not a race condition, it's simply a fact of programming with D. It's not unintentional or spurious. A race condition can LEAD to this being triggered, but the abort itself is not a race condition. And forcing a default of @nogc will not fix this. You said spurious race conditions in memory deallocation patterns that remain undetected and cause random crashes after deployment are ok If you didn't mean that we were *causing* spurious race conditions by allowing non-@nogc calls inside a destructor, and just meant that spurious race conditions were not ok, I agree with you. -Steve
Re: forcing @nogc on class destructors
On Thursday, 22 January 2015 at 14:07:31 UTC, Steven Schveighoffer wrote: resources) and everything works fine. And the runtime catches any slips with an appropriate handling (abort instead of corrupt memory). Runtime errors that could have been a compile time error should always be caught at compile time. Spurious runtime errors are just as bad as null pointer accesses.
Re: forcing @nogc on class destructors
On Thursday, 22 January 2015 at 11:58:13 UTC, Steven Schveighoffer wrote: A race condition can LEAD to this being triggered, but the abort itself is not a race condition. And forcing a default of @nogc will not fix this. Why not? If you have a race condition that leads a memory leak when using explicit deallocation, then that leak will lead to destructor being run by the collector. Which will fail if the destructor gc-allocates. GC is meant to make allocation safer, not less safe.
Re: forcing @nogc on class destructors
Ola Fosheim Grøstad wrote in message news:ncuttyurxztvrziuy...@forum.dlang.org... Runtime errors that could have been a compile time error should always be caught at compile time. Spurious runtime errors are just as bad as null pointer accesses. Allocating in a dtor shouldn't be a runtime or compile-time error, it should just work.
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 18:12:27 UTC, ketmar via Digitalmars-d wrote: Hello. as there is no possibility to doing GC allocations in class destructors, wouldn't it be nice to just force @nogc attribute on such dtors? Classes don't have to be designed to be allocated on the GC heap. Class instances can be allocated on the stack, the C heap, or anywhere else. i know, i know, this will break alot of code. i'm pretty sure that this will break alot of INVALID code, which better be broken at compile-time anyway. There is potentially a plethora of valid code broken by such a change: * Classes designed to be used on the stack or other non-GC-heap memory locations. * Class destructors that never call a GC function, but @nogc was nevertheless not applied throughout the codebase, maybe because it was an old or poorly maintainable codebase. * Users using non-standard or modified GC implementations that don't have this restriction. * Classes in programs that have ripped out the GC to begin with, replacing `new` and other GC primitives with something else. This one is dubious, as from a language standpoint, `new` is exclusively intended for GC allocation. * Class destructors calling into functions that predictively have both an allocating and non-allocating path, thus cannot be @nogc but can be verified at the call-site to be @nogc. Arguably such functions are poorly designed and should be split into two layers. let's see how this proposal will be rejected. will there be some sane reasons, or only the good old song about broken code? make your bets! D is a systems-programming language and should cater to a wide variety of use patterns, including users who wish to use classes in non-GC memory locations. Perhaps splitting class destructors into two separate concepts, a destructor and a finalizer, with the latter being @nogc, could be a more satisfactory solution. Alternatively, maybe we should anticipate a standard GC implementation (or one of several standard implementations, like Java and .NET do) that does not have this restriction.
Re: forcing @nogc on class destructors
On 1/22/15 10:59 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Thursday, 22 January 2015 at 11:58:13 UTC, Steven Schveighoffer wrote: A race condition can LEAD to this being triggered, but the abort itself is not a race condition. And forcing a default of @nogc will not fix this. Why not? If you have a race condition that leads a memory leak when using explicit deallocation, then that leak will lead to destructor being run by the collector. Which will fail if the destructor gc-allocates. GC is meant to make allocation safer, not less safe. Disallowing compilation of the dtor does not fix the race condition. We are no closer to race-free code here. What you are basically saying is, race conditions are OK, as long as they don't allocate memory in dtors. -Steve
Re: forcing @nogc on class destructors
On 1/22/15 11:10 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Thursday, 22 January 2015 at 14:07:31 UTC, Steven Schveighoffer wrote: resources) and everything works fine. And the runtime catches any slips with an appropriate handling (abort instead of corrupt memory). Runtime errors that could have been a compile time error should always be caught at compile time. Spurious runtime errors are just as bad as null pointer accesses. Well, code that does not have errors should not be caught at compile time. I count writeln(in dtor) as one of them. You may disagree, but we aren't making progress on this. -Steve
Re: forcing @nogc on class destructors
On Thursday, 22 January 2015 at 16:53:10 UTC, Steven Schveighoffer wrote: Disallowing compilation of the dtor does not fix the race condition. We are no closer to race-free code here. What you are basically saying is, race conditions are OK, as long as they don't allocate memory in dtors. Most large programs will be deployed with some minor bugs. What I basically say is that a program that would run fine with manual memory management and some occasional leaks, will crash if you use alloc from the gc heap instead of the plain heap.
Re: forcing @nogc on class destructors
On 1/21/15 3:37 AM, Paolo Invernizzi wrote: On Wednesday, 21 January 2015 at 03:02:53 UTC, Steven Schveighoffer wrote: On 1/20/15 9:04 PM, ketmar via Digitalmars-d wrote: On Tue, 20 Jan 2015 20:51:34 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: You can always put @nogc on the dtor if you want. seems that you completely missing my point. (sigh) Nope, not missing it. The mechanics are there. You just have to annotate. that is where you missing it. your answer is like hey, C has all mechanics for doing OOP with virtual methods and type checking, you just have to write the code! No, actually it's not. Adding @nogc to a function is as hard as writing class when you want to do OOP. the whole point of my talk was free programmer from writing the obvious and setup some red tapes for beginners. If he does it wrong, it gives him a stack trace on where to look. What is different here than any other programming error? Are you suggesting that newcomers should learn D by discovering it day by day from stack traces? No, I was saying if something causes an exception/error, it is a programming error, and there just isn't any way for a compiler to prevent people from making *any* mistakes. But calling sometimes-allocating functions inside a dtor that don't allocate when you call them *that* time shouldn't be banned by the compiler. Actually there's nothing on the documentation about class destructors [1] that warns about that specific issue of the current (and default) GC. [1] http://dlang.org/class.html#destructors I think the docs are in need of updating there. It's not meant to be a secret that you cannot allocate inside a GC collection. I'll try to put together a PR. -Steve
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 20:32:14 UTC, Steven Schveighoffer wrote: On 1/21/15 3:37 AM, Paolo Invernizzi wrote: On Wednesday, 21 January 2015 at 03:02:53 UTC, Steven Schveighoffer wrote: On 1/20/15 9:04 PM, ketmar via Digitalmars-d wrote: If he does it wrong, it gives him a stack trace on where to look. What is different here than any other programming error? Are you suggesting that newcomers should learn D by discovering it day by day from stack traces? No, I was saying if something causes an exception/error, it is a programming error, and there just isn't any way for a compiler to prevent people from making *any* mistakes. But calling sometimes-allocating functions inside a dtor that don't allocate when you call them *that* time shouldn't be banned by the compiler. You can't ban them, either now with an annotated @nogc destructor: SetFunctionAttributes.
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 20:15:53 UTC, deadalnix wrote: Ok listen, I'm gonna put a full stop to this conversation : - You provide no evidence of your claims. HAHAHA… - You discard any evidence that contradict your claims without solid argument. What evidence? - You are constantly shitfting the goalspot using more and more ridiculous claims (like malloc should not be used). You can batch-allocate, but you need to use an O(1) allocator if you want fast memory handling. There no point in continuing this discussion. That's right... - A concurrent GC is not suitable for system languages. And there is nothing that suggests that it is. Go is not a system programming language.
Re: forcing @nogc on class destructors
Ok listen, I'm gonna put a full stop to this conversation : - You provide no evidence of your claims. - You discard any evidence that contradict your claims without solid argument. - You are constantly shitfting the goalspot using more and more ridiculous claims (like malloc should not be used). There no point in continuing this discussion. If you have a point, the burden of proof is on you. I remind you that, before you brought all kind of topic to the discussion to avoid having to substantiate anything, your claim was : - A concurrent GC is not suitable for system languages.
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 10:00:21 UTC, deadalnix wrote: 1. less efficient codegen + restrictions related to FFI No. How would that work? You cannot trace pointers that stay in registers. You also cannot do precise unwinding of FFI stacks. 3. intermittent pauses I think you don't quite get what concurrent is about. Parallel: running simultaneously Concurrent: parallel and/or interleaving C# use intermittent pauses/parallel https://msdn.microsoft.com/en-us/library/ee787088(v=vs.110).aspx#concurrent_garbage_collection 4. random cache pollution If you have a multicore (and you have one) no. Multicore won't help on cache level 3. (hyper-threading cores share cache too) Inconvenient if you try to get the most out of the hardware. This is all the contrary. As you may have noticed, most memory allocation intensive benchmark in Java tend to outperform their C++ counter part. Malloc is not fast. If you want performance you use O(1) allocators/deallocators. Bogus benchmarks to make Java look good are not convincing.
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 08:51:08 UTC, Ola Fosheim Grøstad wrote: Implicit concurrent GC collection leads to: 1. less efficient codegen + restrictions related to FFI No. 2. higher memory usage with limited ability to put upper bounds Yes, the memory allocated during the collection will be collected at the next cycle,and you can even get into trashing scenario if you allocate faster than you can collect. 3. intermittent pauses I think you don't quite get what concurrent is about. 4. random cache pollution If you have a multicore (and you have one) no. Inconvenient if you try to get the most out of the hardware. This is all the contrary. As you may have noticed, most memory allocation intensive benchmark in Java tend to outperform their C++ counter part. This is because Java can collect concurrently, giving some level of parallelism for free. Nowadays, multicore machine are all over the place, and getting a part of the workload offloaded on another core for free is something you want in the general case.
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 09:54:32 UTC, deadalnix wrote: What is the status quo is irrelevant. You make a claim, you substantiate. You need to take a course called the philosophy of science. You've made a claim that concurrent GC is what is needed for competitive system level programming support. Competitive meaning you can get close to peak performance. A peer reviewed reference is all you need to provide. That can't be too much to ask for, given that GC is a well researched field in computer science.
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 03:02:53 UTC, Steven Schveighoffer wrote: On 1/20/15 9:04 PM, ketmar via Digitalmars-d wrote: On Tue, 20 Jan 2015 20:51:34 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: You can always put @nogc on the dtor if you want. seems that you completely missing my point. (sigh) Nope, not missing it. The mechanics are there. You just have to annotate. that is where you missing it. your answer is like hey, C has all mechanics for doing OOP with virtual methods and type checking, you just have to write the code! No, actually it's not. Adding @nogc to a function is as hard as writing class when you want to do OOP. the whole point of my talk was free programmer from writing the obvious and setup some red tapes for beginners. If he does it wrong, it gives him a stack trace on where to look. What is different here than any other programming error? Are you suggesting that newcomers should learn D by discovering it day by day from stack traces? Actually there's nothing on the documentation about class destructors [1] that warns about that specific issue of the current (and default) GC. [1] http://dlang.org/class.html#destructors
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 03:03:50 UTC, Steven Schveighoffer wrote: It's neither spurious, nor a race condition. How can you say that a priori? If you are using a framework, then subclass a node (which cannot be made @nogc), then do explicit deallocation to speed it up, then do multi-threading to speed it up... then any spurious memory leak due to races can result in crashes after deployment with this GC scheme. The sad thing is that an innocent rare memory leak with C++ style memory allocation is less critical than a memory leak picked up by the GC. That's bad marketing vs the C++ community which are highly sceptical to using a GC in the first place.
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 01:07:59 UTC, weaselcat wrote: On Wednesday, 21 January 2015 at 01:05:28 UTC, weaselcat wrote: there is no silver bllet for memory and Sorry, meant silver bullet for memory management, bit tired : ) Yeah, but the silver bullet for memory management in real time applications are: 1. preallocation 2. O(1) allocation with upper bounds on total memory usage Then you can use a GC if you 1. know when it runs and have headroom for it 2. can put an upper bound on what it scans Implicit concurrent GC collection leads to: 1. less efficient codegen + restrictions related to FFI 2. higher memory usage with limited ability to put upper bounds 3. intermittent pauses 4. random cache pollution Inconvenient if you try to get the most out of the hardware.
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 00:47:43 UTC, deadalnix wrote: On Tuesday, 20 January 2015 at 23:47:44 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 20 January 2015 at 23:17:28 UTC, deadalnix wrote: Concurrent GC is too expensive for a proper system level language. That is an unsubstanciated claim. And so is «pigs can't fly». That is the old prove a negative bullshit. No. You can make a pig fly, but the landing is going to be ugly compared to the competition. The status quo is that GC is not desirable in typical real time / system level programming. And if it is to be used, it should be predictable and not require special code gen. So that is hypothesis h0. Then it is up to you to replace it with hypothesis h1: concurrent GC is fine for typical system level programming. All you have to do is to provide a documented proof-of-concept, or at least a link to a solid paper that describes a concurrent GC that has the desired properties: 1. no negative effect on code gen 2. predictable upper bounds all around (time, latency and memory) 3. no bad pollution of caches at the wrong time 4. no intermittent pauses at random times You assert that something is too expensive for system language, which is not a negative, so the burden of proof is on you, substantiate your claim or you are just generating noise. Oh... But you are the one that is challenging the status quo. Not me. The excitement created by Rust is proof enough for what the status quo is. Relying on cheap rhetorical trick is not going to fly (and you could add, no more than a pig). GC is an unnecessary pig to begin with. If you want to use a pig to eat your leftovers, then you need to know how hungry it is and how much and how fast it can eat. I think the metaphor is apt. :-)
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 09:03:32 UTC, Ola Fosheim Grøstad wrote: You assert that something is too expensive for system language, which is not a negative, so the burden of proof is on you, substantiate your claim or you are just generating noise. Oh... But you are the one that is challenging the status quo. Not me. The excitement created by Rust is proof enough for what the status quo is. That's irrelevant. Once again, you are giving into the rhetoric bullshit. Or, as to quote Coluche « C'est pas parce qu'il sont nombreux à avoir tort qu'ils ont raison » that I would translate by It is not because many of them are wrong that they are right. What is the status quo is irrelevant. You make a claim, you substantiate.
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 18:12:27 UTC, ketmar via Digitalmars-d wrote: Hello. as there is no possibility to doing GC allocations in class destructors, wouldn't it be nice to just force @nogc attribute on such dtors? i know, i know, this will break alot of code. i'm pretty sure that this will break alot of INVALID code, which better be broken at compile-time anyway. sure, we have alot of code of pre-@nogc era, and alot of code where authord didn't bother to add attributes at all. so we can introduce --force-dtor-nogc CLI arg and document this change, making it opt-in for, say, six month and opt-out after that. and i know that D devs (Walter at least) are resistant to command-line flags that changing compiler behavior. i don't know how to overcome this. say, by adding @gc attribute, which dfix can automatically add? but i still believe that instead of telling people again and again that they should not allocate in class destructors, we can use computer itself to track and stop this behavior. let's see how this proposal will be rejected. will there be some sane reasons, or only the good old song about broken code? make your bets! Isn't this just an implementation detail of the current garbage collector? If so, then we shouldn't tie language semantics to it, as it could change.
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 18:17:56 + Meta via Digitalmars-d digitalmars-d@puremagic.com wrote: On Tuesday, 20 January 2015 at 18:12:27 UTC, ketmar via Digitalmars-d wrote: Hello. as there is no possibility to doing GC allocations in class destructors, wouldn't it be nice to just force @nogc attribute on such dtors? i know, i know, this will break alot of code. i'm pretty sure that this will break alot of INVALID code, which better be broken at compile-time anyway. sure, we have alot of code of pre-@nogc era, and alot of code where authord didn't bother to add attributes at all. so we can introduce --force-dtor-nogc CLI arg and document this change, making it opt-in for, say, six month and opt-out after that. and i know that D devs (Walter at least) are resistant to command-line flags that changing compiler behavior. i don't know how to overcome this. say, by adding @gc attribute, which dfix can automatically add? but i still believe that instead of telling people again and again that they should not allocate in class destructors, we can use computer itself to track and stop this behavior. let's see how this proposal will be rejected. will there be some sane reasons, or only the good old song about broken code? make your bets! Isn't this just an implementation detail of the current garbage collector? If so, then we shouldn't tie language semantics to it, as it could change. how likely this to be changed? is there *any* chances of that in 2015? 2016? and why we can't just remove that restriction when new GC will be implemented? removing the @nogc requirement on class dtors will break *nothing* *at* *all*. yet adding it now, while we don't have that new GC, will prevent alot of bugs that can slip in crack. btw, you won the prize of not talking about broken code! sadly, i forgot to setup the prizes... anyway, thanks for sane argument. signature.asc Description: PGP signature
forcing @nogc on class destructors
Hello. as there is no possibility to doing GC allocations in class destructors, wouldn't it be nice to just force @nogc attribute on such dtors? i know, i know, this will break alot of code. i'm pretty sure that this will break alot of INVALID code, which better be broken at compile-time anyway. sure, we have alot of code of pre-@nogc era, and alot of code where authord didn't bother to add attributes at all. so we can introduce --force-dtor-nogc CLI arg and document this change, making it opt-in for, say, six month and opt-out after that. and i know that D devs (Walter at least) are resistant to command-line flags that changing compiler behavior. i don't know how to overcome this. say, by adding @gc attribute, which dfix can automatically add? but i still believe that instead of telling people again and again that they should not allocate in class destructors, we can use computer itself to track and stop this behavior. let's see how this proposal will be rejected. will there be some sane reasons, or only the good old song about broken code? make your bets! signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 21:37:56 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 20 January 2015 at 21:29:40 UTC, Steven Schveighoffer wrote: How's that? The current runtime aborts on memory allocation inside the GC collection routine, it's not a memory safety issue. Spurious race conditions in memory deallocation patterns that remain undetected and cause random crashes after deployment are ok? That is not memory safety, it is memory safety. The language should prevent this. That is GC implementation issue, not a language issue. If it ain't broken, don't fix it.
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 23:00:26 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 20 January 2015 at 22:25:05 UTC, deadalnix wrote: Any serious GC can run concurrently (instead of stopping the world). That mean any serious GC must be able to handle allocations while collecting. Concurrent GC is too expensive for a proper system level language. That is an unsubstanciated claim.
Re: forcing @nogc on class destructors
On Wed, 21 Jan 2015 00:29:21 + Kapps via Digitalmars-d digitalmars-d@puremagic.com wrote: On Tuesday, 20 January 2015 at 18:12:27 UTC, ketmar via Digitalmars-d wrote: let's see how this proposal will be rejected. will there be some sane reasons, or only the good old song about broken code? make your bets! Lots of functions can theoretically allocate, but don't in the way you call them. For example, a function that checks for invalid arguments and throws an exception if any are passed in. It can't be @nogc because it throws, but it's perfectly valid to call in a constructor. nothing is valid with calling even potentially allocating function in dtor. Also, what about classes allocated with malloc/emplace that are then destroyed/freed? that's easy: mark that destructors with proposed `@gc` attribute. this way you will explicitly tell the compiler that you know what you're doing here and taking all responsibility for your actions. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 21:34:54 + Freddy via Digitalmars-d digitalmars-d@puremagic.com wrote: On Tuesday, 20 January 2015 at 18:12:27 UTC, ketmar via Digitalmars-d wrote: Hello. as there is no possibility to doing GC allocations in class destructors, wouldn't it be nice to just force @nogc attribute on such dtors? i know, i know, this will break alot of code. i'm pretty sure that this will break alot of INVALID code, which better be broken at compile-time anyway. sure, we have alot of code of pre-@nogc era, and alot of code where authord didn't bother to add attributes at all. so we can introduce --force-dtor-nogc CLI arg and document this change, making it opt-in for, say, six month and opt-out after that. and i know that D devs (Walter at least) are resistant to command-line flags that changing compiler behavior. i don't know how to overcome this. say, by adding @gc attribute, which dfix can automatically add? but i still believe that instead of telling people again and again that they should not allocate in class destructors, we can use computer itself to track and stop this behavior. let's see how this proposal will be rejected. will there be some sane reasons, or only the good old song about broken code? make your bets! Not an error, Make it a warning. alas, attribute violations are errors. turning that into warning means that compiler needs new flags and new code to determine if that is hard restriction or just advise. the point of my proposal is that it can be done painlessly with one or two very simple changes in compiler code. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 21:41:15 + Vladimir Panteleev via Digitalmars-d digitalmars-d@puremagic.com wrote: Not an error, Make it a warning. alas, attribute violations are errors. turning that into warning means that compiler needs new flags and new code to determine if that is hard restriction or just advise. the point of my proposal is that it can be done painlessly with one or two very simple changes in compiler code. We already have warnings such as: Warning: toHash() must be declared as extern (D) size_t toHash() const nothrow @safe, not const uint() It could issue a similar warning to destructors declared without @nogc. ah, i missed that, sorry. sure, if there is already mechanics for this, warning will be ok. i just want the compiler to not remaining silent about the possible issue, and making it error seemed the easiest way. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 15:51:17 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: p.s. another point is that all mechanics compiler needs for doing such checks is already there, so it's not a huge change to compiler codebase. it's not something that requires adding a whole new analysis code. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 21:38:50 UTC, ketmar via Digitalmars-d wrote: On Tue, 20 Jan 2015 21:34:54 + Freddy via Digitalmars-d digitalmars-d@puremagic.com wrote: On Tuesday, 20 January 2015 at 18:12:27 UTC, ketmar via Digitalmars-d wrote: Hello. as there is no possibility to doing GC allocations in class destructors, wouldn't it be nice to just force @nogc attribute on such dtors? i know, i know, this will break alot of code. i'm pretty sure that this will break alot of INVALID code, which better be broken at compile-time anyway. sure, we have alot of code of pre-@nogc era, and alot of code where authord didn't bother to add attributes at all. so we can introduce --force-dtor-nogc CLI arg and document this change, making it opt-in for, say, six month and opt-out after that. and i know that D devs (Walter at least) are resistant to command-line flags that changing compiler behavior. i don't know how to overcome this. say, by adding @gc attribute, which dfix can automatically add? but i still believe that instead of telling people again and again that they should not allocate in class destructors, we can use computer itself to track and stop this behavior. let's see how this proposal will be rejected. will there be some sane reasons, or only the good old song about broken code? make your bets! Not an error, Make it a warning. alas, attribute violations are errors. turning that into warning means that compiler needs new flags and new code to determine if that is hard restriction or just advise. the point of my proposal is that it can be done painlessly with one or two very simple changes in compiler code. We already have warnings such as: Warning: toHash() must be declared as extern (D) size_t toHash() const nothrow @safe, not const uint() It could issue a similar warning to destructors declared without @nogc.
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 21:30:14 UTC, Steven Schveighoffer wrote: On 1/20/15 4:06 PM, ketmar via Digitalmars-d wrote: On Tue, 20 Jan 2015 15:51:17 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: p.s. another point is that all mechanics compiler needs for doing such checks is already there, so it's not a huge change to compiler codebase. it's not something that requires adding a whole new analysis code. You can always put @nogc on the dtor if you want. -Steve I got into habit of always doing that (after painfully learning on my own mistakes...) -- I guess the question of the thread is -- is there a SINGLE reason to not force this at the moment?
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 21:47:34 UTC, ketmar via Digitalmars-d wrote: and the funniest thing that it is perfectly able to do that! all the code is already there. Yes. But, destructors should be removed for GC anyway... it is a GC performance killer. So maybe it is better that they stay unreliable, politically... :-]
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 18:12:27 UTC, ketmar via Digitalmars-d wrote: Hello. as there is no possibility to doing GC allocations in class destructors, wouldn't it be nice to just force @nogc attribute on such dtors? i know, i know, this will break alot of code. i'm pretty sure that this will break alot of INVALID code, which better be broken at compile-time anyway. sure, we have alot of code of pre-@nogc era, and alot of code where authord didn't bother to add attributes at all. so we can introduce --force-dtor-nogc CLI arg and document this change, making it opt-in for, say, six month and opt-out after that. and i know that D devs (Walter at least) are resistant to command-line flags that changing compiler behavior. i don't know how to overcome this. say, by adding @gc attribute, which dfix can automatically add? but i still believe that instead of telling people again and again that they should not allocate in class destructors, we can use computer itself to track and stop this behavior. let's see how this proposal will be rejected. will there be some sane reasons, or only the good old song about broken code? make your bets! Not an error, Make it a warning.
Re: forcing @nogc on class destructors
On 1/20/15 4:06 PM, ketmar via Digitalmars-d wrote: On Tue, 20 Jan 2015 15:51:17 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: p.s. another point is that all mechanics compiler needs for doing such checks is already there, so it's not a huge change to compiler codebase. it's not something that requires adding a whole new analysis code. You can always put @nogc on the dtor if you want. -Steve
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 21:37:54 + via Digitalmars-d digitalmars-d@puremagic.com wrote: On Tuesday, 20 January 2015 at 21:29:40 UTC, Steven Schveighoffer wrote: How's that? The current runtime aborts on memory allocation inside the GC collection routine, it's not a memory safety issue. Spurious race conditions in memory deallocation patterns that remain undetected and cause random crashes after deployment are ok? That is not memory safety, it is memory safety. The language should prevent this. and the funniest thing that it is perfectly able to do that! all the code is already there. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 23:47:44 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 20 January 2015 at 23:17:28 UTC, deadalnix wrote: Concurrent GC is too expensive for a proper system level language. That is an unsubstanciated claim. And so is «pigs can't fly». That is the old prove a negative bullshit. You assert that something is too expensive for system language, which is not a negative, so the burden of proof is on you, substantiate your claim or you are just generating noise. Relying on cheap rhetorical trick is not going to fly (and you could add, no more than a pig).
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 15:51:17 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1/20/15 3:39 PM, ketmar via Digitalmars-d wrote: and all that mess can be avoided just by enforcing the one simple rule, which compiler is perfectly able to check. I think the current situation is fine. 1. There are functions that sometimes allocate. I don't want to forbid those, or force someone to write @nogc versions. 2. One can invoke destructors without being inside the GC. This seems like a good job for a lint tool. my point is that those who need to run lint doesn't do that. enforcing @nogc on class dtors protecting people who are relatively new to D, and used to do (maybe invisible) allocations in C++ dtors, for example. yes, this is bad practice even in C++, but it mostly works there. so they coming to D and often just doesn't know that class dtors are very special beasts. to give seasoned programmer some control it's enough to introduce `@gc` attribute. there are periodical requests for cancelling attributes in NG, so it's a perfect excuse to both make compiler more helpful for newcomers (yes, it's better to see error message in compile-time than to get some strange crashes in run-time), and to introduce cancelling attrs. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 22:25:05 UTC, deadalnix wrote: Any serious GC can run concurrently (instead of stopping the world). That mean any serious GC must be able to handle allocations while collecting. Concurrent GC is too expensive for a proper system level language. Stopping the thread/world is OK if you: 1. Statically determine what you need to scan to get full coverage (by static typing) so that you discriminate/classify pointers effectively. 2. Cluster all the pointers that needs scanning on the same cache lines by design. 3. Use exact (precise) scanning. 4. Use a collector that is carefully written for cache locality and minimize cache misses using batching. 5. Generate the runtime to take advantage of information from pre-linking static analysis. Remember that the memory bus can push up to 6GB/s. So in 5ms you can read up to 30MB, which is roughly 400.000 cache lines. So, you should be able to do fine with 50-100.000 GC allocated objects. That means a mix of regular heap and GC convenience is possible, even in a real time app, but you have to design for it all the way around (inclusive language constructs). Not only handling these case is unlikely to make the GC any slower, but in fact, this is required to make the GC faster. Deallocators means you have to track object boundaries and make partial deallocation (deallocating a class instance except a live member that still has a reference to it) more tricky. K.I.S.S. + limited use of GC + GC designed language constructs + good static analysis + generated runtime = fast collection if you write for it.
Re: forcing @nogc on class destructors
On 1/20/15 4:10 PM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Tuesday, 20 January 2015 at 20:51:18 UTC, Steven Schveighoffer wrote: On 1/20/15 3:39 PM, ketmar via Digitalmars-d wrote: and all that mess can be avoided just by enforcing the one simple rule, which compiler is perfectly able to check. I think the current situation is fine. In other words, memory safety is no longer a goal. Excellent! I hope that means the effort wasted on memory safety will be used to address the other weak spots instead, such as the syntax. How's that? The current runtime aborts on memory allocation inside the GC collection routine, it's not a memory safety issue. -Steve
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 22:16:31 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 20 January 2015 at 22:10:36 UTC, deadalnix wrote: That is GC implementation issue, not a language issue. If it ain't broken, don't fix it. Efficient GC with compile time safety is a language issue. Current Class allocation and deallocation patterns are making fast collection unlikely IMHO. Any serious GC can run concurrently (instead of stopping the world). That mean any serious GC must be able to handle allocations while collecting. Not only handling these case is unlikely to make the GC any slower, but in fact, this is required to make the GC faster.
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 21:29:40 UTC, Steven Schveighoffer wrote: How's that? The current runtime aborts on memory allocation inside the GC collection routine, it's not a memory safety issue. Spurious race conditions in memory deallocation patterns that remain undetected and cause random crashes after deployment are ok? That is not memory safety, it is memory safety. The language should prevent this.
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 20:51:18 UTC, Steven Schveighoffer wrote: On 1/20/15 3:39 PM, ketmar via Digitalmars-d wrote: and all that mess can be avoided just by enforcing the one simple rule, which compiler is perfectly able to check. I think the current situation is fine. 1. There are functions that sometimes allocate. I don't want to forbid those, or force someone to write @nogc versions. 2. One can invoke destructors without being inside the GC. This seems like a good job for a lint tool. -Steve +1 If you want to fix something, fix the possibility to escape the this reference. The @nogc thing is simply an limitation of the implementation of the GC.
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 18:12:27 UTC, ketmar via Digitalmars-d wrote: let's see how this proposal will be rejected. will there be some sane reasons, or only the good old song about broken code? make your bets! Lots of functions can theoretically allocate, but don't in the way you call them. For example, a function that checks for invalid arguments and throws an exception if any are passed in. It can't be @nogc because it throws, but it's perfectly valid to call in a constructor. Also, what about classes allocated with malloc/emplace that are then destroyed/freed?
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 20:51:18 UTC, Steven Schveighoffer wrote: On 1/20/15 3:39 PM, ketmar via Digitalmars-d wrote: and all that mess can be avoided just by enforcing the one simple rule, which compiler is perfectly able to check. I think the current situation is fine. In other words, memory safety is no longer a goal. Excellent! I hope that means the effort wasted on memory safety will be used to address the other weak spots instead, such as the syntax.
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 16:30:14 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1/20/15 4:06 PM, ketmar via Digitalmars-d wrote: On Tue, 20 Jan 2015 15:51:17 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: p.s. another point is that all mechanics compiler needs for doing such checks is already there, so it's not a huge change to compiler codebase. it's not something that requires adding a whole new analysis code. You can always put @nogc on the dtor if you want. seems that you completely missing my point. (sigh) signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 22:10:36 UTC, deadalnix wrote: That is GC implementation issue, not a language issue. If it ain't broken, don't fix it. Efficient GC with compile time safety is a language issue. Current Class allocation and deallocation patterns are making fast collection unlikely IMHO.
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 23:17:28 UTC, deadalnix wrote: Concurrent GC is too expensive for a proper system level language. That is an unsubstanciated claim. And so is «pigs can't fly». You want to run the collection when the GC-memory is hot in caches. That basically means you want to run it right after you have traversed the data structure. E.g. clean up dead long lived objects after an iteration in a simulation. A concurrent GC will slow you down, push you out of (soft) real time boundaries and keep dragging irrelevant stuff into the caches. You want a fast and predictable GC that can fire often, not a slow interleaved one (one way or the other you need to compensate).
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 20:51:34 -0500
Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com
wrote:
You can always put @nogc on the dtor if you want.
seems that you completely missing my point. (sigh)
Nope, not missing it. The mechanics are there. You just have to annotate.
that is where you missing it. your answer is like hey, C has all
mechanics for doing OOP with virtual methods and type checking, you
just have to write the code!
the whole point of my talk was free programmer from writing the
obvious and setup some red tapes for beginners.
What's the first thing you do if you aren't sure your destructors are
running?
~this() { writeln(in dtor); }
oops, sorry, can't do that, it's @nogc! I don't think this is a tenable
situation.
but it is! first: we can loosen that restriction somehow for `debug`
parts. your sample should be read like this then:
~this () { debug writeln(in dtor); }
second: it's `writeln` who is bad. one of the reasons that motivated me
to write my `iv.writer` was that `std.stdio.write` is not `@nogc`, and
so it was completely unusable in any of my `@nogc` functions. even
something that simple as `writeln(hi!)` was a disaster (both `@gc`
and `@canthrow`).
this is a sign that Phobos needs some simple output API that can be
used in `@nogc` and `nothrow` functions without hackery.
signature.asc
Description: PGP signature
Re: forcing @nogc on class destructors
On 1/20/15 4:37 PM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Tuesday, 20 January 2015 at 21:29:40 UTC, Steven Schveighoffer wrote: How's that? The current runtime aborts on memory allocation inside the GC collection routine, it's not a memory safety issue. Spurious race conditions in memory deallocation patterns that remain undetected and cause random crashes after deployment are ok? That is not memory safety, it is memory safety. It's neither spurious, nor a race condition. -Steve
Re: forcing @nogc on class destructors
On 1/20/15 9:04 PM, ketmar via Digitalmars-d wrote:
On Tue, 20 Jan 2015 20:51:34 -0500
Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com
wrote:
You can always put @nogc on the dtor if you want.
seems that you completely missing my point. (sigh)
Nope, not missing it. The mechanics are there. You just have to annotate.
that is where you missing it. your answer is like hey, C has all
mechanics for doing OOP with virtual methods and type checking, you
just have to write the code!
No, actually it's not. Adding @nogc to a function is as hard as writing
class when you want to do OOP.
the whole point of my talk was free programmer from writing the
obvious and setup some red tapes for beginners.
If he does it wrong, it gives him a stack trace on where to look. What
is different here than any other programming error?
What's the first thing you do if you aren't sure your destructors are
running?
~this() { writeln(in dtor); }
oops, sorry, can't do that, it's @nogc! I don't think this is a tenable
situation.
but it is! first: we can loosen that restriction somehow for `debug`
parts. your sample should be read like this then:
~this () { debug writeln(in dtor); }
second: it's `writeln` who is bad. one of the reasons that motivated me
to write my `iv.writer` was that `std.stdio.write` is not `@nogc`, and
so it was completely unusable in any of my `@nogc` functions. even
something that simple as `writeln(hi!)` was a disaster (both `@gc`
and `@canthrow`).
writeln(hi!) may need to extend a buffer, but probably not at this
point. This is what I mean by sometimes allocates.
this is a sign that Phobos needs some simple output API that can be
used in `@nogc` and `nothrow` functions without hackery.
You may be able to, but I don't see the point. writeln can work
perfectly fine for the most part inside dtors. But it can't be marked nogc.
-Steve
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 22:02:53 -0500 Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com wrote: You may be able to, but I don't see the point. writeln can work perfectly fine for the most part inside dtors. But it can't be marked nogc. now, that's really nice. it's wrong, it can fail, it even marked as one that can fail, but sometimes it works, so it's ok. i'm off. there is no sense in trying to bring any sanity in this discussion. i'm sorry. Ola was right: seems that safety is not in a list of primary goals. it seems that it's not in a list of goals at all. i hereby promise to not trying to propose anything in the future. it's pointless to proposing anything when wrong code is declared as acceptable code. i'd better go with my fork from now on and stop polluting NGs with my misunderstanding. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 23:47:44 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 20 January 2015 at 23:17:28 UTC, deadalnix wrote: Concurrent GC is too expensive for a proper system level language. That is an unsubstanciated claim. And so is «pigs can't fly». You want to run the collection when the GC-memory is hot in caches. That basically means you want to run it right after you have traversed the data structure. E.g. clean up dead long lived objects after an iteration in a simulation. A concurrent GC will slow you down, push you out of (soft) real time boundaries and keep dragging irrelevant stuff into the caches. You want a fast and predictable GC that can fire often, not a slow interleaved one (one way or the other you need to compensate). Strongly disagree, there is no silver bllet for memory and especially not for GCs. I'd imagine a game developer would prefer a concurrent GC over a stop the world because it's better to run slightly slower than full freeze.
Re: forcing @nogc on class destructors
On Wednesday, 21 January 2015 at 01:05:28 UTC, weaselcat wrote: there is no silver bllet for memory and Sorry, meant silver bullet for memory management, bit tired : )
Re: forcing @nogc on class destructors
On 1/20/15 4:39 PM, ketmar via Digitalmars-d wrote:
On Tue, 20 Jan 2015 16:30:14 -0500
Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com
wrote:
On 1/20/15 4:06 PM, ketmar via Digitalmars-d wrote:
On Tue, 20 Jan 2015 15:51:17 -0500
Steven Schveighoffer via Digitalmars-d digitalmars-d@puremagic.com
wrote:
p.s. another point is that all mechanics compiler needs for doing such
checks is already there, so it's not a huge change to compiler
codebase. it's not something that requires adding a whole new analysis
code.
You can always put @nogc on the dtor if you want.
seems that you completely missing my point. (sigh)
Nope, not missing it. The mechanics are there. You just have to annotate.
What's the first thing you do if you aren't sure your destructors are
running?
~this() { writeln(in dtor); }
oops, sorry, can't do that, it's @nogc! I don't think this is a tenable
situation.
-Steve
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 20:25:15 UTC, Meta wrote: Is it that subtle of a bug? Your program crashes once, you go on the forums and find the answer, and then you know never to do it again. Furthermore, this is something that seems like it'd be incredibly simple to add to dscanner (of course, everyone doesn't use dscanner). Someone who wants to avoid hidden allocations in a destructor will mark all their class destructors @nogc anyway, so enforcing it seems like it wouldn't add much value over what can already be done.
Re: forcing @nogc on class destructors
On Tuesday, 20 January 2015 at 18:25:42 UTC, ketmar via Digitalmars-d wrote: how likely this to be changed? is there *any* chances of that in 2015? 2016? and why we can't just remove that restriction when new GC will be implemented? removing the @nogc requirement on class dtors will break *nothing* *at* *all*. yet adding it now, while we don't have that new GC, will prevent alot of bugs that can slip in crack. btw, you won the prize of not talking about broken code! sadly, i forgot to setup the prizes... anyway, thanks for sane argument. Is it that subtle of a bug? Your program crashes once, you go on the forums and find the answer, and then you know never to do it again.
Re: forcing @nogc on class destructors
On Tue, 20 Jan 2015 20:25:13 + Meta via Digitalmars-d digitalmars-d@puremagic.com wrote: On Tuesday, 20 January 2015 at 18:25:42 UTC, ketmar via Digitalmars-d wrote: how likely this to be changed? is there *any* chances of that in 2015? 2016? and why we can't just remove that restriction when new GC will be implemented? removing the @nogc requirement on class dtors will break *nothing* *at* *all*. yet adding it now, while we don't have that new GC, will prevent alot of bugs that can slip in crack. btw, you won the prize of not talking about broken code! sadly, i forgot to setup the prizes... anyway, thanks for sane argument. Is it that subtle of a bug? Your program crashes once, you go on the forums and find the answer, and then you know never to do it again. and then you will inevitably do it again and again, 'cause compiler is silent about allocations in destructor. and you may accidentally call functions from another libraries which allocating. and then it can be hidden behind some `if`'s, so it will not crash for you, but will crash for someone other. and all that mess can be avoided just by enforcing the one simple rule, which compiler is perfectly able to check. Someone who wants to avoid hidden allocations in a destructor will mark all their class destructors @nogc anyway that if he KNOWS about hidden traps of allocating in class destructors. this way we can stop doing any checking at all, 'cause someone who knows what he wants will write the code to check what he wants anyway. signature.asc Description: PGP signature
Re: forcing @nogc on class destructors
On 1/20/15 3:39 PM, ketmar via Digitalmars-d wrote: and all that mess can be avoided just by enforcing the one simple rule, which compiler is perfectly able to check. I think the current situation is fine. 1. There are functions that sometimes allocate. I don't want to forbid those, or force someone to write @nogc versions. 2. One can invoke destructors without being inside the GC. This seems like a good job for a lint tool. -Steve
