Re: DIP 1008 Preliminary Review Round 1
On Friday, 26 May 2017 at 08:45:40 UTC, Walter Bright wrote: Why doesn't the rethrow count as a move? There is no way it can be reused in the scope that rethrows. -- Andrei It could be - it's just that there's nothing currently in the compiler to support the notion of moves. So the destructor will still wind up getting called on it. It's a good idea for a future enhancement, though. Even worth a hack right now to avoid ref counting for those exceptions IMO.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 26 May 2017 at 21:31:20 UTC, Steven Schveighoffer wrote: Part of this may mean clarifying what @nogc actually means. Does it mean no interaction with the GC system, or does it mean "cannot run a collection cycle"? I was pleased to find GC.addRange is now @nogc, so it seems potential interaction with the GC is OK so long as allocation and collection don't happen. Non-@nogc addRange was quite a blocker for smart pointer implementation. This does seem compatible with Walter's idea of not requiring linking of the GC runtime, at least in that the GC functions (such as addRange) can be stubbed out.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote: DIP 1008 is titled "Exceptions and @nogc". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md I would like the DIP to fully articulate the choice that it's facing, that special-casing the language for `throw new` comes with some downsides, but that the alternative is to force programmers to rewrite all their exceptions if they want to mark their functions `@nogc`. This requires an analysis of the similarities and differences between `throw new` and other uses of `new`. My personal impression is that exceptions do indeed fall into a distinct category, where they are easier to contain and shorter-lived than ordinary `new` memory. But the downside is that the language loses a little elegance by having another special case, and there might be corner cases where the special case is undesirable. I can only assume that at minimum, should this DIP be _rejected_, it should be replaced by some official documentation and methods on how to properly throw exceptions in `@nogc` code.
Re: DIP 1008 Preliminary Review Round 1
On 5/26/2017 11:50 PM, Atila Neves wrote: On Saturday, 27 May 2017 at 02:40:47 UTC, Walter Bright wrote: On 5/26/2017 11:51 AM, Atila Neves wrote: Since it's `scope`, where would it be copied to? This is assuming dip1000, of course. The rethrow case must be allowed: throw ex; Then either: 1. Elide the destructor call in that situation That's Andrei's "move semantics" idea, and it involves major code effort in the compiler. There's the "chain" case, too. 2. Make the developer write `throw ex.dup` The point is to not require them to rewrite their code. I'd prefer #2, since it would make a lot more sense for `scope`. `throw` escapes the ex IMHO. My $0.02 Atila
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 27 May 2017 at 02:40:47 UTC, Walter Bright wrote: On 5/26/2017 11:51 AM, Atila Neves wrote: Since it's `scope`, where would it be copied to? This is assuming dip1000, of course. The rethrow case must be allowed: throw ex; Then either: 1. Elide the destructor call in that situation 2. Make the developer write `throw ex.dup` I'd prefer #2, since it would make a lot more sense for `scope`. `throw` escapes the ex IMHO. My $0.02 Atila
Re: DIP 1008 Preliminary Review Round 1
On 5/26/2017 11:51 AM, Atila Neves wrote: Since it's `scope`, where would it be copied to? This is assuming dip1000, of course. The rethrow case must be allowed: throw ex;
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 27 May 2017 at 01:27:24 UTC, Walter Bright wrote: On 5/26/2017 11:58 AM, Atila Neves wrote: On Friday, 26 May 2017 at 16:36:16 UTC, Walter Bright wrote: On 5/26/2017 4:50 AM, Steven Schveighoffer wrote: But to answer your question, don't mark the exception scope in that case. The compiler should complain if you copy it outside the scope, no? I suspect if you proceed with that line of reasoning, you'll wind up in the same place I did with DIP 1008. Could you explain the line of reasoning that led you do dip1008? Thanks, Follow every construct that enables a reference to an Exception to escape and figure out what you're going to do about it. The idea would be that this only happens with `catch(scope T ex)`, so that the exception _can't_ escape. It can't be stored somewhere else, it can't be rethrown. It's going away so it's ok to call the destructor on it. After all, the call to free() must happen exactly once per Exception that was allocated with malloc(), and one must also deal with GC allocated Exceptions in the mix. GC exceptions would just not do any work in the destructor. A typical omission is not considering with the rethrow case nor the chaining case. I omitted the chaining case in my example for brevity. A production-grade implementation would call the destructors of the exceptions in the chain. Sorry for not making that clear. My idea is that every exception knows how to get rid of itself if by chance it's going away soon. And that if anyone wants to store a `@nogc` exception then that exception instance provides a `.dup` function. I got to this idea by considering what it would like if we could catch smart pointers: catch(RefCounted!Exception ex) Then I made it simpler. TBH I'd rather be able to catch a smart pointer. Counter-proposals that don't do this are legion and I've critiqued a lot of them for missing crucial cases. I'm getting more reluctant to continue doing that, so I ask submitters of them to be both much more complete, and if they are complete solutions, provide a rationale as to why they are better than DIP 1008 rather than being arguably equivalent. I understand that, I'd have the same attitude towards other proposals if I were you. I know and confess I tend to be overly optimistic and fail to consider everything that can go wrong. The devil is indeed in the details. Why do I think this scheme is better? 1. The programmer controls how allocation is done 2. Less of a special case (I admit it's a bit "special-casey") 3. Doesn't require another 2 druntime functions 4. Arguably easier to explain how it works Also keep in mind that DIP 1008 has an implementation sitting in the PR queue, and it's fairly simple. Solutions that require redesigning D or the compiler or both are not likely to get much traction. The list of things I have to do next reminds me of trying to run up the down escalator while it constantly increases speed. I could be completely wrong, but my intuition says that what I proposed wouldn't be too hard to implement either. Atila
Re: DIP 1008 Preliminary Review Round 1
sorry i replied to the wrong email in my inbox and didn't notice until it was already sent plz ignore me
Re: DIP 1008 Preliminary Review Round 1
what do you mean by this: "Look into http: vs https: link handling. Some errors with https" are you trying to publish https links or go to https://ink.vu links?
Re: DIP 1008 Preliminary Review Round 1
On 5/26/2017 2:31 PM, Steven Schveighoffer wrote:
"The destructor for Throwable will, if the refcount is 1, call
_d_delThrowable(e.next), i.e. on the head of the chained list of exceptions."
How does this work if the throwable is being destroyed by the GC? In this case,
e.next may be a dangling pointer.
The GC calls a finalizer which calls _d_delThrowable on the e.next value.
"it will call the new function _d_newThrowable() which will allocate E and
intialize it for refcounting."
Where is it allocated?
My implementation uses malloc().
If on the GC heap,
It isn't.
how can it be @nogc?
And that's how it can be @nogc.
If not, then how does it interact with GC pointers?
The refcount of 0 means it is GC allocated.
e.g. the e.next pointer may point to a
GC-allocated exception, how to stop the GC from collecting it early?
The usual way - register the roots with the GC. Take a look at the PRs for this,
they do that.
Part of this may mean clarifying what @nogc actually means. Does it mean no
interaction with the GC system, or does it mean "cannot run a collection cycle"?
@nogc means no GC allocations occur in the @nogc code, the same as usual, no
more, no less.
"In catch blocks, e is regarded as scope so that it cannot escape the catch
block."
If e is scope, then the destructor is called. However, since e is a reference
type, the data it points at isn't really stack allocated, and the scope-ness
ends at the reference, no? So what if you have something like this?
void *foo;
try
{
func();
}
catch(MyException e)
{
foo = e.voidptr; // get pointer to some member
}
Should foo be allowed to capture pieces of e? How does the compiler stop that if
not?
That's what DIP 1000 addresses. DIP 1008 relies on DIP 1000.
What expectations can e assume in terms of its members that are references?
Can it assume that it is in charge of the management of that memory if it's ref
counted, or does it need to assume GC usage for those items?
See DIP 1000.
Re: DIP 1008 Preliminary Review Round 1
On 5/26/2017 11:58 AM, Atila Neves wrote: On Friday, 26 May 2017 at 16:36:16 UTC, Walter Bright wrote: On 5/26/2017 4:50 AM, Steven Schveighoffer wrote: But to answer your question, don't mark the exception scope in that case. The compiler should complain if you copy it outside the scope, no? I suspect if you proceed with that line of reasoning, you'll wind up in the same place I did with DIP 1008. Could you explain the line of reasoning that led you do dip1008? Thanks, Follow every construct that enables a reference to an Exception to escape and figure out what you're going to do about it. After all, the call to free() must happen exactly once per Exception that was allocated with malloc(), and one must also deal with GC allocated Exceptions in the mix. A typical omission is not considering with the rethrow case nor the chaining case. Pragmatically speaking: Counter-proposals that don't do this are legion and I've critiqued a lot of them for missing crucial cases. I'm getting more reluctant to continue doing that, so I ask submitters of them to be both much more complete, and if they are complete solutions, provide a rationale as to why they are better than DIP 1008 rather than being arguably equivalent. Also keep in mind that DIP 1008 has an implementation sitting in the PR queue, and it's fairly simple. Solutions that require redesigning D or the compiler or both are not likely to get much traction. The list of things I have to do next reminds me of trying to run up the down escalator while it constantly increases speed.
Re: DIP 1008 Preliminary Review Round 1
On 5/19/17 11:45 AM, Mike Parker wrote:
DIP 1008 is titled "Exceptions and @nogc".
https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md
All review-related feedback on and discussion of the DIP should occur in
this thread. The review period will end at 11:59 PM ET on June 2 (3:59
AM GMT June 3), or when I make a post declaring it complete.
At the end of Round 1, if further review is deemed necessary, the DIP
will be scheduled for another round. Otherwise, it will be queued for
the formal review and evaluation by the language authors.
Extensive discussion of this DIP has already taken place in two threads,
both linked from the document. You may find it beneficial to skim
through those threads before posting any feedback here.
Thanks in advance to all who participate.
Destroy!
Some comments:
"The destructor for Throwable will, if the refcount is 1, call
_d_delThrowable(e.next), i.e. on the head of the chained list of
exceptions."
How does this work if the throwable is being destroyed by the GC? In
this case, e.next may be a dangling pointer.
"it will call the new function _d_newThrowable() which will allocate E
and intialize it for refcounting."
Where is it allocated? If on the GC heap, how can it be @nogc? If not,
then how does it interact with GC pointers? e.g. the e.next pointer may
point to a GC-allocated exception, how to stop the GC from collecting it
early?
Part of this may mean clarifying what @nogc actually means. Does it mean
no interaction with the GC system, or does it mean "cannot run a
collection cycle"?
"In catch blocks, e is regarded as scope so that it cannot escape the
catch block."
If e is scope, then the destructor is called. However, since e is a
reference type, the data it points at isn't really stack allocated, and
the scope-ness ends at the reference, no? So what if you have something
like this?
void *foo;
try
{
func();
}
catch(MyException e)
{
foo = e.voidptr; // get pointer to some member
}
Should foo be allowed to capture pieces of e? How does the compiler stop
that if not? What expectations can e assume in terms of its members that
are references? Can it assume that it is in charge of the management of
that memory if it's ref counted, or does it need to assume GC usage for
those items?
-Steve
Re: DIP 1008 Preliminary Review Round 1
On Friday, 26 May 2017 at 19:31:26 UTC, Steven Schveighoffer wrote: Note that the implication in your strawman is that you need a special exception to be nogc. I'd rather leave the (de)allocation of the exception up to the thrower/catcher, and have the exception not care about its own location. That the thrower should decide how to allocate, I agree with. Or at least have a sane default that can be optionally changed. But deallocation should just automagically work, otherwise people will forget to do it and end up with leaks at the very least. Atila
Re: DIP 1008 Preliminary Review Round 1
On 5/26/17 2:58 PM, Atila Neves wrote:
On Friday, 26 May 2017 at 11:50:40 UTC, Steven Schveighoffer wrote:
On 5/26/17 4:49 AM, Walter Bright wrote:
On 5/25/2017 4:54 PM, Atila Neves wrote:
I think maybe the problem isn't with `throw` but with `catch`. What if
instead we make it so that:
catch(scope T ex) { /*...*/; }
Means:
catch(scope T ex) { /*...*/; ex.__dtor; }
The trouble comes in when one starts copying exception references
around. Who then is responsible for destroying it?
This isn't the trouble. The trouble is that `new Exception` is not
@nogc, and there isn't a way to fix all existing exception code easily.
True, but a hypothetical `NoGcException.create` _is_ `@nogc` (same as my
MallocException), and is there really any difference between
reading/writing `new Foo` and `Foo.create`?
There isn't, but Foo.create doesn't exist in current code. Someone has
to go through it all and update.
Note that the implication in your strawman is that you need a special
exception to be nogc. I'd rather leave the (de)allocation of the
exception up to the thrower/catcher, and have the exception not care
about its own location.
Also, there's `enforce`, which is where I suspect a lot of throwing
happens anyway. It could use DbI to figure out from the exception type
what to do.
Good point, we could change enforce, and that would fix a lot of the
usage of exceptions.
My $0.02: Either we are going to make `new Exception` be @nogc, or we
are going to require people to type something different.
Or be able to customise `new T`, which feels less hacky than it
magically meaning something different if the type is a Throwable. I know
that there were class allocators in D1 and that they're depecreated but
I wasn't around back then to know why.
Deprecated, but still there.
However, it's the auto-destruction that isn't there. There isn't a way
to tell the compiler to destroy on the catch scope automatically.
Currently, when you override class allocation, you need to explicitly
delete.
-Steve
Re: DIP 1008 Preliminary Review Round 1
On Friday, 26 May 2017 at 16:36:16 UTC, Walter Bright wrote: On 5/26/2017 4:50 AM, Steven Schveighoffer wrote: But to answer your question, don't mark the exception scope in that case. The compiler should complain if you copy it outside the scope, no? I suspect if you proceed with that line of reasoning, you'll wind up in the same place I did with DIP 1008. Could you explain the line of reasoning that led you do dip1008? Thanks, Atila
Re: DIP 1008 Preliminary Review Round 1
On Friday, 26 May 2017 at 11:50:40 UTC, Steven Schveighoffer
wrote:
On 5/26/17 4:49 AM, Walter Bright wrote:
On 5/25/2017 4:54 PM, Atila Neves wrote:
I think maybe the problem isn't with `throw` but with
`catch`. What if
instead we make it so that:
catch(scope T ex) { /*...*/; }
Means:
catch(scope T ex) { /*...*/; ex.__dtor; }
The trouble comes in when one starts copying exception
references
around. Who then is responsible for destroying it?
This isn't the trouble. The trouble is that `new Exception` is
not @nogc, and there isn't a way to fix all existing exception
code easily.
True, but a hypothetical `NoGcException.create` _is_ `@nogc`
(same as my MallocException), and is there really any difference
between reading/writing `new Foo` and `Foo.create`?
Also, there's `enforce`, which is where I suspect a lot of
throwing happens anyway. It could use DbI to figure out from the
exception type what to do.
My $0.02: Either we are going to make `new Exception` be @nogc,
or we are going to require people to type something different.
Or be able to customise `new T`, which feels less hacky than it
magically meaning something different if the type is a Throwable.
I know that there were class allocators in D1 and that they're
depecreated but I wasn't around back then to know why.
Atila
Re: DIP 1008 Preliminary Review Round 1
On Friday, 26 May 2017 at 08:49:42 UTC, Walter Bright wrote:
On 5/25/2017 4:54 PM, Atila Neves wrote:
I think maybe the problem isn't with `throw` but with `catch`.
What if instead we make it so that:
catch(scope T ex) { /*...*/; }
Means:
catch(scope T ex) { /*...*/; ex.__dtor; }
The trouble comes in when one starts copying exception
references around. Who then is responsible for destroying it?
Since it's `scope`, where would it be copied to? This is assuming
dip1000, of course.
Atila
Re: DIP 1008 Preliminary Review Round 1
On 5/26/2017 4:50 AM, Steven Schveighoffer wrote: But to answer your question, don't mark the exception scope in that case. The compiler should complain if you copy it outside the scope, no? I suspect if you proceed with that line of reasoning, you'll wind up in the same place I did with DIP 1008.
Re: DIP 1008 Preliminary Review Round 1
On 5/26/17 4:49 AM, Walter Bright wrote:
On 5/25/2017 4:54 PM, Atila Neves wrote:
I think maybe the problem isn't with `throw` but with `catch`. What if
instead we make it so that:
catch(scope T ex) { /*...*/; }
Means:
catch(scope T ex) { /*...*/; ex.__dtor; }
The trouble comes in when one starts copying exception references
around. Who then is responsible for destroying it?
This isn't the trouble. The trouble is that `new Exception` is not
@nogc, and there isn't a way to fix all existing exception code easily.
But to answer your question, don't mark the exception scope in that
case. The compiler should complain if you copy it outside the scope, no?
My $0.02: Either we are going to make `new Exception` be @nogc, or we
are going to require people to type something different. IMO, I feel if
we can create a library solution, and use dfix or similar tool to allow
people to update their code, then we are in a much better place.
I need to review the DIP again, but last time I read it, there were some
issues I saw. I will post those in a while.
-Steve
Re: DIP 1008 Preliminary Review Round 1
On 5/25/2017 4:54 PM, Atila Neves wrote:
I think maybe the problem isn't with `throw` but with `catch`. What if instead
we make it so that:
catch(scope T ex) { /*...*/; }
Means:
catch(scope T ex) { /*...*/; ex.__dtor; }
The trouble comes in when one starts copying exception references around. Who
then is responsible for destroying it?
Re: DIP 1008 Preliminary Review Round 1
On 5/25/2017 12:30 AM, Andrei Alexandrescu wrote: On 5/25/17 6:24 AM, Walter Bright wrote: On 5/23/2017 3:40 PM, Martin Nowak wrote: Why does it have to be refcounted? Seems like there is only ever one reference to the current exception (the catch variable). Rethrowing the catch variable makes for 2 references. Why doesn't the rethrow count as a move? There is no way it can be reused in the scope that rethrows. -- Andrei It could be - it's just that there's nothing currently in the compiler to support the notion of moves. So the destructor will still wind up getting called on it. It's a good idea for a future enhancement, though.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:
DIP 1008 is titled "Exceptions and @nogc".
https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md
All review-related feedback on and discussion of the DIP should
occur in this thread. The review period will end at 11:59 PM ET
on June 2 (3:59 AM GMT June 3), or when I make a post declaring
it complete.
At the end of Round 1, if further review is deemed necessary,
the DIP will be scheduled for another round. Otherwise, it will
be queued for the formal review and evaluation by the language
authors.
Extensive discussion of this DIP has already taken place in two
threads, both linked from the document. You may find it
beneficial to skim through those threads before posting any
feedback here.
Thanks in advance to all who participate.
Destroy!
As many others, I dislike special-casing "throw new" and enough
has been said about that.
I also don't like adding hooks to druntime, or that by doing so
the allocation strategy is baked in (but I guess only when using
the `new` operator so there's that).
I think maybe the problem isn't with `throw` but with `catch`.
What if instead we make it so that:
catch(scope T ex) { /*...*/; }
Means:
catch(scope T ex) { /*...*/; ex.__dtor; }
That way whoever is throwing the exception allocates however he
or she sees fit and the memory is handled by the class's
destructor. Sort of proof of concept using malloc (if I did this
for real I'd use std.allocator):
class MallocException: Exception {
char* buffer;
size_t length;
// has to be public or `emplace` won't compile
this(string msg) @nogc {
import core.stdc.stdlib: malloc;
length = msg.length;
buffer = cast(char*)malloc(length);
buffer[0 .. length] = msg[];
super(cast(string)buffer[0 .. length]);
}
~this() @nogc {
import core.stdc.stdlib: free;
free(buffer);
free(cast(void*)this);
}
static MallocException create(string msg) @nogc {
import core.stdc.stdlib: malloc;
import std.conv: emplace;
enum size = __traits(classInstanceSize, MallocException);
auto buf = malloc(size);
return emplace!MallocException(buf[0 .. size], msg);
}
}
void main() @nogc {
try {
import core.stdc.stdlib: malloc;
throw MallocException.create("oops");
} catch(MallocException ex) {
ex.__dtor;
}
}
The code above works today, the annoying thing is having to
invoke the destructor manually (and also knowing it's called
`__dtor`).
This would however mean delaying making phobos @nogc since each
`throw` site would have to be changed.
And if someone never remembers to catch by scope somewhere and is
allocating memory themselves, well... use the GC, Luke.
Atila
Re: DIP 1008 Preliminary Review Round 1
On 5/25/17 6:24 AM, Walter Bright wrote: On 5/23/2017 3:40 PM, Martin Nowak wrote: Why does it have to be refcounted? Seems like there is only ever one reference to the current exception (the catch variable). Rethrowing the catch variable makes for 2 references. Why doesn't the rethrow count as a move? There is no way it can be reused in the scope that rethrows. -- Andrei
Re: DIP 1008 Preliminary Review Round 1
On 5/23/2017 3:40 PM, Martin Nowak wrote: Why does it have to be refcounted? Seems like there is only ever one reference to the current exception (the catch variable). Rethrowing the catch variable makes for 2 references. Has staticError been considered? It has a potential issue with multiple nested exceptions, but otherwise works fine. https://github.com/dlang/druntime/blob/bc832b18430ce1c85bf2dded07bbcfe348ff0813/src/core/exception.d#L683 Doesn't work for chained exceptions.
Re: DIP 1008 Preliminary Review Round 1
On Tuesday, 23 May 2017 at 22:40:43 UTC, Martin Nowak wrote: The proposal is a very mechanical fix, throwing several special cases at one specific problem. Why does it have to be refcounted? Seems like there is only ever one reference to the current exception (the catch variable). The only thing that seems necessary is to require scope on catch variable declarations, so that people do not escape the class reference, then this info might be used by the runtime to free exception objects after the catch handler is done. Amaury put a bit more words into that. http://forum.dlang.org/post/gtqsojgqqaorubcsn...@forum.dlang.org Has staticError been considered? It has a potential issue with multiple nested exceptions, but otherwise works fine. https://github.com/dlang/druntime/blob/bc832b18430ce1c85bf2dded07bbcfe348ff0813/src/core/exception.d#L683 I'm trying to understand your and Amaury's point. Normally, when you say `new` you get memory from the GC allocator. The @nogc attribute is supposed to prevent this, if I understand it correctly. Are you saying that `@nogc` as such is misconceived, because what a good language feature should really be doing is identifying and preventing new memory that _can't_ be deterministically destroyed? Is the problem here with the @nogc attribute? Because I think Walter's goal with this DIP is to make it so that you can put @nogc on _called_ functions that throw using `new`. Whereas your solution is to ignore that `new Exception` allocation, on account of the fact that you can deterministically destroy the Exception, provided you use `scope` catch blocks at, or above, the call site. Your solution might actually have its priorities straight, and `@nogc` may be designed badly because it clumps all GC allocations into one big basket. However, getting new memory from the GC still could trigger a collection cycle, which is what @nogc was created for, and simply knowing that you can reliably destroy the allocated memory doesn't change that. Thus, if I understand correctly, you and Amaury are arguing that `@nogc` as currently designed is a false goal to be chasing, that the more important goal is memory that can be deterministically destroyed, and therefore it distresses you that the language may be altered to chase the false prize of `@nogc` everywhere, instead of focusing on a real prize worth attaining?
Re: DIP 1008 Preliminary Review Round 1
On Monday, 22 May 2017 at 12:00:30 UTC, Nick Treleaven wrote:
//FIXME: uniqueToString should return @nogc UniquePtr!(const
char[])
import std.conv;
alias uniqueToString = to!(const char[]);
class MessageEx(E, sinkArgs...) : E
{
this(A...)(A args)
{
super(args);
}
//FIXME: delegate not @nogc
/*@nogc*/ override void toString(scope void delegate(in
char[]) sink) const
{
foreach (a; sinkArgs)
sink(uniqueToString(a));
}
}
unittest
{
auto x = 7;
throw new MessageEx!(Exception, "x = ", x)(null);
}
The result of uniqueToString would free any memory allocated
after the call to sink.
Heh, I actually ran into this problem earlier today and just saw
this post https://issues.dlang.org/show_bug.cgi?id=17420
Weird coincidence.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote: Extensive discussion of this DIP has already taken place in two threads, both linked from the document. You may find it beneficial to skim through those threads before posting any feedback here. Thanks in advance to all who participate. Destroy! The proposal is a very mechanical fix, throwing several special cases at one specific problem. Why does it have to be refcounted? Seems like there is only ever one reference to the current exception (the catch variable). The only thing that seems necessary is to require scope on catch variable declarations, so that people do not escape the class reference, then this info might be used by the runtime to free exception objects after the catch handler is done. Amaury put a bit more words into that. http://forum.dlang.org/post/gtqsojgqqaorubcsn...@forum.dlang.org Has staticError been considered? It has a potential issue with multiple nested exceptions, but otherwise works fine. https://github.com/dlang/druntime/blob/bc832b18430ce1c85bf2dded07bbcfe348ff0813/src/core/exception.d#L683
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 09:35:34 UTC, Stanislav Blinov wrote: On Saturday, 20 May 2017 at 02:25:45 UTC, Walter Bright wrote: void foo(scope string s); string s = callCAPIAndAllocateString(); foo(s ~ "abc"); What will happen? The compiler will generate different code? Won't compile? The former means invisible performance gap. The latter means an unpleasant surprise. Neither are good. Do we really need such special cases? It's no different from when s is GC allocated, s[] has to be copied for the concatenation. I'm not sure how Walter wants to lower this, but maybe it could use a region/stacked allocator. That could mean allocation in constant time when the region is already big enough, and deallocation would be constant time (except if freeing an unused region ahead of the current region).
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 02:05:21 UTC, Jonathan M Davis wrote:
What we would probably need would be to change msg is a
function which generates a message so that derived classes can
override that rather than passing a message string.
Further to Moritz's reply showing the existing toString overload
taking a delegate. This delegate is not @nogc. Otherwise I was
thinking of doing something like this:
//FIXME: uniqueToString should return @nogc UniquePtr!(const
char[])
import std.conv;
alias uniqueToString = to!(const char[]);
class MessageEx(E, sinkArgs...) : E
{
this(A...)(A args)
{
super(args);
}
//FIXME: delegate not @nogc
/*@nogc*/ override void toString(scope void delegate(in
char[]) sink) const
{
foreach (a; sinkArgs)
sink(uniqueToString(a));
}
}
unittest
{
auto x = 7;
throw new MessageEx!(Exception, "x = ", x)(null);
}
The result of uniqueToString would free any memory allocated
after the call to sink.
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 13:06:01 UTC, Jonathan M Davis wrote: ... Let's take the CTFE discussion to a different thread
Re: DIP 1008 Preliminary Review Round 1
On Saturday, May 20, 2017 12:34:09 PM PDT H. S. Teoh via Digitalmars-d wrote: > On Sat, May 20, 2017 at 07:53:58AM +, Stefan Koch via Digitalmars-d wrote: > > On Saturday, 20 May 2017 at 07:02:10 UTC, Walter Bright wrote: > > > Also, have a GC makes CTFE real nice. > > > > Having to implement a GC for newCTFE won't be nice though :o) > > I agree though being able to allocate memory makes ctfe much more > > useful then it would otherwise be. > > I think we might be able to get away without implementing a GC for > newCTFE. All you need is a memory pool allocator, i.e., a > bump-the-pointer algorithm to a block of memory (or maybe a linked list > of blocks if we need to make it growable) allocated when you enter CTFE, > then upon exiting CTFE, copy out the return value(s) and free the entire > pool. > > As long as we don't anticipate CTFE code that requires massive amounts > of allocation / deallocation before producing a result, this ought to be > good enough. Well, from the perspective of ther user's code, there really isn't any difference. They can allocate memory with new, and the compiler takes care of managing the memory. - Jonathan M Davis
Re: DIP 1008 Preliminary Review Round 1
On Sat, May 20, 2017 at 07:53:58AM +, Stefan Koch via Digitalmars-d wrote: > On Saturday, 20 May 2017 at 07:02:10 UTC, Walter Bright wrote: > > > > Also, have a GC makes CTFE real nice. > > Having to implement a GC for newCTFE won't be nice though :o) > I agree though being able to allocate memory makes ctfe much more > useful then it would otherwise be. I think we might be able to get away without implementing a GC for newCTFE. All you need is a memory pool allocator, i.e., a bump-the-pointer algorithm to a block of memory (or maybe a linked list of blocks if we need to make it growable) allocated when you enter CTFE, then upon exiting CTFE, copy out the return value(s) and free the entire pool. As long as we don't anticipate CTFE code that requires massive amounts of allocation / deallocation before producing a result, this ought to be good enough. T -- What do you get if you drop a piano down a mineshaft? A flat minor.
Re: DIP 1008 Preliminary Review Round 1
On Sat, May 20, 2017 at 03:05:44PM +, Stefan Koch via Digitalmars-d wrote: > On Saturday, 20 May 2017 at 14:59:37 UTC, Ola Fosheim Grøstad wrote: > > On Saturday, 20 May 2017 at 13:36:14 UTC, Stefan Koch wrote: > > > unions and other ABI-related things will be tricky. > > > > Isn't the unions issue quite easily solved by tagging behind the > > scenes? > > Ah tagging behind the scene is an option, it comes with runtime cost > though. And tagging would disallow the tricky and very common usecase > of overlaying and int and a float. > > This I understand, is heavily used. We only need this for floating-point operations. The current CTFE engine already special-cases floats for repainting, so we could potentially just special-case unions involving floats and leave everything else unsupported. Supporting unions in CTFE can be a bear. T -- Computers aren't intelligent; they only think they are.
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 15:05:44 UTC, Stefan Koch wrote: Ah tagging behind the scene is an option, it comes with runtime cost though. (I guess you meant compile-time cost) And tagging would disallow the tricky and very common usecase of overlaying and int and a float. This I understand, is heavily used. So this is allowed by the language spec? Anyway, since compile-time is harder to debug than run-time that seems to be a reasonable restriction. (C++ has this restriction as a general rule: one are only allowed to read from a union field if it was the most recently written to.)
Re: DIP 1008 Preliminary Review Round 1
On Saturday, May 20, 2017 3:05:44 PM PDT Stefan Koch via Digitalmars-d wrote: > On Saturday, 20 May 2017 at 14:59:37 UTC, Ola Fosheim Grøstad > > wrote: > > On Saturday, 20 May 2017 at 13:36:14 UTC, Stefan Koch wrote: > >> unions and other ABI-related things will be tricky. > > > > Isn't the unions issue quite easily solved by tagging behind > > the scenes? > > Ah tagging behind the scene is an option, it comes with runtime > cost though. > And tagging would disallow the tricky and very common usecase of > overlaying and int and a float. > > This I understand, is heavily used. std.bitmanip definitely uses that sort of trick (it also does it between a static array of ubytes) and integer types for stuff like swapping endianness). - Jonathan M Davis
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 14:59:37 UTC, Ola Fosheim Grøstad wrote: On Saturday, 20 May 2017 at 13:36:14 UTC, Stefan Koch wrote: unions and other ABI-related things will be tricky. Isn't the unions issue quite easily solved by tagging behind the scenes? Ah tagging behind the scene is an option, it comes with runtime cost though. And tagging would disallow the tricky and very common usecase of overlaying and int and a float. This I understand, is heavily used.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote: Destroy! In catch blocks, e is regarded as scope so that it cannot escape the catch block. ... Code that needs to leak the thrown exception object can clone the object. There's a contradiction here. Generic cloning cannot be implemented without storing exceptions for rethrowing later (in case member destructors throw). Furthermore: 2. Disallowing Exception objects with postblit fields. What about fields with destructors? I detect a double-free. If we need to clone, we need postblits and destructors, or neither. It looks as if that clause is added specifically to deal with cloning. Yet no information on cloning implementation is provided.
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 13:36:14 UTC, Stefan Koch wrote: unions and other ABI-related things will be tricky. Isn't the unions issue quite easily solved by tagging behind the scenes?
Re: DIP 1008 Preliminary Review Round 1
On Saturday, May 20, 2017 1:36:14 PM PDT Stefan Koch via Digitalmars-d wrote: > On Saturday, 20 May 2017 at 13:06:01 UTC, Jonathan M Davis wrote: > > Yeah, especially when you're not allowed to manually allocate > > memory in CTFE. :) > > > > And given that Stephan thinks that ref is too hard to implement > > in newCTFE (IIRC from what he said at dconf anyway), I'd hate > > to think what it would take to allow something like malloc or > > free. > > Did I say that ? > > ref is working. I thought you did, but I could have misremembered. Regardless, if it's working now, all the better. :) > unions and other ABI-related things will be tricky. unions are useful for certain things, but really, they're pretty terrible once you get beyond stuff like int and float. They _can_ be used safely for more complicated stuff, but it definitely does get tricky. And I'm sure that it's that much worse with CTFE. :( - Jonathan M Davis
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 13:06:01 UTC, Jonathan M Davis wrote: Yeah, especially when you're not allowed to manually allocate memory in CTFE. :) And given that Stephan thinks that ref is too hard to implement in newCTFE (IIRC from what he said at dconf anyway), I'd hate to think what it would take to allow something like malloc or free. Did I say that ? ref is working. unions and other ABI-related things will be tricky.
Re: DIP 1008 Preliminary Review Round 1
On Saturday, May 20, 2017 12:02:10 AM PDT Walter Bright via Digitalmars-d wrote: > On 5/19/2017 8:54 PM, Jonathan M Davis via Digitalmars-d wrote: > > And the reality of the matter is that using the GC has real benefits, > > and > > trying to avoid it comes at a real cost, much as a number of C++ > > progammers want to complain and deride as soon as they hear that D has > > a GC. And honestly, even having @nogc all over the place won't make > > many of them happy, because the GC is still in the language. > > Also, have a GC makes CTFE real nice. Yeah, especially when you're not allowed to manually allocate memory in CTFE. :) And given that Stephan thinks that ref is too hard to implement in newCTFE (IIRC from what he said at dconf anyway), I'd hate to think what it would take to allow something like malloc or free. - Jonathan M Davis
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 02:05:21 UTC, Jonathan M Davis wrote:
2. This really isn't going to fix the @nogc problem with
exceptions without either seriously overhauling how exceptions
are generated and printed or by having less informative error
messages. The problem is with how exception messages are
generated. They take a string, and that pretty much means that
either they're given a string literal (which can be @nogc but
does not allow for customizing the error message with stuff
like what the bad input was), or they're given a constructed
string (usually by using format) - and that can't be @nogc.
And you can't even create an alternate constructor to get
around the problem. Everything relies on the msg member which
is set by the constructor. Code that wants the message accesses
msg directly, and when the exception is printed when it isn't
caught, it's msg that is used. Not even overiding toString gets
around the issue. For instance, this code
class E : Exception
{
this(int i, string file = __FILE__, size_t line = __LINE__)
{
super("no message", file, line);
_i = i;
}
override string toString()
{
import std.format;
return format("The value was %s", _i);
}
int _i;
}
void main()
{
throw new E(42);
}
prints
foo.E@foo.d(20): no message
[...]
---
class E : Exception
{
this(int i, string file = __FILE__, size_t line = __LINE__)
{
super("no message", file, line);
_i = i;
}
override void toString(scope void delegate(in char[]) sink)
const
{
import std.format;
sink(format("The value was %s", _i));
}
int _i;
}
void main()
{
throw new E(42);
}
---
prints "The value was 42".
Personally, I use a string literal for msg in the constructor,
add some value members to the exception, and then override the
above toString.
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 03:54:43 UTC, Jonathan M Davis wrote: Because of the issue of lifetimes, some language features simply cannot be implemented without the GC, and I think don't see any point in trying to make it so that you can use all features of D without the GC. That simply won't work. By the very nature of the language, completely avoiding the GC means completely avoiding some features. Even with the GC we have guns to shoot us in the foot with, which are .destroy() and GC.free(). The GC itself is not an issue at all, it's the lack of choice in the language that is the problem. @nogc attribute alone is not enough. D's dynamic arrays fundamentally require the GC, because they do not manage their own memory. They're just a pointer and a length and literally do not care what memory backs them. As long as all you're doing is slicing them and passing them around (i.e. restrict yourself to the range-based functions), then the GC is not involved, and doesn't need to be, but as soon as you concatenate or append, the GC has to be involved. For that not to be the case, dynamic arrays would have to manage their own memory (e.g. be ref-counted), which means that they could not be what they are now. A different data structure would be required. That is not necessary. See my previous comment. We can amend the type system so it understands when it can't use the GC. // Syntax is temporary, for illustration purposes. It is currently ambiguous with the language int[] (nogc) myArray; auto a = myArray ~ [1,2,3]; // error, cannot concatenate nogc and gc arrays. auto b = myArray ~ [1,2,3] (myAllocator); // implies compiler-generated: // auto block = myAllocator.reallocate(myArray, (myArray.length + 3)*int.sizeof); // handle out-of-memory, etc... // int[] (nogc) result = (cast(int[])block.ptr)[0..myArray.length+3]; // return result; Yes, verbose, and yes, ugly. Manual memory management is that. But just flat-out forbidding users to use certain features is no less verbose and no less ugly. Similarly, stuff like closures require the GC. They need something to manage their memory. They're designed to be automatic. They were designed long ago, perhaps that design needs revisiting. They absolutely do not *have* to manage their memory. It is convenient when they do and very pleasant when working with GC. But that makes them a niche feature at best. If the user is given a little bit more control over captures, we'd get more cases when allocation is not needed. If the user is given control of the allocation itself, even better, as it gives them back a feature taken away by the GC. Explicit (nogc) requirement can be devised for the closures too, we just need to put effort into that instead of silently ignoring it. Honestly, I think that this push for @nogc and manual memory mangement is toxic. Yes, we should strive to not require the GC where reasonable, but some things simply are going to require the GC to work well, and avoiding the GC very quickly gives you a lot of the problems that you have with languages like C and C++. For instance, at dconf, Atila talked about the D wrapper for excel that he wrote. He decided to use @nogc and std.exception.allocator, and not only did that make it much harder for him to come up with a good, workable design, it meant that he suddenly had to deal with memory corruption bugs that you simply never have with the GC. He felt like he was stuck programming in C++ again - only worse, because he had issues with valgrind that made it so that he couldn't effectively use it to locate his memory corruption problems. That is *mostly* due to the lack of facilities in the language *and* standard library. It's not written with manual memory management in mind and so does not provide any ready-made primitives for that, which means you have to write your own, which means you will have bugs. At least more bugs than you would've had have you had the help. The GC makes it far easier to write clean, memory-safe code. It is a _huge_ boon for us to have the GC. Yes, there are cases where you can't afford to use the GC, or you have to limit its use in order for your code to be as performant as it needs to be, but that's the exception, not the norm. And avoiding the GC comes at a real cost. All is true except the last sentence. The cost should not be huge, but for that the language has to work with us. Explicitly, without any "special cases". And the reality of the matter is that using the GC has real benefits, and trying to avoid it comes at a real cost, much as a number of C++ progammers want to complain and deride as soon as they hear that D has a GC. And honestly, even having @nogc all over the place won't make many of them happy, because the GC is still in the language. If people simply want to assume the GC is bad and turn away, let them. The community or the language won't suffer from it. OTOH, for pe
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 02:25:45 UTC, Walter Bright wrote: We'll be doing more invisible special casing in the future. For example, void foo(scope string s); ... string s; ... foo(s ~ "abc"); This array concatenation does not need to be done with the GC. It's not fundamentally different from what the compiler does now with: s = "abc" ~ "def"; not doing a GC allocation, either. string s = callCAPIAndAllocateString(); foo(s ~ "abc"); What will happen? The compiler will generate different code? Won't compile? The former means invisible performance gap. The latter means an unpleasant surprise. Neither are good. Do we really need such special cases? The compiler also detects when it can allocate a closure on the stack rather than on the GC. We expect the compiler to do such transformations. You're saying "detects" as if it always does that, which is not true. And this avoids the issue, not addresses it. We have *no* control over how the closure is allocated. It's either none, or GC. Which means we can't have dynamically-allocated closures in @nogc code. Which forces user to write verbose code (i.e. structs with opCall), losing the benefit of anonymous functions altogether. In general, if the compiler can determine the lifetime of an allocation, and can control "leakage" of any references to that allocation, then it is fair game to not use the GC for it. *If*. And if it can't, it won't compile. Which is: a) Frustrating when it should've detected it. b) Makes you think of allocating things manually from the start, to not deal with sudden failure to compile. The recent 'scope' improvements have uncovered a lot more opportunities for this, and DIP 1008 is the first fruit of it. That is true, benefits of 'scope' are indeed huge. But please, consider how fragile all the "special-casing" is. It's based on preconditions and assumptions made by the compiler, and is beyond user control. It hides potential maintenance problems.
Re: DIP 1008 Preliminary Review Round 1
On Saturday, 20 May 2017 at 07:02:10 UTC, Walter Bright wrote: Also, have a GC makes CTFE real nice. Having to implement a GC for newCTFE won't be nice though :o) I agree though being able to allocate memory makes ctfe much more useful then it would otherwise be.
Re: DIP 1008 Preliminary Review Round 1
On 5/19/2017 8:54 PM, Jonathan M Davis via Digitalmars-d wrote: And the reality of the matter is that using the GC has real benefits, and trying to avoid it comes at a real cost, much as a number of C++ progammers want to complain and deride as soon as they hear that D has a GC. And honestly, even having @nogc all over the place won't make many of them happy, because the GC is still in the language. Also, have a GC makes CTFE real nice.
Re: DIP 1008 Preliminary Review Round 1
On Friday, May 19, 2017 11:35:54 AM PDT H. S. Teoh via Digitalmars-d wrote: > I agree with this, and having looked at std.experimental.allocator > recently, I think Andrei may even have made certain design decisions > with this in mind. I think the ideal goal (I'm not sure how achievable > it is) would be to make theAllocator part of *druntime*, upon which you > can actually use array concatenations, AA's, etc., (almost) freely, and > the user would be able to control exactly how allocations would work. At > least in the current state of things, I don't see this as being > particularly possible due to various issues, the main one being that > code written with GC in mind generally does not track lifetimes, which > is a big obstacle to successfully being able to just assign a different > allocator to theAllocator and have things Just Work(tm). Because of the issue of lifetimes, some language features simply cannot be implemented without the GC, and I think don't see any point in trying to make it so that you can use all features of D without the GC. That simply won't work. By the very nature of the language, completely avoiding the GC means completely avoiding some features. D's dynamic arrays fundamentally require the GC, because they do not manage their own memory. They're just a pointer and a length and literally do not care what memory backs them. As long as all you're doing is slicing them and passing them around (i.e. restrict yourself to the range-based functions), then the GC is not involved, and doesn't need to be, but as soon as you concatenate or append, the GC has to be involved. For that not to be the case, dynamic arrays would have to manage their own memory (e.g. be ref-counted), which means that they could not be what they are now. A different data structure would be required. Similarly, stuff like closures require the GC. They need something to manage their memory. They're designed to be automatic. Honestly, I think that this push for @nogc and manual memory mangement is toxic. Yes, we should strive to not require the GC where reasonable, but some things simply are going to require the GC to work well, and avoiding the GC very quickly gives you a lot of the problems that you have with languages like C and C++. For instance, at dconf, Atila talked about the D wrapper for excel that he wrote. He decided to use @nogc and std.exception.allocator, and not only did that make it much harder for him to come up with a good, workable design, it meant that he suddenly had to deal with memory corruption bugs that you simply never have with the GC. He felt like he was stuck programming in C++ again - only worse, because he had issues with valgrind that made it so that he couldn't effectively use it to locate his memory corruption problems. The GC makes it far easier to write clean, memory-safe code. It is a _huge_ boon for us to have the GC. Yes, there are cases where you can't afford to use the GC, or you have to limit its use in order for your code to be as performant as it needs to be, but that's the exception, not the norm. And avoiding the GC comes at a real cost. I have no problem whatsoever telling folks that some features of D require the GC and that while D's GC is optional, if you avoid it, you avoid certain features. There is no free lunch. We do need to make sure that we do not accidentally or erroneously require the GC so that code can work with @nogc when folks require that if it's reasonable for that code to be @nogc. Where reasonable, allocators should be an option so that folks can decide whether to use the GC or not for a particular piece of code. But sometimes, it's just not reasonable to expect the same functionality when not using the GC as you get when using the GC. And the reality of the matter is that using the GC has real benefits, and trying to avoid it comes at a real cost, much as a number of C++ progammers want to complain and deride as soon as they hear that D has a GC. And honestly, even having @nogc all over the place won't make many of them happy, because the GC is still in the language. - Jonathan M Davis
Re: DIP 1008 Preliminary Review Round 1
On 5/19/2017 6:23 PM, Meta wrote: Like others, I do not like the special-casing of `throw new`. However, several designs have already been explored and found lacking. This proposal also has the advantage that it (hopefully) doesn't break existing code and all existing code gets the benefit for free. I think this benefit has been hugely underestimated; we must consider that large swathes of code that were previously non-@nogc due to exception allocation can now be *inferred automatically* to be @nogc. That's a huge advantage and may be worth the special case. We'll be doing more invisible special casing in the future. For example, void foo(scope string s); ... string s; ... foo(s ~ "abc"); This array concatenation does not need to be done with the GC. It's not fundamentally different from what the compiler does now with: s = "abc" ~ "def"; not doing a GC allocation, either. The compiler also detects when it can allocate a closure on the stack rather than on the GC. We expect the compiler to do such transformations. In general, if the compiler can determine the lifetime of an allocation, and can control "leakage" of any references to that allocation, then it is fair game to not use the GC for it. The recent 'scope' improvements have uncovered a lot more opportunities for this, and DIP 1008 is the first fruit of it.
Re: DIP 1008 Preliminary Review Round 1
On Friday, May 19, 2017 3:45:28 PM PDT Mike Parker via Digitalmars-d wrote:
> DIP 1008 is titled "Exceptions and @nogc".
>
> https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md
>
> All review-related feedback on and discussion of the DIP should
> occur in this thread. The review period will end at 11:59 PM ET
> on June 2 (3:59 AM GMT June 3), or when I make a post declaring
> it complete.
>
> At the end of Round 1, if further review is deemed necessary, the
> DIP will be scheduled for another round. Otherwise, it will be
> queued for the formal review and evaluation by the language
> authors.
>
> Extensive discussion of this DIP has already taken place in two
> threads, both linked from the document. You may find it
> beneficial to skim through those threads before posting any
> feedback here.
>
> Thanks in advance to all who participate.
Overall, I think that this is a decent proposal for making exceptions @nogc
- and exceptions are definitely one of the few things that prevents code
that
really should be @nogc from being @nogc. However, I do have two objections:
1. I thought that we were going to come up with a general solution for
ref-counted classes, not something for just exceptions. If that's the case,
then this makes a lot less sense. However, that being said, if this solution
can be transparently transitioned to a more general ref-counting solution,
then this is probably a good stop-gap solution (and if we can't get
ref-counted classes for some reason, then at least we'd have it for
exceptions).
2. This really isn't going to fix the @nogc problem with exceptions without
either seriously overhauling how exceptions are generated and printed or by
having less informative error messages. The problem is with how exception
messages are generated. They take a string, and that pretty much means that
either they're given a string literal (which can be @nogc but does not allow
for customizing the error message with stuff like what the bad input was),
or they're given a constructed string (usually by using format) - and that
can't be @nogc.
And you can't even create an alternate constructor to get around the
problem. Everything relies on the msg member which is set by the
constructor. Code that wants the message accesses msg directly, and when the
exception is printed when it isn't caught, it's msg that is used. Not even
overiding toString gets around the issue. For instance, this code
class E : Exception
{
this(int i, string file = __FILE__, size_t line = __LINE__)
{
super("no message", file, line);
_i = i;
}
override string toString()
{
import std.format;
return format("The value was %s", _i);
}
int _i;
}
void main()
{
throw new E(42);
}
prints
foo.E@foo.d(20): no message
??:? _Dmain [0xd0d473ce]
??:? _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFNlZv
[0xd0d526db]
??:? scope void rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).tryExec(scope void delegate()) [0xd0d52607]
??:? scope void rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).runAll() [0xd0d52684]
??:? scope void rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).tryExec(scope void delegate()) [0xd0d52607]
??:? _d_run_main [0xd0d52577]
??:? main [0xd0d5040b]
??:? __libc_start_main [0xf798a3f0]
toString wasn't used anywhere. toString would only be used if the exception
were caught and printed, e.g.
void main()
{
import std.stdio;
writeln(new E(42));
}
would print
The value was 42
whereas without the toString override, it would print
foo.E@foo.d(15): no message
So, as is stands, we need to pass strings to exception constructors, and if
we want those messsages to be informative, they cannot be string literals
and thus have to be GC-allocated if you don't want to risk memory leaks or
accessing memory that's no longer valid. And Walter's proposal does nothing
to fix that, so even with his proposal, it's not going to work well to have
code that throws exceptions be @nogc and have the error messages be useful.
What we would probably need would be to change msg is a function which
generates a message so that derived classes can override that rather than
passing a message string. Then at least the allocation of the string would
just happen when the exception was printed. And if it's okay to make getting
the exception @system, then it could even return a const(char)[] to a member
variable (be it a static array or a string) and potentially avoid allocating
altogether. Alternatively, if we're willing to have only one output range
type work with it (so that the function can be virtual), then we could
replace msg with a function that took an output range.
Regardless, with how exceptions are currently constructed, I don't think
that Walter's proposal goes far enough to actually fix the problem with
@nogc and exceptions, even if it does solve a critical piece of the prob
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote: DIP 1008 is titled "Exceptions and @nogc". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on June 2 (3:59 AM GMT June 3), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Extensive discussion of this DIP has already taken place in two threads, both linked from the document. You may find it beneficial to skim through those threads before posting any feedback here. Thanks in advance to all who participate. Destroy! Like others, I do not like the special-casing of `throw new`. However, several designs have already been explored and found lacking. This proposal also has the advantage that it (hopefully) doesn't break existing code and all existing code gets the benefit for free. I think this benefit has been hugely underestimated; we must consider that large swathes of code that were previously non-@nogc due to exception allocation can now be *inferred automatically* to be @nogc. That's a huge advantage and may be worth the special case. Because of the transitive nature of attributes, even one function in the call graph that is non-@nogc "paints" connecting nodes which in turn paint *their* connecting nodes, etc.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 21:24:51 UTC, Adam D. Ruppe wrote:
"NewExpressions are used to allocate memory on the garbage
collected heap (default) or using a class or struct specific
allocator. "
"If a NewExpression is used as an initializer for a function
local variable with scope storage class, and the ArgumentList
to new is empty, then the instance is allocated on the stack
rather than the heap or using the class specific allocator. "
IMHO, this has to go. Having alignment control now, and with
DIP1000 solving reference escaping, there's absolutely no need in
this special syntax; stack-allocated classes are possible as
library types.
But it may be beneficial to reconsider the 'new (AllocatorOpts)'
syntax, with more thought on interaction with the type system. As
in, it's not necessary to have type-specific allocation
functions. But some sort of type system flag is required if we
want to make the language aware of our allocation schemes. To
expand on my previous reply, something like this comes to mind:
// Delcaring class/struct not supporting being allocated by a
non-@nogc allocator:
class [(nogc)] ClassName [(template parameters)] { ... }
struct [(nogc)] StructName [(template parameters)] { ... }
// Declaring arrays:
T[][(nogc)] arr;
So it could look like this:
class Allocator
{
void[] allocate(size_t, TypeInfo ti = null) @nogc { ... }
void deallocate(void[]) @nogc { ... }
}
Allocator myAllocator = /* however is desired */;
class (nogc) MyClass
{
int[] gcArray;
int[] (nogc) nonGCArray;
// note the ctor itself is not @nogc, since it allocates
// gcArray, so interoperability is possible
this()
{
gcArray = [1, 2]; // fine
nonGCArray = [1, 2]; // error
nonGCArray = new (myAllocator) int[2];
}
~this()
{
myAllocator.dispose(nonGCArray);
}
}
auto a = new MyClass; // error, no allocator provided, GC
assumed, MyClass cannot be allocated by GC
auto b = new (myAllocator) MyClass; // fine
auto c = new (theAllocator) MyClass; // error, theAllocator is
not @nogc
---
Not a very pretty syntax, but I can't think of a way of making it
any prettier...
Bringing this back to exceptions, it should "just work":
class (nogc) NoGCException : Exception { ... }
throw new (myAllocator) Exception("Argh!");
//...
catch (NoGCException e) {
} // error, e is not rethrown or disposed
catch (NoGCException e) {
myAllocator.dispose(e);
} // fine, e was disposed
This, however, means teaching the language a few extra library
constructs. And of course, there's the danger of deallocating
with the wrong allocator, but being careful comes with the
territory as far as memory management is concerned.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 19:46:07 UTC, nkm1 wrote: I like this proposal... But it looks like people are concerned about 'new' becoming contextual keyword (that in some contexts it allocates with GC and in others it does something else) It *already* does that. `new` can be overloaded and modified with `scope`. http://dlang.org/spec/expression.html "NewExpressions are used to allocate memory on the garbage collected heap (default) or using a class or struct specific allocator. " "If a NewExpression is used as an initializer for a function local variable with scope storage class, and the ArgumentList to new is empty, then the instance is allocated on the stack rather than the heap or using the class specific allocator. " I also believe the compiler *should* be free to optimize it to a different allocation scheme if it proves it safely can.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 19:46:07 UTC, nkm1 wrote:
As someone who iis interested in @nogc (more precisely: in
avoiding GC pauses), I like this proposal... But it looks like
people are concerned about 'new' becoming contextual keyword
(that in some contexts it allocates with GC and in others it
does something else). So maybe different syntax can be used?
How about "throw scope Exception"? ("scope" already does
various things :)
This syntax was already proposed and mostly rejected by the
community. See the original thread listed in the DIP.
But memory management is not only about allocations, it's also
about deallocations! Manually deallocating stuff (like freeing
exception objects) is error prone and bothersome.
That's why we have the GC.
Refcounted exceptions seems to me a pretty good solution (and I
think the general plan for D is to use more reference counting?
At least, that's my impression...)
That was my impression too. However Walter has made it clear that
@safe ref-counting of classes is not planned as he doesn't think
it's possible. I'm unclear on the details of that, so you'll have
to ask him.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 18:16:33 UTC, Jack Stouffer wrote:
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:
...
Secondly, I'm not a fan of special casing syntax, especially
when I don't think the given benefits outweigh the above listed
costs...
You're raising an extremely important point. Special-casing
exceptions in such a way, while may help *some* code, is still a
crutch, and a very ugly one at that. The bigger issue of @nogc is
the implicit allocations. It would be much better to solve the
fundamental problem, that is, give user code optional, but
complete control over memory allocation.
All three main areas need to be addressed in concert: classes,
arrays and delegates. Working on just one area, and even just a
subset of it (exceptions) in the long run will further the
problem, not help solve it.
As an example, operator new could accept additional syntax (outer
[] mean "optional"):
new [(TAllocator)] Type [[count]] [(init)]
Where TAllocator is a class providing at least minimal allocator
interface (allocate/deallocate). @nogc can be inferred from the
allocator.
This lets user code decide on their class and array allocations.
OTOH, this will incur storage penalty, since all
dynamically-allocated objects (including arrays) will have to
carry the allocator reference with them. It doesn't, however,
solve the @nogc inference in general, since allocator is not part
of the type. Attempting to concatenate two arrays at a distance
from their allocation site would not be considered @nogc. That
being said, arrays with complete inference support could be made
library types, so perhaps that is not *that* big of an issue.
Delegates, as they are now, require a lot more attention.
Firstly, there is the dubious design decision that any outer
scope access is always by reference, which imposes the GC
allocation of closures in the first place. @nogc cannot be solved
for delegates without addressing control over captures. Then we
have the allocation itself, in cases when it is desired. And that
will completely destroy existing delegate syntax.
There is, however, another side to this problem, and that is
interaction of exceptions with the runtime. Allowing users to
employ any desired allocation scheme precludes reliable exception
handling, since the runtime effectively loses control over
exception lifetime. Catching garbage in a catch(...) block would
never be a pleasant surprise. That being said, it's not that it
is in control right now either:
void innocent() {
throw new Exception("I am innocent");
}
void malicious() {
try {
innocent();
} catch (Exception e) {
// innocent, eh? well I am not...
e.destroy;
throw e;
}
}
void oblivious() {
try {
malicious();
} catch (Exception e) {
writeln("Caught: ", e.msg);
}
}
void main() {
oblivious();
}
Hello, segmentation fault. If users want to be difficult, they'd
find a way :)
So perhaps, instead of tackling exceptions on their own, we
really should focus on allocation in general. Everything else
should (hopefully) come automagically.
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:
DIP 1008 is titled "Exceptions and @nogc".
As someone who iis interested in @nogc (more precisely: in
avoiding GC pauses), I like this proposal... But it looks like
people are concerned about 'new' becoming contextual keyword
(that in some contexts it allocates with GC and in others it does
something else). So maybe different syntax can be used? How about
"throw scope Exception"? ("scope" already does various things :)
Instead, we should be asking... How can allocations in Phobos
and user code be transferred to an allocation strategy of the
user's choosing/needs?
But memory management is not only about allocations, it's also
about deallocations! Manually deallocating stuff (like freeing
exception objects) is error prone and bothersome. Refcounted
exceptions seems to me a pretty good solution (and I think the
general plan for D is to use more reference counting? At least,
that's my impression...)
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 18:10:50 UTC, H. S. Teoh wrote: On Fri, May 19, 2017 at 05:48:55PM +, Stanislav Blinov via Digitalmars-d wrote: On Friday, 19 May 2017 at 17:05:09 UTC, H. S. Teoh wrote: [...] > AFAIK, there is no way to clone classes, unless the class > writer implemented it explicitly. Only arrays support .dup, > no other type does (unless user code explicitly implements > it, or its equivalent). > > > T Well, not that it's a complete implementation or anything, but it definitely could be done with a library: /// Make a shallow clone of a class instance /// (members that are classes are not cloned) [...] That's the problem right there: you cannot guarantee that a shallow copy will have the correct semantics. For example, modifying a class member through the original copy will also modify what is seen through the clone, which may not be the intention. That's true. Also, you have to take care that if the class member is ref-counted, the clone function needs to increment the refcount, otherwise you could end up with a dangling pointer. It does call postblits. But of course, a complete implementation should do more than that. It probably has to lock the monitor for the duration, correctly annotate the GC block if memory is being GC-allocated, etc... Meaning, in the general case, you need to be careful about calling postblits and such. Furthermore, a generic deep-copying clone function may not be possible in the general case... Of course. For such cloning to be officially supported by the language, at least one additional primitive is needed: __postblit for classes. This will let user code correctly handle cloning as needed. Surely, a much better solution would be actual owning/non-owning pointers and references, but I doubt we'd ever get there.
Re: DIP 1008 Preliminary Review Round 1
On Fri, May 19, 2017 at 06:16:33PM +, Jack Stouffer via Digitalmars-d wrote: [...] > Instead, we should be asking > > > How can allocations in Phobos and user code be transferred to an > > allocation strategy of the user's choosing/needs? > > If the user values code simplicity and safety, then the GC is fine, > and template functions which allocate can then be marked as @safe due > to their use of GC. But if the user needs other speed or usage > requirements, then the user needs to offer something else, and that > will automatically be marked as @system. I think that integrating > std.allocator across Phobos in a consistent manner will allow this, > where functions can accept allocators, and their internal allocations > can all be done on the given allocator. This would also allow > exceptions and other objects can be manually freed after a function is > called if needed, or just left to the GC to collect if the GC > allocator was passed to the function. I agree with this, and having looked at std.experimental.allocator recently, I think Andrei may even have made certain design decisions with this in mind. I think the ideal goal (I'm not sure how achievable it is) would be to make theAllocator part of *druntime*, upon which you can actually use array concatenations, AA's, etc., (almost) freely, and the user would be able to control exactly how allocations would work. At least in the current state of things, I don't see this as being particularly possible due to various issues, the main one being that code written with GC in mind generally does not track lifetimes, which is a big obstacle to successfully being able to just assign a different allocator to theAllocator and have things Just Work(tm). Short of putting theAllocator in druntime outright, the next best thing would be for all Phobos code to use it by default, eschewing any constructs that might bypass it. This seems more attainable, though it does imply a LOT of churn in Phobos, and may require significant time and effort to pull off. Some Phobos code, of course, might be able to take in allocators as template parameters or some such mechanism of finer-grained control, but given the very large amount of code we're dealing with, defaulting to theAllocator seems like a (slightly) more reachable goal. > In summary, I believe what Phobos needs is a wholistic approach to the > problem of memory management, and not a bunch of small solutions over > time. To be sure, this is a much harder task, but I think if it's > done right, with D we can have the first language which has compete > user control to those who need it while offering ease of use to > everyone else. Yes, D needs to return to its original vision of getting the fundamental design decisions right, rather than trying to retroactively patch over existing flaws. Otherwise, we risk becoming the next C++, which would be a great pity. T -- "Computer Science is no more about computers than astronomy is about telescopes." -- E.W. Dijkstra
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote: ... I have already made my objections known in the other threads, but I'll list them here for posterity. Firstly, and most importantly IMO, this does not solve the surface level problem, which is the lack of @nogc in much of Phobos. While the DIP is correct that in a lot of the absence of @nogc is due to exceptions, most notably the auto decoding functions in std.range, it not the only reason. There are many things in Phobos which stop @nogc, such as array literals, array appending, AAs, and delegates. While many functions can be made @nogc via this change, there will be many functions left in Phobos still allocating, and this is a major problem in my estimation. The reason I take this all or nothing view of the situation is that @nogc exists largely for a group of people who want to avoid the GC __completely__. If lack of any GC usage is actually a requirement for a project, having many very useful parts of Phobos off limits (a few that spring to mind are bigint, digest, and a lot of stdio) is still a problem when our competitors in this GC free arena are Rust and C++ which will both have a lot more functionality in this limited scope. Secondly, I'm not a fan of special casing syntax, especially when I don't think the given benefits outweigh the above listed costs. Making operator new do something completely different in one specific context is a continuation of the trend in the recent past of making the language more and more complicated, which I find troubling as one of D's main selling points is it's practicality and its low learning curve. And while I don't think the code breakage will be a huge problem, it still is something to consider when asking what are we getting for all of these costs. Thirdly, I don't think the DIP asks the right question. The DIP asks How can we make exceptions, and thereby many Phobos functions, @nogc? IMO the right question to ask is a level of analysis above that. As already mentioned, there are many things which will still be allocated on the GC. We need to fix these as well if we want to market D as a GC optional language. Instead, we should be asking How can allocations in Phobos and user code be transferred to an allocation strategy of the user's choosing/needs? If the user values code simplicity and safety, then the GC is fine, and template functions which allocate can then be marked as @safe due to their use of GC. But if the user needs other speed or usage requirements, then the user needs to offer something else, and that will automatically be marked as @system. I think that integrating std.allocator across Phobos in a consistent manner will allow this, where functions can accept allocators, and their internal allocations can all be done on the given allocator. This would also allow exceptions and other objects can be manually freed after a function is called if needed, or just left to the GC to collect if the GC allocator was passed to the function. In summary, I believe what Phobos needs is a wholistic approach to the problem of memory management, and not a bunch of small solutions over time. To be sure, this is a much harder task, but I think if it's done right, with D we can have the first language which has compete user control to those who need it while offering ease of use to everyone else.
Re: DIP 1008 Preliminary Review Round 1
On Fri, May 19, 2017 at 05:48:55PM +, Stanislav Blinov via Digitalmars-d wrote: > On Friday, 19 May 2017 at 17:05:09 UTC, H. S. Teoh wrote: [...] > > AFAIK, there is no way to clone classes, unless the class writer > > implemented it explicitly. Only arrays support .dup, no other type does > > (unless user code explicitly implements it, or its equivalent). > > > > > > T > > Well, not that it's a complete implementation or anything, but it > definitely could be done with a library: > > /// Make a shallow clone of a class instance > /// (members that are classes are not cloned) [...] That's the problem right there: you cannot guarantee that a shallow copy will have the correct semantics. For example, modifying a class member through the original copy will also modify what is seen through the clone, which may not be the intention. Also, you have to take care that if the class member is ref-counted, the clone function needs to increment the refcount, otherwise you could end up with a dangling pointer. Meaning, in the general case, you need to be careful about calling postblits and such. Furthermore, a generic deep-copying clone function may not be possible in the general case, because sometimes you don't know if a reference member is intended to be a reference to external data, or an owned value that needs to be deep-copied. For example, if a class C encapsulates a container of references to data, then C.clone should not copy the data, but if a class D is a container of references to data that it owns, then the data should be copied. A generic function cannot decide which one it is unless the intent is made clear, e.g. via a standard UDA, or some other such means. T -- It's amazing how careful choice of punctuation can leave you hanging:
Re: DIP 1008 Preliminary Review Round 1
On Friday, 19 May 2017 at 17:05:09 UTC, H. S. Teoh wrote:
On Fri, May 19, 2017 at 05:13:34PM +0100, rikki cattermole via
Digitalmars-d wrote: [...]
"Code that needs to leak the thrown exception object can clone
the object."
Errors:
```D
import std.stdio;
void main() {
auto e = new Exception("foo");
e = e.dup;
writeln(e.toString());
}
```
Let's just say, I am unaware of a way to duplicate classes.
Assuming I'm messing something, a code example would be very
appreciated in the DIP.
AFAIK, there is no way to clone classes, unless the class
writer implemented it explicitly. Only arrays support .dup, no
other type does (unless user code explicitly implements it, or
its equivalent).
T
Well, not that it's a complete implementation or anything, but it
definitely could be done with a library:
/// Make a shallow clone of a class instance
/// (members that are classes are not cloned)
C clone(C)(C src) if (is(C == class))
{
// could probably just dynamic cast and copy
// from most derived...
assert(typeid(src) == typeid(C));
// can use any memory allocator here,
// assuming deallocation is handled correctly
import core.memory : GC;
import core.exception : onOutOfMemoryError;
auto ptr = GC.malloc(__traits(classInstanceSize, C));
ptr || onOutOfMemoryError;
scope (failure) GC.free(ptr);
C dst = cast(C)ptr;
import std.traits : BaseClassesTuple;
import std.meta : AliasSeq;
import core.stdc.string : memcpy;
foreach(T; AliasSeq!(C, BaseClassesTuple!C)) {
T bsrc = src;
T bdst = dst;
foreach(i, ref _; bsrc.tupleof) {
alias M = typeof(_);
memcpy(&bdst.tupleof[i], &bsrc.tupleof[i], M.sizeof);
static if (__traits(hasMember, M, "__postblit"))
bdst.tupleof[i].__postblit;
}
}
return dst;
}
void main() {
import std.stdio;
struct Struct {
int[] data;
this(this) {
data = data.dup;
writeln("Struct is copied");
}
}
class Klass {
int value;
Struct s;
this(int v) {
value = v;
s = Struct([1, 2, 3, 4, 5]);
}
}
auto c1 = new Klass(42);
auto c2 = c1.clone;
assert(c2);
assert(c2 !is c1);
assert(c2.value == 42);
assert(c2.s.data);
assert(c2.s.data !is c1.s.data);
assert(c2.s.data == c1.s.data);
}
Re: DIP 1008 Preliminary Review Round 1
On Fri, May 19, 2017 at 05:13:34PM +0100, rikki cattermole via Digitalmars-d
wrote:
[...]
> "Code that needs to leak the thrown exception object can clone the
> object."
>
> Errors:
> ```D
> import std.stdio;
> void main() {
> auto e = new Exception("foo");
> e = e.dup;
> writeln(e.toString());
> }
> ```
>
> Let's just say, I am unaware of a way to duplicate classes. Assuming
> I'm messing something, a code example would be very appreciated in the
> DIP.
AFAIK, there is no way to clone classes, unless the class writer
implemented it explicitly. Only arrays support .dup, no other type does
(unless user code explicitly implements it, or its equivalent).
T
--
This is not a sentence.
Re: DIP 1008 Preliminary Review Round 1
On 19/05/2017 4:45 PM, Mike Parker wrote:
DIP 1008 is titled "Exceptions and @nogc".
https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md
All review-related feedback on and discussion of the DIP should occur in
this thread. The review period will end at 11:59 PM ET on June 2 (3:59
AM GMT June 3), or when I make a post declaring it complete.
At the end of Round 1, if further review is deemed necessary, the DIP
will be scheduled for another round. Otherwise, it will be queued for
the formal review and evaluation by the language authors.
Extensive discussion of this DIP has already taken place in two threads,
both linked from the document. You may find it beneficial to skim
through those threads before posting any feedback here.
Thanks in advance to all who participate.
Destroy!
"Code that needs to leak the thrown exception object can clone the object."
Errors:
```D
import std.stdio;
void main() {
auto e = new Exception("foo");
e = e.dup;
writeln(e.toString());
}
```
Let's just say, I am unaware of a way to duplicate classes. Assuming I'm
messing something, a code example would be very appreciated in the DIP.
DIP 1008 Preliminary Review Round 1
DIP 1008 is titled "Exceptions and @nogc". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on June 2 (3:59 AM GMT June 3), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Extensive discussion of this DIP has already taken place in two threads, both linked from the document. You may find it beneficial to skim through those threads before posting any feedback here. Thanks in advance to all who participate. Destroy!
