Re: Debugging silent exit of threads in Phobos calls
For anyone still interested in this problem: Steve's pull request fixing the problem is included in D 2.081.0. I can experiment as soon as DMD 2.081.0 is released but will have to wait till the fixes get into LDC to create production code. Moral of the story: silent, except during gdb debugging, assertion fails in Phobos are a pain in the arse. -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls (bugzilla issue submitted)
On Saturday, 2 June 2018 at 09:52:59 UTC, Russel Winder wrote: I get ldc2 from the Debian/Fedora repositories and dmd from d-apt (no Fedora equivalent) I really want to avoid fiddling with files that are installed via packaging. Though I guess any changes can be fixed by a reinstallation of the package. Editing in place would be the easiest and shouldn't be a problem, but at least ldc is relocatable: import folder is specified in config, so you can have it anywhere.
Re: Debugging silent exit of threads in Phobos calls
On Fri, 2018-06-01 at 16:19 -0400, Steven Schveighoffer via
Digitalmars-d-learn wrote:
> On 6/1/18 1:41 PM, Russel Winder wrote:
> > struct Datum {
> > public const int a;
> > public const int b;
> > }
> >
> > struct Message {
> > Datum datum;
> > }
>
> I found the bug. Basically, the Variant static if is failing because
> the
> assignment it is checking (assigning a Message to a Message) would
> overwrite const data. But it's not really overwriting existing data,
> it's emplacing into new data (always). So this check is not correct.
I think then my bug report 18934 is slightly wrong and it isn't the
struct within struct that is the problem, it is the const fields in a
struct that is?
> Much simpler test case:
>
> struct S
> {
> const int x;
> }
>
> void main()
> {
> import std.variant;
> import std.stdio;
> Variant v = S(1);
> writeln(v.get!S); // same failure
> }
>
Aha, much nicer since it is far more localised to the issue. I was
still focused on the receive and the struct within struct, which seems
to be the wrong issue: it is the const fields that are the problem.
Feel free to ignore my example and stack trace on
https://issues.dlang.org/show_bug.cgi?id=18934
and replace it with the above!
> If I replace the check in std.variant:
>
> static if (is(typeof(*cast(T*) target = *src)) ||
>
> with:
>
> static if (is(typeof(delegate T() {return *src;}))
> ||
>
> Then it works (both your code, and the simple example).
>
> Note that in all cases, variant is returning a COPY of the data, not
> a
> reference, so it shouldn't be possible to violate const.
Excellently done. Thanks for attacking this problem and finding a
solution. I will not now even contemplate hacking up my code. Actually
I would have put const in and still got a problem it seems, as I had
the wrong reason for the problem.
> Please, file a bug. I will see if I can submit a PR to fix.
>
Bug report is 18934.
If there can be a fast release of this via 2.080.1 with subsequence
fast releases via d-apt and Debian and Fedora packaging, I would be a
very happy bunny.
In the meantime onward with ACCU 2019 organisation.
ACCU needs more D content. Yes it is 50% C++, but that is exactly why
it needs D content. It also needs Go content. in 2018 we had Rust
content and it went down well.
--
Russel.
===
Dr Russel Winder t: +44 20 7585 2200
41 Buckmaster Roadm: +44 7770 465 077
London SW11 1EN, UK w: www.russel.org.uk
signature.asc
Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls (bugzilla issue submitted)
On Fri, 2018-06-01 at 14:02 -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: > […] > Perfect, put this into a bugzilla entry. Most definitely it's a bug > in > phobos, it's clear from the assert(false, ...) which is NEVER meant > to > happen. Bug report submitted. https://issues.dlang.org/show_bug.cgi?id=18934 Hopefully it is easy to fix and 2.081 can be released quickly as my only way forward is to hack the code to avoid the problem which then essentially destroys the abstractions. OK so it would be a retrievable hack, but one has to draw the line somewhere. Also I have to stop having fun with Me-TV_D and get back on to organising ACCU 2019. It is time D people submitted sessions to ACCU: ACCU 2018 had lots of Rust sessions to counterbalance the C++ stuff and it went down well. > BTW, yes I was saying edit the global phobos sources :) I do this > all > the time to try and debug something. You just have to remember to put > it > back the way it was. In this case, you would just be printing > something > before crashing, so it actually could stay there. > > One "nice" aspect of pretty much everything being a template. I get ldc2 from the Debian/Fedora repositories and dmd from d-apt (no Fedora equivalent) I really want to avoid fiddling with files that are installed via packaging. Though I guess any changes can be fixed by a reinstallation of the package. -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls
On 6/1/18 1:41 PM, Russel Winder wrote:
struct Datum {
public const int a;
public const int b;
}
struct Message {
Datum datum;
}
I found the bug. Basically, the Variant static if is failing because the
assignment it is checking (assigning a Message to a Message) would
overwrite const data. But it's not really overwriting existing data,
it's emplacing into new data (always). So this check is not correct.
Much simpler test case:
struct S
{
const int x;
}
void main()
{
import std.variant;
import std.stdio;
Variant v = S(1);
writeln(v.get!S); // same failure
}
If I replace the check in std.variant:
static if (is(typeof(*cast(T*) target = *src)) ||
with:
static if (is(typeof(delegate T() {return *src;})) ||
Then it works (both your code, and the simple example).
Note that in all cases, variant is returning a COPY of the data, not a
reference, so it shouldn't be possible to violate const.
Please, file a bug. I will see if I can submit a PR to fix.
-Steve
Re: Debugging silent exit of threads in Phobos calls
On 6/1/18 1:41 PM, Russel Winder wrote:
On Fri, 2018-06-01 at 18:30 +0100, Russel Winder wrote:
[…]
I'll trim this sample code down to the minimum so it can be used in
the
test suite of Phobos creating a red.
Here it is, a small bit of code that breaks Phobos'
std.concurrency.receive.
import core.thread: Thread;
import core.time: seconds;
import std.concurrency: Tid, OwnerTerminated, receive, receiveTimeout, send,
spawn;
import std.conv: to;
import std.stdio: writeln;
struct Datum {
public const int a;
public const int b;
}
struct Message {
Datum datum;
}
void sender(Tid receiver) {
writeln("Sender sending.");
receiver.send(Message(Datum(0, 0)));
writeln("Sender finished.");
}
void receiver() {
writeln("Receiver going into receive.");
try {
receive(
(Message message) {
writeln("Receiver received ",
to!string(message));
},
);
} catch (Throwable t) {
writeln("Receiver receive threw " ~ to!string(t));
}
writeln("Receiver finished.");
}
void mainloop() {
Thread.sleep(2.seconds);
}
int main() {
auto receiver = spawn();
spawn(, receiver);
mainloop();
return 0;
}
Perfect, put this into a bugzilla entry. Most definitely it's a bug in
phobos, it's clear from the assert(false, ...) which is NEVER meant to
happen.
BTW, yes I was saying edit the global phobos sources :) I do this all
the time to try and debug something. You just have to remember to put it
back the way it was. In this case, you would just be printing something
before crashing, so it actually could stay there.
One "nice" aspect of pretty much everything being a template.
-Steve
Re: Debugging silent exit of threads in Phobos calls
On Fri, 2018-06-01 at 18:30 +0100, Russel Winder wrote:
> […]
>
> I'll trim this sample code down to the minimum so it can be used in
> the
> test suite of Phobos creating a red.
>
Here it is, a small bit of code that breaks Phobos'
std.concurrency.receive.
import core.thread: Thread;
import core.time: seconds;
import std.concurrency: Tid, OwnerTerminated, receive, receiveTimeout, send,
spawn;
import std.conv: to;
import std.stdio: writeln;
struct Datum {
public const int a;
public const int b;
}
struct Message {
Datum datum;
}
void sender(Tid receiver) {
writeln("Sender sending.");
receiver.send(Message(Datum(0, 0)));
writeln("Sender finished.");
}
void receiver() {
writeln("Receiver going into receive.");
try {
receive(
(Message message) {
writeln("Receiver received ",
to!string(message));
},
);
} catch (Throwable t) {
writeln("Receiver receive threw " ~ to!string(t));
}
writeln("Receiver finished.");
}
void mainloop() {
Thread.sleep(2.seconds);
}
int main() {
auto receiver = spawn();
spawn(, receiver);
mainloop();
return 0;
}
--
Russel.
===
Dr Russel Winder t: +44 20 7585 2200
41 Buckmaster Roadm: +44 7770 465 077
London SW11 1EN, UK w: www.russel.org.uk
signature.asc
Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls
On Fri, 2018-06-01 at 09:56 -0700, Ali Çehreli via Digitalmars-d-learn
wrote:
> On 06/01/2018 09:40 AM, Russel Winder wrote:
>
> > The assert is in Phobos, so I am not sure I can.
>
> Not the cleanest solution but one can always "carefully" :) edit the
> installed Phobos files. Mine are under /usr/include/dmd/phobos/std/
> It
> will most likely work as they are almost always templates, meaning,
> compiled with your code.
>
> Ali
I may have avoided needing to do this. If I change my stripped down
trial code from using:
struct Message {
int counter
}
to using the equivalent of what happens in Me TV:
struct Datum {
public const int a;
public const int b;
}
struct Message {
Datum datum;
}
Then my stripped down trial code fails in a not dissimilar way. I think
therefore this is a bug in Phobos.
I'll trim this sample code down to the minimum so it can be used in the
test suite of Phobos creating a red.
--
Russel.
===
Dr Russel Winder t: +44 20 7585 2200
41 Buckmaster Roadm: +44 7770 465 077
London SW11 1EN, UK w: www.russel.org.uk
signature.asc
Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls
On 06/01/2018 09:40 AM, Russel Winder wrote: > The assert is in Phobos, so I am not sure I can. Not the cleanest solution but one can always "carefully" :) edit the installed Phobos files. Mine are under /usr/include/dmd/phobos/std/ It will most likely work as they are almost always templates, meaning, compiled with your code. Ali
Re: Debugging silent exit of threads in Phobos calls
On Fri, 2018-06-01 at 11:20 -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: > […] > That assertion is something that should never happen. Something is > wrong > with the type checking here. > > If you look up earlier, clearly the type id matches. There is > something > weird about that static if that is not working. > > Can you put in some more debug messages and see what the exact types > of > A and T are? Just put it right before the assert. > The assert is in Phobos, so I am not sure I can. (Sorry to sound like a beginner at this, it is probably because I feel like one just at the moment!) -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls
On Fri, 2018-06-01 at 18:21 +0300, ketmar via Digitalmars-d-learn wrote: > […] > it may be something with struct copying. variant wants to copy a > struct > into itself, and somehow failed. either there is no room, or > something > prevented it to do that. it is hard to say more without full source > code. Full source is at: https://github.com/russel/Me-TV_D -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls
Steven Schveighoffer wrote: Can you put in some more debug messages and see what the exact types of A and T are? Just put it right before the assert. you prolly asked Russel here, as i don't have his sources to experiment with. ;-)
Re: Debugging silent exit of threads in Phobos calls
Russel Winder wrote: On Fri, 2018-06-01 at 17:53 +0300, ketmar via Digitalmars-d-learn wrote: […] it looks like "// type T is not constructible from A" phobos assertion triggered. that is, std.variant cannot wrap the struct, and all hell breaks loose. An instance of FrontendAppeared is created in frontend_manager module and sent to the actor defined in control_window module which has: import frontend_manager: FrontendAppeared It seems the symbol FrontendAppeared in control_window is matching the FrontendAppeared type of the message sent, and yet the assertion fails. it may be something with struct copying. variant wants to copy a struct into itself, and somehow failed. either there is no room, or something prevented it to do that. it is hard to say more without full source code.
Re: Debugging silent exit of threads in Phobos calls
On 6/1/18 10:53 AM, ketmar wrote: Russel Winder wrote: On Fri, 2018-06-01 at 16:37 +0300, ketmar via Digitalmars-d-learn wrote: […] yeah. if it receives something it doesn't expect (and there is no `Variant` clause to catch it), it throws. and exceptions from threads are silently dropped on the floor -- along with the dead threads. so it is better to always wrap your threads in `try/catch`, and at least log an exception. It seems that in the case he presence of the Variant doesn't matter. The message sent is of type frontend_manager.FrontendAppeared and one of the receive types is frontend_manager.FrontendAppreared yet this fails to match but it is not treated as a Variant. The stack trace is: it looks like "// type T is not constructible from A" phobos assertion triggered. that is, std.variant cannot wrap the struct, and all hell breaks loose. That assertion is something that should never happen. Something is wrong with the type checking here. If you look up earlier, clearly the type id matches. There is something weird about that static if that is not working. Can you put in some more debug messages and see what the exact types of A and T are? Just put it right before the assert. -Steve
Re: Debugging silent exit of threads in Phobos calls
On Fri, 2018-06-01 at 17:53 +0300, ketmar via Digitalmars-d-learn wrote: > […] > it looks like "// type T is not constructible from A" phobos > assertion > triggered. that is, std.variant cannot wrap the struct, and all hell > breaks > loose. An instance of FrontendAppeared is created in frontend_manager module and sent to the actor defined in control_window module which has: import frontend_manager: FrontendAppeared It seems the symbol FrontendAppeared in control_window is matching the FrontendAppeared type of the message sent, and yet the assertion fails. -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls
On Fri, 2018-06-01 at 07:59 -0700, Ali Çehreli via Digitalmars-d-learn wrote: > On 06/01/2018 04:12 AM, Russel Winder wrote: > > > Obviously std.concurrency.receive should never terminate a thread, > > and > > it should never terminate a thread silently, but given that it > > clearly > > does (using dmd 2.080 from d-apt on Debian Sid) how is one to find > > out > > useful information as to why it is exiting silently. > > It sounds like one of the message handlers is throwing. In that case, > it > sounds like a good idea to wrap each handler with try-catch. The try/catch allows one to see that there appears to be a problem inside Phobos, i.e. it determines that x.X is not the same type as x.X and throws an assertion error. To handle this in the application is to accept that Phobos is broken and not fixable. :-( Unless this is actually just that x.X is not x.X even though it is x.X. In this case it would be a module naming error in D and Phobos is not wrong. > […] -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls
On 06/01/2018 04:12 AM, Russel Winder wrote: Obviously std.concurrency.receive should never terminate a thread, and it should never terminate a thread silently, but given that it clearly does (using dmd 2.080 from d-apt on Debian Sid) how is one to find out useful information as to why it is exiting silently. It sounds like one of the message handlers is throwing. In that case, it sounds like a good idea to wrap each handler with try-catch. Ali P.S. In case it's useful for someone, and I know it's not for the same case, I include something about exceptions in a worker thread, which shows that it's possible to receive a shared(Exception): http://ddili.org/ders/d.en/concurrency.html#ix_concurrency.exception,%20concurrency However, it fails to suggest catching Throwable or Error at least for debugging.
Re: Debugging silent exit of threads in Phobos calls
Russel Winder wrote: On Fri, 2018-06-01 at 16:37 +0300, ketmar via Digitalmars-d-learn wrote: […] yeah. if it receives something it doesn't expect (and there is no `Variant` clause to catch it), it throws. and exceptions from threads are silently dropped on the floor -- along with the dead threads. so it is better to always wrap your threads in `try/catch`, and at least log an exception. It seems that in the case he presence of the Variant doesn't matter. The message sent is of type frontend_manager.FrontendAppeared and one of the receive types is frontend_manager.FrontendAppreared yet this fails to match but it is not treated as a Variant. The stack trace is: it looks like "// type T is not constructible from A" phobos assertion triggered. that is, std.variant cannot wrap the struct, and all hell breaks loose.
Re: Debugging silent exit of threads in Phobos calls
On Fri, 2018-06-01 at 16:37 +0300, ketmar via Digitalmars-d-learn wrote: > […] > yeah. if it receives something it doesn't expect (and there is no > `Variant` > clause to catch it), it throws. and exceptions from threads are > silently > dropped on the floor -- along with the dead threads. so it is better > to > always wrap your threads in `try/catch`, and at least log an > exception. It seems that in the case he presence of the Variant doesn't matter. The message sent is of type frontend_manager.FrontendAppeared and one of the receive types is frontend_manager.FrontendAppreared yet this fails to match but it is not treated as a Variant. The stack trace is: ControlWindowDaemon: receive threw an exception core.exception.AssertError@/usr/include/dmd/phobos/std/variant.d(323): FrontendAppeared ??:? _d_assert_msg [0x55e72826] ??:? bool std.variant.VariantN!(32uL).VariantN.handler!(frontend_manager.FrontendAppeared).handler(std.variant.VariantN!(32uL).VariantN.OpID, ubyte[32]*, void*).tryPutting(frontend_manager.FrontendAppeared*, TypeInfo, void*) [0x55ba1dc1] ??:? long std.variant.VariantN!(32uL).VariantN.handler!(frontend_manager.FrontendAppeared).handler(std.variant.VariantN!(32uL).VariantN.OpID, ubyte[32]*, void*) [0x55ba1949] ??:? inout @property inout(frontend_manager.FrontendAppeared) std.variant.VariantN!(32uL).VariantN.get!(frontend_manager.FrontendAppeared).get() [0x55b9fadf] ??:? void std.concurrency.Message.map!(void function(frontend_manager.FrontendAppeared) @safe*).map(void function(frontend_manager.FrontendAppeared) @safe*) [0x55b9fa9c] ??:? bool std.concurrency.MessageBox.get!(void function(frontend_manager.FrontendAppeared) @safe*, void function(frontend_manager.FrontendDisappeared) @safe*, void delegate(std.concurrency.OwnerTerminated) @safe).get(scope void function(frontend_manager.FrontendAppeared) @safe*, scope void function(frontend_manager.FrontendDisappeared) @safe*, scope void delegate(std.concurrency.OwnerTerminated) @safe).onStandardMsg(ref std.concurrency.Message) [0x55b9f3b4] ??:? bool std.concurrency.MessageBox.get!(void function(frontend_manager.FrontendAppeared) @safe*, void function(frontend_manager.FrontendDisappeared) @safe*, void delegate(std.concurrency.OwnerTerminated) @safe).get(scope void function(frontend_manager.FrontendAppeared) @safe*, scope void function(frontend_manager.FrontendDisappeared) @safe*, scope void delegate(std.concurrency.OwnerTerminated) @safe).scan(ref std.concurrency.List!(std.concurrency.Message).List) [0x55b9f834] ??:? bool std.concurrency.MessageBox.get!(void function(frontend_manager.FrontendAppeared) @safe*, void function(frontend_manager.FrontendDisappeared) @safe*, void delegate(std.concurrency.OwnerTerminated) @safe).get(scope void function(frontend_manager.FrontendAppeared) @safe*, scope void function(frontend_manager.FrontendDisappeared) @safe*, scope void delegate(std.concurrency.OwnerTerminated) @safe) [0x55b9f2c8] ??:? void std.concurrency.receive!(void function(frontend_manager.FrontendAppeared) @safe*, void function(frontend_manager.FrontendDisappeared) @safe*, void delegate(std.concurrency.OwnerTerminated) @safe).receive(void function(frontend_manager.FrontendAppeared) @safe*, void function(frontend_manager.FrontendDisappeared) @safe*, void delegate(std.concurrency.OwnerTerminated) @safe) [0x55b9f0ec] ??:? void control_window.runControlWindowDaemon() [0x55b9edb3] ??:? void std.concurrency._spawn!(void function()*)._spawn(bool, void function()*).exec() [0x55ba3f4f] ??:? void core.thread.Thread.run() [0x55e739a7] ??:? thread_entryPoint [0x55ea5267] ??:? [0xf79bd5a9] -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls
On Fri, 2018-06-01 at 09:31 -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: > On 6/1/18 7:12 AM, Russel Winder wrote: > > So I had a play and gdb seems to be useless for trying to find out > > why > > calls to std.concurrency.receive exit silently. > > > > Obviously std.concurrency.receive should never terminate a thread, > > and > > it should never terminate a thread silently, but given that it > > clearly > > does (using dmd 2.080 from d-apt on Debian Sid) how is one to find > > out > > useful information as to why it is exiting silently. > > I remember something like a receive thread that throws terminates > silently. Do a try/catch to see if it's that. Tried to catch Exception but that failed to work, fortunately remembered to catch Throwable and it turned out to be a receive threw an exception core.exception.AssertError@/usr/include/dmd/phobos/std/variant.d(323): It seems that the type frontend_manager.FrontendAppeared is not the same as frontend_manager.FrontendAppeared. :-( Who knew. -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Debugging silent exit of threads in Phobos calls
Steven Schveighoffer wrote: On 6/1/18 7:12 AM, Russel Winder wrote: So I had a play and gdb seems to be useless for trying to find out why calls to std.concurrency.receive exit silently. Obviously std.concurrency.receive should never terminate a thread, and it should never terminate a thread silently, but given that it clearly does (using dmd 2.080 from d-apt on Debian Sid) how is one to find out useful information as to why it is exiting silently. I remember something like a receive thread that throws terminates silently. Do a try/catch to see if it's that. -Steve yeah. if it receives something it doesn't expect (and there is no `Variant` clause to catch it), it throws. and exceptions from threads are silently dropped on the floor -- along with the dead threads. so it is better to always wrap your threads in `try/catch`, and at least log an exception.
Re: Debugging silent exit of threads in Phobos calls
On 6/1/18 7:12 AM, Russel Winder wrote: So I had a play and gdb seems to be useless for trying to find out why calls to std.concurrency.receive exit silently. Obviously std.concurrency.receive should never terminate a thread, and it should never terminate a thread silently, but given that it clearly does (using dmd 2.080 from d-apt on Debian Sid) how is one to find out useful information as to why it is exiting silently. I remember something like a receive thread that throws terminates silently. Do a try/catch to see if it's that. -Steve
Debugging silent exit of threads in Phobos calls
So I had a play and gdb seems to be useless for trying to find out why calls to std.concurrency.receive exit silently. Obviously std.concurrency.receive should never terminate a thread, and it should never terminate a thread silently, but given that it clearly does (using dmd 2.080 from d-apt on Debian Sid) how is one to find out useful information as to why it is exiting silently. -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
