Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 03:42:29 UTC, safety0ff wrote: On Friday, 24 October 2014 at 02:51:20 UTC, tcak wrote: I don't want to blame dmd directly because as far as I see from the search I did with __lll_lock_wait_private, some C++ programs are having same problem with malloc operation as well. But still, can this be because of compiler? Looks like bug #11981 [1], which should be fixed in the latest versions of the compiler. Which version are you using? [1] https://issues.dlang.org/show_bug.cgi?id=11981 I am on DMD 2.066 64-bit Linux 3.13.0-37-generic
Re: new(malloc) locks everything in multithreading
If it's deterministic, looks more like https://issues.dlang.org/show_bug.cgi?id=4890 (11981 is not deterministic)
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 08:47:55 UTC, Kagamin wrote: If it's deterministic, looks more like https://issues.dlang.org/show_bug.cgi?id=4890 (11981 is not deterministic) Yes, it is deterministic. Run it as many times as you want, and it does the same thing. I ran it now again, and still same.
Re: new(malloc) locks everything in multithreading
Do you see recursive call to malloc in the stack trace?
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 08:55:17 UTC, Kagamin wrote:
Do you see recursive call to malloc in the stack trace?
I further simplified the example:
import std.stdio;
import core.thread;
class ThreadTest{
public this(){
new core.thread.Thread( threadRun ).start();
}
private void threadRun(){
writeln(It works);
readln();
}
}
void main(){
new ThreadTest();
char[] abc = new char[4096];
}
This is what I see on screen:
http://imgur.com/Pv9Rulw
Same result.
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 09:12:57 UTC, tcak wrote:
On Friday, 24 October 2014 at 08:55:17 UTC, Kagamin wrote:
Do you see recursive call to malloc in the stack trace?
I further simplified the example:
import std.stdio;
import core.thread;
class ThreadTest{
public this(){
new core.thread.Thread( threadRun ).start();
}
private void threadRun(){
writeln(It works);
readln();
}
}
void main(){
new ThreadTest();
char[] abc = new char[4096];
}
This is what I see on screen:
http://imgur.com/Pv9Rulw
Same result.
And this is the thread of malloc.
http://imgur.com/e8ofRte
It suspends all threads and cannot get out of there it seems like.
Re: new(malloc) locks everything in multithreading
Looks like your IDE filters too much. Can you configure it to filter less and show address locations?
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 10:46:57 UTC, tcak wrote: On Friday, 24 October 2014 at 10:29:10 UTC, Kagamin wrote: Looks like your IDE filters too much. Can you configure it to filter less and show address locations? This is what I have found: Main Thread http://i.imgur.com/6ElZ3Fm.png Second Thread (TestThread) http://i.imgur.com/w4y5gYB.png BTW, instead of using Monodevelop with Mono-D, I used same code on a text file, and compiled with dmd test.d, then run with ./test, then everything works fine. When I run it with gdb ./test, then I can see those errors again.
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 10:29:10 UTC, Kagamin wrote: Looks like your IDE filters too much. Can you configure it to filter less and show address locations? This is what I have found: Main Thread http://i.imgur.com/6ElZ3Fm.png Second Thread (TestThread) http://i.imgur.com/w4y5gYB.png
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 10:46:57 UTC, tcak wrote: Second Thread (TestThread) http://i.imgur.com/w4y5gYB.png Hmm... where is __lll_lock_wait_private now? And how mmap can hang at all?
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 10:49:42 UTC, tcak wrote: On Friday, 24 October 2014 at 10:46:57 UTC, tcak wrote: On Friday, 24 October 2014 at 10:29:10 UTC, Kagamin wrote: Looks like your IDE filters too much. Can you configure it to filter less and show address locations? This is what I have found: Main Thread http://i.imgur.com/6ElZ3Fm.png Second Thread (TestThread) http://i.imgur.com/w4y5gYB.png BTW, instead of using Monodevelop with Mono-D, I used same code on a text file, and compiled with dmd test.d, then run with ./test, then everything works fine. When I run it with gdb ./test, then I can see those errors again. Not sure if this is the same issue, but by default gdb breaks on signals that the GC uses, which would explain why it's breaking in gdb but not normally. What happens if you try: handle SIGUSR1 noprint nostop handle SIGUSR2 noprint nostop In GDB before starting execution of the program?
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 12:38:48 UTC, Kagamin wrote: On Friday, 24 October 2014 at 10:46:57 UTC, tcak wrote: Second Thread (TestThread) http://i.imgur.com/w4y5gYB.png Hmm... where is __lll_lock_wait_private now? And how mmap can hang at all? Here it is. http://i.imgur.com/5ZDuYRF.png I didn't notice mmap before you said it, but it seems like that happened as well.
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 16:51:02 UTC, Kapps wrote:
On Friday, 24 October 2014 at 10:49:42 UTC, tcak wrote:
Not sure if this is the same issue, but by default gdb breaks
on signals that the GC uses, which would explain why it's
breaking in gdb but not normally.
What happens if you try:
handle SIGUSR1 noprint nostop
handle SIGUSR2 noprint nostop
In GDB before starting execution of the program?
This is what I did on shell: (I put some spaces for readability)
tolga@tolga:~/dev/d/bug$ dmd -gc -debug test.d
tolga@tolga:~/dev/d/bug$ gdb ./test
GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type show
copying
and show warranty for details.
This GDB was configured as x86_64-linux-gnu.
Type show configuration for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type help.
Type apropos word to search for commands related to word...
Reading symbols from ./test...done.
(gdb) run
Starting program: /home/tolga/dev/d/bug/test
[Thread debugging using libthread_db enabled]
Using host libthread_db library
/lib/x86_64-linux-gnu/libthread_db.so.1.
[New Thread 0x775ed700 (LWP 4940)]
Program received signal SIGUSR1, User defined signal 1.
[Switching to Thread 0x775ed700 (LWP 4940)]
__lll_lock_wait_private () at
../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
95 ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: No such
file or directory.
(gdb) backtrace full
#0 __lll_lock_wait_private () at
../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
No locals.
#1 0x77bc52f1 in _L_lock_3358 () from
/lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#2 0x77bc4382 in start_thread (arg=0x775ed700) at
pthread_create.c:301
oldtype = 0
pd = 0x775ed700
now = optimised out
unwind_buf = {cancel_jmp_buf = {{jmp_buf =
{140737343575808, 9112392114346628883, 1, 0,
140737343576512, 140737343575808,
-9112375355952097517, -9112375093712283885},
mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0,
0x0}, data = {prev = 0x0,
cleanup = 0x0, canceltype = 0}}}
not_first_call = optimised out
pagesize_m1 = optimised out
sp = optimised out
freesize = optimised out
__PRETTY_FUNCTION__ = start_thread
#3 0x776e8fbd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.
(gdb)
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 18:38:39 UTC, tcak wrote: On Friday, 24 October 2014 at 16:51:02 UTC, Kapps wrote: On Friday, 24 October 2014 at 10:49:42 UTC, tcak wrote: Not sure if this is the same issue, but by default gdb breaks on signals that the GC uses, which would explain why it's breaking in gdb but not normally. What happens if you try: handle SIGUSR1 noprint nostop handle SIGUSR2 noprint nostop In GDB before starting execution of the program? This is what I did on shell: (I put some spaces for readability) tolga@tolga:~/dev/d/bug$ dmd -gc -debug test.d tolga@tolga:~/dev/d/bug$ gdb ./test Yes, GDB is stopping on SIGUSR1 / SIGUSR2 since that's the default settings. D's GC uses these signals for suspending / resuming threads during a collection. You need to type what I said above, prior to typing 'run'.
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 21:02:05 UTC, Kapps wrote: Yes, GDB is stopping on SIGUSR1 / SIGUSR2 since that's the default settings. D's GC uses these signals for suspending / resuming threads during a collection. You need to type what I said above, prior to typing 'run'. I took a look at the Boehm GC earlier today and it appears they've made the signal set configurable, both to try and not use SIGUSR1/2 by default and to let the user specify another signal set if they need SIGUSR1/2 for some reason. It's probably worth doing this in our own code as well. The Boehm GC also does some magic with clearing signals at various points to make sure the right signal handlers will be called. Probably another enhancement request.
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 21:02:05 UTC, Kapps wrote: On Friday, 24 October 2014 at 18:38:39 UTC, tcak wrote: On Friday, 24 October 2014 at 16:51:02 UTC, Kapps wrote: This is what I did on shell: (I put some spaces for readability) tolga@tolga:~/dev/d/bug$ dmd -gc -debug test.d tolga@tolga:~/dev/d/bug$ gdb ./test Yes, GDB is stopping on SIGUSR1 / SIGUSR2 since that's the default settings. D's GC uses these signals for suspending / resuming threads during a collection. You need to type what I said above, prior to typing 'run'. I was desperately looking for a solution how to do this on Mono-D instead of entering on shell. Then I have found this link: http://www.mono-project.com/docs/debug+profile/debug/ Under the title Debugging with GDB, it says to create .gdbinit file in home folder and put those handle SIGUSR1... things into it. Then I opened the preferences of MonoDevelop, and brought the GDB with D language support as first in preferred debugger list. Tada! It works now. I really thank you. BUT, since Mono-D comes with GDB with D language support as well, this process should have been automated in my opinion knowing that GC would be using those signals.
new(malloc) locks everything in multithreading
This must be my special day that everything I try gets broken.
import core.thread;
import std.stdio;
class ThreadTest{
private core.thread.Thread th;
public this() shared{
th = cast( shared )(
new core.thread.Thread(
cast( void delegate() )( threadRun )
)
);
}
public void start() shared{
(cast()th).start();
}
private void threadRun() shared{
writeln(It works);
readln();
}
}
void main(){
new shared ThreadTest().start();
char[] abc = new char[4096];
}
=
Run the program (Release or Debug doesn't matter).
- Mono Application Output -
[Thread debugging using libthread_db enabled]
Using host libthread_db library
\/lib/x86_64-linux-gnu/libthread_db.so.1\.
[New Thread 0x775ed700 (LWP 18480)]
[Switching to Thread 0x775ed700 (LWP 18480)]
In call stack, program comes to __lll_lock_wait_private () in
/build/buildd/eglibc-2.19/nptl/../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95,
and
this locks everything.
Remove the char[] abc = ne line, and everything is fine.
Remove the new shared Thre... line and everything is fine.
I don't want to blame dmd directly because as far as I see from
the search I did with __lll_lock_wait_private, some C++
programs are having same problem with malloc operation as well.
But still, can this be because of compiler?
Re: new(malloc) locks everything in multithreading
On Friday, 24 October 2014 at 02:51:20 UTC, tcak wrote: I don't want to blame dmd directly because as far as I see from the search I did with __lll_lock_wait_private, some C++ programs are having same problem with malloc operation as well. But still, can this be because of compiler? Looks like bug #11981 [1], which should be fixed in the latest versions of the compiler. Which version are you using? [1] https://issues.dlang.org/show_bug.cgi?id=11981
