[maemo-developers] Problem with GDB and threads...
Hi. I have a problem when trying to debug a multi-threaded application on the Nokia 770 (OS 2006). This is my test program: #include #include #include void* routine( void* arg ) { while( 1 ) { } return 0; } int main( int argc, char** argv ) { pthread_t handle; int err = pthread_create( &handle, 0, routine, 0 ); assert( err == 0 ); sleep( 60 ); return 0; } On the PC, I do the following in gdb: (gdb) inf thr 2 Thread -1479910480 (LWP 29458) routine (arg=0x0) at test.cpp:8 * 1 Thread -1479907648 (LWP 29455) 0xa7d3a2bc in nanosleep () from /lib/tls/libc.so.6 (gdb) thread apply all bt Thread 2 (Thread -1479910480 (LWP 29458)): #0 routine (arg=0x0) at test.cpp:8 #1 0xa7ef7ced in start_thread () from /lib/tls/libpthread.so.0 #2 0xa7d79dee in clone () from /lib/tls/libc.so.6 Thread 1 (Thread -1479907648 (LWP 29455)): #0 0xa7d3a2bc in nanosleep () from /lib/tls/libc.so.6 #1 0xa7d3a0c4 in sleep () from /lib/tls/libc.so.6 #2 0x08048562 in main (argc=1, argv=0xafa2c274) at test.cpp:22 On the Nokia 770, the backtrace goes on forever and i have to kill gdb: (gdb) inf thr 3 Thread 16386 (LWP 2269) 0x84b0 in routine () 2 Thread 32769 (LWP 2268) 0x402a3474 in poll () from /lib/libc.so.6 * 1 Thread 16384 (LWP 2265) 0x4027e37c in nanosleep () from /lib/libc.so.6 (gdb) thread apply all bt Thread 3 (Thread 16386 (LWP 2269)): #0 0x84b0 in routine () #1 0x40024a20 in pthread_start_thread () from /lib/libpthread.so.0 #2 0x40024a20 in pthread_start_thread () from /lib/libpthread.so.0 #3 0x40024a20 in pthread_start_thread () from /lib/libpthread.so.0 #4 0x40024a20 in pthread_start_thread () from /lib/libpthread.so.0 #5 0x40024a20 in pthread_start_thread () from /lib/libpthread.so.0 #6 0x40024a20 in pthread_start_thread () from /lib/libpthread.so.0 [...] Has anyone else experienced this? Also, why is there an extra thread? thanks, Christian Henz ___ maemo-developers mailing list maemo-developers@maemo.org https://maemo.org/mailman/listinfo/maemo-developers
Re: [maemo-developers] Problem with GDB and threads...
Hi, > On the Nokia 770, the backtrace goes on forever and i have to kill gdb Have you compiled your binary: - without optimization (or used -fno-omit-frame-pointer) - with -g - not stripped it ? If not, if you do that and try again, does the debugging work better? If it still doesn't work, does it work if you install 'libc6-dbg' package and run your program, or GDB starting your program, with: LD_LIBRARY_PATH=/usr/lib/debug:$LD_LIBRARY_PATH gdb /path/app ? > Has anyone else experienced this? Also, why is there an extra thread? Thread manager polling the other threads? (Using threads on ARM should be avoided unless absolutely necessary) - Eero ___ maemo-developers mailing list maemo-developers@maemo.org https://maemo.org/mailman/listinfo/maemo-developers
Re: [maemo-developers] Problem with GDB and threads...
On Friday 14 July 2006 08:37, Eero Tamminen wrote: > Hi, > > > On the Nokia 770, the backtrace goes on forever and i have to kill gdb > > Have you compiled your binary: > - without optimization (or used -no-omit-frame-pointer) > - with -g > - not stripped it > ? > > If not, if you do that and try again, does the debugging work better? > I simply compiled with "-g" before, adding the "-no-omit-frame-pointer" directive did not help. > If it still doesn't work, does it work if you install 'libc6-dbg' > package and run your program, or GDB starting your program, with: > LD_LIBRARY_PATH=/usr/lib/debug:$LD_LIBRARY_PATH gdb /path/app > ? > I can't seem to find that package in the mistral repositories. cheers, Christian Henz ___ maemo-developers mailing list maemo-developers@maemo.org https://maemo.org/mailman/listinfo/maemo-developers
Re: [maemo-developers] Problem with GDB and threads...
Hi, > I simply compiled with "-g" before, adding the "-no-omit-frame-pointer" > directive did not help. Actually I think the library where the function giving infinite backtrace is should be compiled with -g (or debug package for it should be installed). If that doesn't help then you need to compile that library / binary with -fno-omit-frame-pointer. At least one reason for the infinite backtraces has been __attribute__ ((__noreturn__)) specified to functions. That is given to functions that do not return and for them -fno-omit-frame-pointer is needed to get working backtrace, for them -g is not enough. Normally those are exit functions, but maybe thread creation function doesn't return either? (I've seen some thread usage example code where __noreturn__ attribute is used for them) - Eero ___ maemo-developers mailing list maemo-developers@maemo.org https://maemo.org/mailman/listinfo/maemo-developers
Re: [maemo-developers] Problem with GDB and threads...
On Tuesday 18 July 2006 15:23, Eero Tamminen wrote: > Hi, > > > I simply compiled with "-g" before, adding the "-no-omit-frame-pointer" > > directive did not help. > > Actually I think the library where the function giving infinite > backtrace is should be compiled with -g (or debug package for > it should be installed). If that doesn't help then you need > to compile that library / binary with -fno-omit-frame-pointer. > Since I could not find a binary libc6-dbg package, I compiled it myself from the package source. But it still does not work: # gdb ./testc-armel GNU gdb 6.4 Copyright 2005 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "arm-linux-gnueabi"...Using host libthread_db library "/usr/lib/debug/libthread_db.so.1". (gdb) r Starting program: /mnt/nessie/nokia770/testc-armel [Thread debugging using libthread_db enabled] [New Thread 16384 (LWP 1146)] BFD: /usr/lib/debug/lib/ld-2.3.6.so: warning: sh_link not set for section `.ARM.exidx' [New Thread 32769 (LWP 1149)] [New Thread 16386 (LWP 1150)] Program received signal SIGINT, Interrupt. [Switching to Thread 16386 (LWP 1150)] 0x400f937c in nanosleep () at stdlib.h:338 338 stdlib.h: No such file or directory. in stdlib.h (gdb) thread apply all bt Thread 3 (Thread 16386 (LWP 1150)): #0 0x400f937c in nanosleep () at stdlib.h:338 #1 0x400f9164 in __sleep (seconds=0) at ../sysdeps/unix/sysv/linux/sleep.c:138 #2 0x8488 in routine (arg=0x0) at test.c:11 #3 0x40024a20 in pthread_start_thread (arg=0xfdfc) at manager.c:310 #4 0x40024a20 in pthread_start_thread (arg=0xfdfc) at manager.c:310 #5 0x40024a20 in pthread_start_thread (arg=0xfdfc) at manager.c:310 #6 0x40024a20 in pthread_start_thread (arg=0xfdfc) at manager.c:310 #7 0x40024a20 in pthread_start_thread (arg=0xfdfc) at manager.c:310 #8 0x40024a20 in pthread_start_thread (arg=0xfdfc) at manager.c:310 #9 0x40024a20 in pthread_start_thread (arg=0xfdfc) at manager.c:310 #10 0x40024a20 in pthread_start_thread (arg=0xfdfc) at manager.c:310 #11 0x40024a20 in pthread_start_thread (arg=0xfdfc) at manager.c:310 [...] > At least one reason for the infinite backtraces has been __attribute__ > ((__noreturn__)) specified to functions. That is given to functions > that do not return and for them -fno-omit-frame-pointer is needed to > get working backtrace, for them -g is not enough. Normally those are > exit functions, but maybe thread creation function doesn't return > either? (I've seen some thread usage example code where __noreturn__ > attribute is used for them) > The compilation of glibc did not explicitly use "-fomit-frame-pointer", but the GCC manual states that "-O" implies it on architectures where it doesn't break debugging. Unfortunately it doesn't mention which architectures those are. I'm currently recompiling glibc with explicitly setting "-fno-omit-frame-pointer". Thanks so far :-) Christian ___ maemo-developers mailing list maemo-developers@maemo.org https://maemo.org/mailman/listinfo/maemo-developers
Re: [maemo-developers] Problem with GDB and threads...
On Tuesday 18 July 2006 19:35, Christian Henz wrote: > > The compilation of glibc did not explicitly use "-fomit-frame-pointer", but > the GCC manual states that "-O" implies it on architectures where it > doesn't break debugging. Unfortunately it doesn't mention which > architectures those are. I'm currently recompiling glibc with explicitly > setting > "-fno-omit-frame-pointer". > This finally did the trick. I configured and compiled glibc using ./configure [...] CFLAGS="-g -O2 -fno-omit-frame-pointer" Now the backtrace doesn't repeat forever, although I still get "Previous frame identical to this frame (corrupt stack?)". # gdb ./testc-armel GNU gdb 6.4 Copyright 2005 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "arm-linux-gnueabi"...Using host libthread_db library "/usr/lib/debug/libthread_db.so.1". (gdb) break test.c:26 Breakpoint 1 at 0x84f4: file test.c, line 26. (gdb) run Starting program: /mnt/nessie/nokia770/testc-armel [Thread debugging using libthread_db enabled] [New Thread 16384 (LWP 1127)] BFD: /usr/lib/debug/lib/ld-2.3.6.so: warning: sh_link not set for section `.ARM.exidx' [New Thread 32769 (LWP 1130)] [New Thread 16386 (LWP 1131)] [Switching to Thread 16384 (LWP 1127)] Breakpoint 1, main (argc=1, argv=0xbe8e2744) at test.c:26 26 test.c: No such file or directory. in test.c (gdb) thread apply all bt Thread 3 (Thread 16386 (LWP 1131)): #0 0x40115a84 in sched_yield () at regex_internal.c:71 #1 0x40029af8 in __pthread_acquire (spinlock=0x40037d1c) at spinlock.c:711 #2 0x40024c90 in pthread_start_thread_event (arg=0xbe1ffbe0) at manager.c:329 #3 0x4012ae10 in clone () from /usr/lib/debug/libc.so.6 #4 0x4012ae10 in clone () from /usr/lib/debug/libc.so.6 Previous frame identical to this frame (corrupt stack?) Thread 2 (Thread 32769 (LWP 1130)): #0 0x40121f08 in *__GI___poll (fds=0x13000, nfds=1, timeout=2000) at ../sysdeps/unix/sysv/linux/poll.c:86 #1 0x4002520c in __pthread_manager (arg=0xfffc) at manager.c:152 #2 0x40025a88 in __pthread_manager_event (arg=0x46b) at manager.c:249 #3 0x4012ae10 in clone () from /usr/lib/debug/libc.so.6 #4 0x4012ae10 in clone () from /usr/lib/debug/libc.so.6 Previous frame identical to this frame (corrupt stack?) Thread 1 (Thread 16384 (LWP 1127)): #0 main (argc=1, argv=0xbe8e2744) at test.c:26 (gdb) cheers, Christian Henz #include #include #include void* routine( void* arg ) { int i = 0; while( i < 15 ) { sleep( 1 ); i++; } return 0; } int main( int argc, char** argv ) { pthread_t handle; void* retval; int err = pthread_create( &handle, 0, routine, 0 ); assert( err == 0 ); err = pthread_join( handle, &retval ); assert( err == 0 ); return 0; } ___ maemo-developers mailing list maemo-developers@maemo.org https://maemo.org/mailman/listinfo/maemo-developers
Re: [maemo-developers] Problem with GDB and threads...
Hi, > This finally did the trick. I configured and compiled glibc using > ./configure [...] CFLAGS="-g -O2 -fno-omit-frame-pointer" > > Now the backtrace doesn't repeat forever, although I still get > "Previous frame identical to this frame (corrupt stack?)". To debug functions which do not return (have attribute((noreturn))), you need to compile those functions with -fno-omit-frame-pointer (or remove the noreturn attribute). Both omitting a frame-pointer and noreturn attribute are just optimizations. To debug a static function (by debugging I mean having a backtrace going through that function), you need debug symbols for the library in which the function resides. libc6 debug package is quite large, but you can remove the .so files directly in /usr/lib/debug/ as Gdb works just fine also with the (smaller/newer) debug symbol files in the subdirectories of /usr/lib/debug/. - Eero ___ maemo-developers mailing list maemo-developers@maemo.org https://maemo.org/mailman/listinfo/maemo-developers