Curiouser and curiouser said alice... I booted Owl into FreeBSD 4.9
FreeBSD owl.burtonstrauss.us 4.9-RELEASE FreeBSD 4.9-RELEASE #0: Mon Oct 27 17:51:09 GMT 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC i386 synced the cvs, built and run, basic command: # /usr/bin/ntop -i sis0 @/etc/ntop.conf -u ntop -d # cat /etc/ntop.conf # BMS 07-2003 -u ntop -P /usr/share/ntop -w 3000 -t 4 --log-extra 2 -m ... --disable-instantsessionpurge And it does hang... # ps axf | grep ntop 21738 ?? Ss 0:06.05 /usr/bin/ntop -i sis0 @/etc/ntop.conf -u ntop -d top shows: PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND 21751 ntop 4 0 40616K 30052K bpf 0:06 8.22% 6.64% ntop Sounds like the same symptoms as y'all. Whatever it's doing, however, it's certainly using cpu. Let's gdb: (gdb) info thread 8 process 21648, thread 8 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 7 process 21648, thread 7 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 6 process 21648, thread 6 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 5 process 21648, thread 5 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 4 process 21648, thread 4 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 3 process 21648, thread 3 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 2 process 21648, thread 2 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 * 1 process 21648, thread 1 0x291d1900 in __sys_poll () from /usr/lib/libc_r.so.4 ODD... they're all in a wait state - that poll explains the cpu usage, however... (gdb) thread 8 [Switching to thread 8 (process 21648, thread 8)] #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 #1 0x291d0a3e in _thread_kern_sched_state_unlock () from /usr/lib/libc_r.so.4 #2 0x291d26b9 in pthread_cond_wait () from /usr/lib/libc_r.so.4 #3 0x2919278d in sem_wait () from /usr/lib/libc_r.so.4 #4 0x281436de in waitSem (semId=0x28156dec) at util.c:2058 #5 0x28133471 in dequeuePacket (notUsed=0x0) at pbuf.c:2199 #6 0x2919411c in _thread_start () from /usr/lib/libc_r.so.4 #7 0x0 in ?? () (gdb) thread 7 [Switching to thread 7 (process 21648, thread 7)] #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 #1 0x291d0a3e in _thread_kern_sched_state_unlock () from /usr/lib/libc_r.so.4 #2 0x291d2919 in pthread_cond_timedwait () from /usr/lib/libc_r.so.4 #3 0x291ba271 in _thread_gc () from /usr/lib/libc_r.so.4 #4 0x2919411c in _thread_start () from /usr/lib/libc_r.so.4 #5 0x0 in ?? () (gdb) thread 6 [Switching to thread 6 (process 21648, thread 6)] #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 #1 0x291d09d1 in _thread_kern_sched_state () from /usr/lib/libc_r.so.4 #2 0x291ede48 in _nanosleep () from /usr/lib/libc_r.so.4 #3 0x291cb0da in __sleep () from /usr/lib/libc_r.so.4 #4 0x29191a65 in sleep () from /usr/lib/libc_r.so.4 #5 0x2814604d in ntop_sleep (secs=60) at util.c:3560 #6 0x2812c1c8 in scanIdleLoop (notUsed=0x0) at ntop.c:613 #7 0x2919411c in _thread_start () from /usr/lib/libc_r.so.4 #8 0x0 in ?? () (gdb) thread 5 [Switching to thread 5 (process 21648, thread 5)] #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 #1 0x291d0a3e in _thread_kern_sched_state_unlock () from /usr/lib/libc_r.so.4 #2 0x291d26b9 in pthread_cond_wait () from /usr/lib/libc_r.so.4 #3 0x2919278d in sem_wait () from /usr/lib/libc_r.so.4 #4 0x281436de in waitSem (semId=0x28156df0) at util.c:2058 #5 0x2811d1b3 in dequeueAddress (notUsed=0x0) at address.c:566 #6 0x2919411c in _thread_start () from /usr/lib/libc_r.so.4 #7 0x0 in ?? () (gdb) thread 4 [Switching to thread 4 (process 21648, thread 4)] #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 #1 0x291d09d1 in _thread_kern_sched_state () from /usr/lib/libc_r.so.4 #2 0x291ede48 in _nanosleep () from /usr/lib/libc_r.so.4 #3 0x291cb0da in __sleep () from /usr/lib/libc_r.so.4 #4 0x29191a65 in sleep () from /usr/lib/libc_r.so.4 #5 0x2814604d in ntop_sleep (secs=207) at util.c:3560 #6 0x29570beb in rrdMainLoop (notUsed=0x0) at rrdPlugin.c:1537 #7 0x2919411c in _thread_start () from /usr/lib/libc_r.so.4 #8 0x0 in ?? () (gdb) thread 3 [Switching to thread 3 (process 21648, thread 3)] #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 #1 0x291d09d1 in _thread_kern_sched_state () from /usr/lib/libc_r.so.4 #2 0x291cda27 in _select () from /usr/lib/libc_r.so.4 #3 0x291cdc16 in select () from /usr/lib/libc_r.so.4 #4 0x280eacc7 in handleWebConnections (notUsed=0x0) at webInterface.c:8128 #5 0x2919411c in _thread_start () from /usr/lib/libc_r.so.4 #6 0x0 in ?? () (gdb) thread 2 [Switching to thread 2 (process 21648, thread 2)] #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 #1 0x291d09d1 in _thread_kern_sched_state () from /usr/lib/libc_r.so.4 #2 0x291cdd3c in _read () from /usr/lib/libc_r.so.4 #3 0x291cddb6 in read () from /usr/lib/libc_r.so.4 #4 0x29418de2 in pcap_read () from /usr/lib/libpcap.so.2 #5 0x2941897f in pcap_dispatch () from /usr/lib/libpcap.so.2 #6 0x2812b143 in pcapDispatch (_i=0x0) at ntop.c:83 #7 0x2919411c in _thread_start () from /usr/lib/libc_r.so.4 #8 0x0 in ?? () (gdb) thread 1 [Switching to thread 1 (process 21648, thread 1)] #0 0x291d1900 in __sys_poll () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d1900 in __sys_poll () from /usr/lib/libc_r.so.4 #1 0x291d0e4c in _thread_kern_sched_state_unlock () from /usr/lib/libc_r.so.4 #2 0x291d0811 in _thread_kern_scheduler () from /usr/lib/libc_r.so.4 #3 0x0 in ?? () So somehow we've got an OS deadlock in the thread scheduling routine... Stanley, we went through this once before ... something about different threading models. October 2003... http://search.gmane.org/search.php?query=freebsd+_thread_kern_sched&email=&g roup=gmane.linux.ntop.devel&sort=date Part of that was solved, the FILEDESCRIPTORBUG: Work-around activated [MSGID8498161] stuff. I know we were talking about something different in this message, http://article.gmane.org/gmane.linux.ntop.devel/3976 But the info threads data is eerily similar: (gdb) info threads 8 process 42844, thread 8 0x28303e77 in _thread_kern_sched () from /usr/lib/libc_r.so.4 7 process 42844, thread 7 0x28303e77 in _thread_kern_sched () from /usr/lib/libc_r.so.4 6 process 42844, thread 6 0x28303e77 in _thread_kern_sched () from /usr/lib/libc_r.so.4 5 process 42844, thread 5 0x28303e77 in _thread_kern_sched () from /usr/lib/libc_r.so.4 4 process 42844, thread 4 0x28303e77 in _thread_kern_sched () from /usr/lib/libc_r.so.4 3 process 42844, thread 3 0x28303e77 in _thread_kern_sched () from /usr/lib/libc_r.so.4 2 process 42844, thread 2 0x28303e77 in _thread_kern_sched () from /usr/lib/libc_r.so.4 * 1 process 42844, thread 1 0x280fd853 in mapGlobalToLocalIdx (port=1054) at ntop.c:549 I know I reported it... yeah: http://lists.freebsd.org/pipermail/freebsd-bugs/2003-September/003029.html it's still in the FreeBSD database, http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/56339, with not even a response... It does NOT look odd... dequeuePacket() and dequeueAddress() are waiting... that's normal scanIdleLoop() is sleeping... that's normal rrdMainLoop() is sleeping... also normal handleWebConnections() is waiting in select()... that's normal, waiting for a web server request pcapDispatch is the libpcap thread, waiting in read() isn't abnormal either as I understand things, thread 1 is the thread manager, so poll() there isn't weird... Now that leaves two threads I don't understand. 6 and 7... (gdb) thread 7 [Switching to thread 7 (process 21648, thread 7)] #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 #1 0x291d0a3e in _thread_kern_sched_state_unlock () from /usr/lib/libc_r.so.4 #2 0x291d2919 in pthread_cond_timedwait () from /usr/lib/libc_r.so.4 #3 0x291ba271 in _thread_gc () from /usr/lib/libc_r.so.4 #4 0x2919411c in _thread_start () from /usr/lib/libc_r.so.4 #5 0x0 in ?? () I'm guessing that gc means garbage control. (gdb) thread 6 [Switching to thread 6 (process 21648, thread 6)] #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 (gdb) info stack #0 0x291d020f in _thread_kern_sched () from /usr/lib/libc_r.so.4 #1 0x291d09d1 in _thread_kern_sched_state () from /usr/lib/libc_r.so.4 #2 0x291ede48 in _nanosleep () from /usr/lib/libc_r.so.4 #3 0x291cb0da in __sleep () from /usr/lib/libc_r.so.4 #4 0x29191a65 in sleep () from /usr/lib/libc_r.so.4 #7 0x2919411c in _thread_start () from /usr/lib/libc_r.so.4 #8 0x0 in ?? () If you google for "freebsd posix threads c" in the groups discussion there's a huge and interesting thread about a change in FreeBSD 4.9 regarding -pthreads linkage. More stuff... http://www.tcpdump.org/lists/workers/2001/10/msg00073.html old, but seemingly on point - we use select() and bpf. But that never got another response. More later... -----Burton _______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
