It happens both with dnacluster interfaces and dna without the use of
dnacluster. It acts almost like demo mode in some ways as a few seconds
of traffic get logged before the segfault. I have both DNA and libzero
licenses installed and have used them previous to the update. The main
change to this system is an OS update and then running a newer version
of PF_RING (5.6.2 to 6.0.3). Here is an example debug of this with a DNA
interface and a back trace including threads (the last one listed seems
to be the one having problems):
gdb /nsm/bro/bin/bro
GNU gdb (GDB) SLES Expanded Support platform (7.2-75.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /nsm/bro/bin/bro...done.
(gdb) run -i dna0@0
Starting program: /nsm/bro/bin/bro -i dna0@0
[Thread debugging using libthread_db enabled]
listening on dna0@0, capture length 8192 bytes
[New Thread 0x7fffe4a18700 (LWP 46337)]
[New Thread 0x7fffdffff700 (LWP 46338)]
[New Thread 0x7fffdf5fe700 (LWP 46339)]
[New Thread 0x7fffd7fff700 (LWP 46340)]
[New Thread 0x7fffdebfd700 (LWP 46341)]
[New Thread 0x7fffde1fc700 (LWP 46342)]
[New Thread 0x7fffdd7fb700 (LWP 46343)]
[New Thread 0x7fffdcdfa700 (LWP 46344)]
[New Thread 0x7fffd75fe700 (LWP 46345)]
[New Thread 0x7fffd6bfd700 (LWP 46346)]
[New Thread 0x7fffd61fc700 (LWP 46347)]
[New Thread 0x7fffd57fb700 (LWP 46348)]
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7959506 in pcap_read_packet (handle=0x26315d0,
callback=0x7ffff795d720 <pcap_oneshot>, userdata=0x7fffffffd9c0 "") at
./pcap-linux.c:1807
1807 for (cmsg = CMSG_FIRSTHDR(&msg); cmsg; cmsg =
CMSG_NXTHDR(&msg, cmsg)) {
Missing separate debuginfos, use: debuginfo-install
GeoIP-1.5.1-5.el6.x86_64 glibc-2.12-1.149.el6_6.5.x86_64
keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64
libcom_err-1.41.12-21.el6.x86_64 libgcc-4.4.7-11.el6.x86_64
libselinux-2.0.94-5.8.el6.x86_64 libstdc++-4.4.7-11.el6.x86_64
numactl-2.0.9-2.el6.x86_64 openssl-1.0.1e-30.el6_6.5.x86_64
zlib-1.2.3-29.el6.x86_64
(gdb) thread apply all bt
Thread 13 (Thread 0x7fffd57fb700 (LWP 46348)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0xf1ac458) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0xf1ac420) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0xf1ac420) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0xf1ac420) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 12 (Thread 0x7fffd61fc700 (LWP 46347)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0xb75c698) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0xb75c660) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0xb75c660) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0xb75c660) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 11 (Thread 0x7fffd6bfd700 (LWP 46346)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0xb7561a8) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0xb756170) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0xb756170) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0xb756170) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 10 (Thread 0x7fffd75fe700 (LWP 46345)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0x80d2798) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0x80d2760) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0x80d2760) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0x80d2760) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 9 (Thread 0x7fffdcdfa700 (LWP 46344)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0x49e9fa8) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0x49e9f70) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0x49e9f70) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0x49e9f70) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 8 (Thread 0x7fffdd7fb700 (LWP 46343)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0x578cf28) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0x578cef0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0x578cef0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0x578cef0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x7fffde1fc700 (LWP 46342)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0x56f4578) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0x56f4540) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0x56f4540) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0x56f4540) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x7fffdebfd700 (LWP 46341)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0x496e358) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0x496e320) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0x496e320) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0x496e320) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7fffd7fff700 (LWP 46340)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0x36c8488) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0x36c8450) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0x36c8450) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0x36c8450) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x7fffdf5fe700 (LWP 46339)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0x361f3e8) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0x361f3b0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0x361f3b0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0x361f3b0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7fffdffff700 (LWP 46338)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0x36101d8) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0x36101a0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0x36101a0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0x36101a0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
---Type <return> to continue, or q <return> to quit---
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7fffe4a18700 (LWP 46337)):
#0 0x00007ffff5f5e98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000081b39a in
threading::Queue<threading::BasicInputMessage*>::Get (this=0x357ecf8) at
/nsm/bro/git/bro2.3-419/bro/src/threading/Queue.h:173
#2 0x0000000000819b49 in threading::MsgThread::RetrieveIn
(this=0x357ecc0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:349
#3 0x0000000000819c64 in threading::MsgThread::Run (this=0x357ecc0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/MsgThread.cc:366
#4 0x0000000000816058 in threading::BasicThread::launcher
(arg=0x357ecc0) at
/nsm/bro/git/bro2.3-419/bro/src/threading/BasicThread.cc:201
#5 0x00007ffff5f5a9d1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff62588fd in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7ffff7fe07e0 (LWP 46334)):
#0 0x00007ffff7959506 in pcap_read_packet (handle=0x26315d0,
callback=0x7ffff795d720 <pcap_oneshot>, userdata=0x7fffffffd9c0 "") at
./pcap-linux.c:1807
#1 0x00007ffff795d79b in pcap_next (p=<value optimized out>, h=<value
optimized out>) at ./pcap.c:218
#2 0x0000000000a4a490 in iosource::pcap::PcapSource::ExtractNextPacket
(this=0x26313c0, pkt=0x26313f8) at
/nsm/bro/git/bro2.3-419/bro/src/iosource/pcap/Source.cc:151
#3 0x0000000000a7580c in iosource::PktSrc::ExtractNextPacketInternal
(this=0x26313c0) at /nsm/bro/git/bro2.3-419/bro/src/iosource/PktSrc.cc:432
#4 0x0000000000a7511b in iosource::PktSrc::NextTimestamp
(this=0x26313c0, local_network_time=0x7fffffffdc58) at
/nsm/bro/git/bro2.3-419/bro/src/iosource/PktSrc.cc:241
#5 0x0000000000a71193 in iosource::Manager::FindSoonest (this=0xf232e0,
ts=0x7fffffffdd68) at /nsm/bro/git/bro2.3-419/bro/src/iosource/Manager.cc:82
#6 0x00000000007895d1 in net_run () at
/nsm/bro/git/bro2.3-419/bro/src/Net.cc:301
#7 0x00000000006d8ed7 in main (argc=3, argv=0x7fffffffe438) at
/nsm/bro/git/bro2.3-419/bro/src/main.cc:1200
On 2/25/2015 11:54 AM, Alfredo Cardigliano wrote:
This is definitely strange, I should take a look at the bro code to see what it
does,
is this happening with “dnacluster:X" interafaces only?
Alfredo
On 25 Feb 2015, at 17:12, Gary Faulkner <[email protected]> wrote:
pcount appears to work fine and doesn't produce any sort of segfault or crash.
On 2/25/2015 2:48 AM, Alfredo Cardigliano wrote:
Hi Gary
I do not see anything obvious from the trace, could you try also with pcount
(without f, that is our sample application over pcap),
to figure out where the problem could be?
Thank you
Alfredo
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
