Yeah.. lots of expires may have something to do with it, your traceback shows
TableEntryVal16ExpireAccessTimeEv But I also wonder what you are doing that is triggering Dictionary9NextEntryERP7HashKeyRP10IterCookiei which would be void* Dictionary::NextEntry(HashKey*& h, IterCookie*& cookie, int return_hash) const Tables should be fine, but I wonder what you're doing that is triggering so much iteration. -- - Justin Azoff > On Jan 18, 2017, at 1:40 PM, Aashish Sharma <[email protected]> wrote: > > Yes, I have been making heavy use of tables ( think a million entries a day > and million expires a day) > > Let me figure out a way to upload the scripts on github or send them yours > and justin's way otherwise. > > Strangely this code kept running fine for last month and reasonably stable. I > am not sure what little thing I added/changed that has caused bro to run but > all workers in uwait state with 6% CPU. (I'll be doing svn diffs to figure > out) > > Seems like bro is stuck in: > > 0x00000004039ccadc in _umtx_op_err () from /lib/libthr.so.3 > (gdb) bt > #0 0x00000004039ccadc in _umtx_op_err () from /lib/libthr.so.3 > #1 0x00000004039c750b in _thr_umtx_timedwait_uint () from /lib/libthr.so.3 > #2 0x00000004039cea06 in ?? () from /lib/libthr.so.3 > #3 0x00000000009042ee in > threading::Queue<threading::BasicInputMessage*>::Get (this=0x404543038) at > /home/bro/install/bro-2.5/src/threading/Queue.h:173 > #4 0x0000000000902a31 in threading::MsgThread::RetrieveIn (this=0x404543000) > at /home/bro/install/bro-2.5/src/threading/MsgThread.cc:349 > #5 0x0000000000902ce4 in threading::MsgThread::Run (this=0x404543000) at > /home/bro/install/bro-2.5/src/threading/MsgThread.cc:366 > #6 0x00000000008fb952 in threading::BasicThread::launcher (arg=0x404543000) > at /home/bro/install/bro-2.5/src/threading/BasicThread.cc:201 > #7 0x00000004039c6260 in ?? () from /lib/libthr.so.3 > #8 0x0000000000000000 in ?? () > Backtrace stopped: Cannot access memory at address 0x7fffffbfe000 > (gdb) > > > > Aashish > > On Wed, Jan 18, 2017 at 06:37:08PM +0100, Jan Grashöfer wrote: >> Hi Aashish, >> >>> So I am running a new detection package and everything seemed right but >>> somehow since yesterday each worker is running at 5.7% to 6.3% CPU and not >>> generating logs. >> >> my guess would be that the script makes (heavy) use of tables and table >> expiration, right? Can you share the script? >> >> Jan >> _______________________________________________ >> bro-dev mailing list >> [email protected] >> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > _______________________________________________ > bro-dev mailing list > [email protected] > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev _______________________________________________ bro-dev mailing list [email protected] http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
