On Fri, Oct 28, 2011 at 11:52 AM, Vincent Riquer
<v.riq...@b2f-concept.com>wrote:
> Hi
>
> I noticed that sometimes, LiveStatus stopped being updated while all
> other components of shinken are still fully functional.
>
> I also noticed that when this happens, the brokerd consumes more and
> more memory, until it's OOM-killed. I understand the latter is caused by
> brokerd queuing events for livestatus. Maybe the broker should decide to
> restart a module that has more than 'x' queued events, with 'x' being
> configurable ?
>
> [...]
Traceback (most recent call last):
> File
>
> "/usr/local/lib/python2.6/dist-packages/multiprocessing-2.6.2.1-py2.6-linux-x86_64.egg/multiprocessing/queues.py",
> line 242, in _feed
> send(obj)
> RuntimeError: dictionary changed size during iteration
> Begin Loop : managing old broks 20
> External Queue len (Livestatus) : 19
> External Queue len (NPCD) : 13
> External Queue len (WebUI) : 10
>
Hi,
I just got such error too, and I don't know where it came from because the
traceback is too small here :(
I add this morning a new parameter for daemons :
# External modules watchdog. If a module got a brok queue() higher
# than this value, it will be killed and restart. Put to 0 to disable it
max_queue_size=100000
So if the external module is still "alive" from a process point of view but
it is not taking broks, it will be restarted. It's not a perfect way, more a
watchdog to be sure it will not eat all the server memory.
I'm wondering why we are getting the same error the same time, it's strange.
I don't remember change something in the Broker Queue management lately. Are
you also running python 2.6 on Centos5?
Jean
------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
Shinken-devel mailing list
Shinken-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shinken-devel
