Hi
This indeed seems to be a valid solution, I launched it many times, with no
more issue on the broker. I'm going to put back my script in production,
thanks to your help.
Many thanks Gerhard
Denis
2011/11/2 Gerhard Lausser <gerhard.laus...@consol.de>
> Hi,****
>
> ** **
>
> maybe the Nagios livestatus module is more robust. I took your script and
> it crashed, but only when I had hosts without a comment. I rewrote it a
> little bit and it works now (CentOS5.6,64,Python 2.4.3)****
>
> ** **
>
> def main_function():****
>
> host_table = send_query("GET hosts\nFilter: state = 0\nColumns:
> name comments\nOutputFormat: csv\n")****
>
> for host in host_table:****
>
> host_name, host_comment_ids = host****
>
> ** **
>
> try:****
>
> host_last_comment_id = host_comment_ids.split(',')[-1]
> #use only last comment ****
>
> host_last_comment_id = str(int(host_last_comment_id))
> # if it is not a number (or empty), then it raises an exception****
>
> comment = send_query("GET comments\nColumns:
> comment_id comment\nFilter: comment_id = "+host_last_comment_id+"\n")****
>
> print comment****
>
> except Exception:****
>
> print host_name + " has no comments"****
>
> ** **
>
> OMD[denis]:~$ python denis.py ****
>
> [['16', 'kaas']]****
>
> [['2', 'schmarrn']]****
>
> omd11 has no comments****
>
> [['4', 'schmarrn']]****
>
> [['5', 'schmarrn']]****
>
> [['6', 'schmarrn']]****
>
> [['7', 'schmarrn']]****
>
> [['17', 'kaas-omd-2']]****
>
> [['9', 'schmarrn']]****
>
> [['10', 'schmarrn']]****
>
> [['11', 'schmarrn']]****
>
> [['12', 'schmarrn']]****
>
> [['13', 'schmarrn']]****
>
> [['14', 'schmarrn']]****
>
> [['15', 'schmarrn']]****
>
> ** **
>
> Even if I add lots of main_function() calls to the script, I get valid
> output.****
>
> Can you please try it with my modifications?****
>
> ** **
>
> Gerhard****
>
> ** **
>
> *Von:* Denis GERMAIN [mailto:dt.germ...@gmail.com]
> *Gesendet:* Mittwoch, 2. November 2011 18:08
> *An:* shinken-devel@lists.sourceforge.net
> *Betreff:* Re: [Shinken-devel] Slight problem with livestatus module with
> direct queries****
>
> ** **
>
> Hi again,
>
> I might have been a little optimistic, because my prod crashed again.
>
> This time I used python code which has been adapted from the website from
> Livestatus conceptor, so I guess that this time the queries are correctly
> done
>
> This error is really strange, because it seems to be realted to the number
> of request I do in a short period of time. I managed to reproduce the error
> on my dev environment *relief*. Here, everything works fine until the
> broker crashes.
>
> I might be totally wrong here as I don't know much of Livestatus coding in
> Shinken, but is it possible that the maximum number of socket has been
> reached or something like that?
>
> Attached, you can see the scenario I used to reproduce the error :
> I first added comments on running hosts (in order to have a certain amount
> of hosts with comments on them)
> Then I launched my script
> Livestatus answered correctly for some hosts, then crashed, like on prod
> env
>
> If this is indeed a problem with max number of sockets in livestatus
> module, I could recode my script in order to avoid making too many queries
> (getting all hosts, then getting all comments, only 2 queries here), but
> I'd like to be sure it's really that in order to avoid further issues on
> prod environnement
>
> Regards
>
> Denis GERMAIN****
>
> 2011/10/27 Olivier Hanesse <olivier.hane...@gmail.com>****
>
> hi,****
>
> ** **
>
> I used to work with netcat too, and I got some issues in the past with
> some netcat version.****
>
> On debian for example, you have 2 versions : ****
>
> ** **
>
> netcat-openbsd - TCP/IP swiss army knife****
>
> netcat-traditional - TCP/IP swiss army knife****
>
> ** **
>
> One was working, but not the other one.****
>
> Maybe you could give a try and install the other version.****
>
> ** **
>
> Regards****
>
> ** **
>
> Olivier****
>
> ** **
>
> 2011/10/27 Denis GERMAIN <dt.germ...@gmail.com>****
>
> Hi,
>
> Thanks for the answer, I had forgotten that we used netcat instead of
> unixcat. I have still to confirm this, but i'm think we have a problem
> because we're not correctly closing the connection with netcat.
>
> I'm going to code it in python in order to do this in a proper way.
>
> I'll keep you posted if this is not the issue but I'm pretty sure that
> this is our problem.
>
> Thanks!
>
> Denis****
>
> 2011/10/26 Gerhard Lausser <gerhard.laus...@consol.de>****
>
> Hi,****
>
> ****
>
> which version of Shinken do you use?****
>
> I have here a 0.8 and can’t reproduce your error. There’s one host
> (omd-disco) which I made DOWN by submitting a passive command and which has
> two comments attached.****
>
> ****
>
> OMD[disco]:~$ printf "GET hosts\nFilter: state = 1\nColumns: name
> comments\n"|unixcat /omd/sites/disco/tmp/run/live****
>
> omd-disco;3,4****
>
> ****
>
> OMD[disco]:~$ printf 'GET comments\nColumns: comment_id comment\n' |
> unixcat /omd/sites/disco/tmp/run/live ****
>
> 3;i bi hi****
>
> 4;i kos ned oft gnou sogn, i bi hi****
>
> ****
>
> OMD[disco]:~$ printf 'GET comments\nColumns: comment_id comment\nFilter:
> comment_id = 3\n' | unixcat /omd/sites/disco/tmp/run/live ****
>
> 3;i bi hi****
>
> ****
>
> Gerhard****
>
> ****
>
> ****
>
> *Von:* Denis GERMAIN [mailto:dt.germ...@gmail.com]
> *Gesendet:* Mittwoch, 26. Oktober 2011 16:40
> *An:* shinken-devel@lists.sourceforge.net
> *Betreff:* [Shinken-devel] Slight problem with livestatus module with
> direct queries****
>
> ****
>
> Hello list,
>
> For historic reporting purpose, we have a developper in our team that
> directly execute queries to the livestatus module via commands like
> printf 'GET hosts\nFilter: state = 1' | nc 10.197.64.97 50000 >
> /data/ftptrace/nagios/thruk.txt
> printf 'GET comments\nFilter comment_id = '$commid | nc 10.197.64.97 50000
> | tail -1 | awk -F";" '{print $2}'*
>
> *This gives us informations about hosts down which are then reformatted
> in a dusty old report sent to our client.
> At the beginning we used only the first request for our report, as
> comments were directly available from the host information given by the
> query. After an update, we didn't realised that the comments were not
> directly in the query result, but only the comment ID (which I grant you is
> much more "clean")
>
> 1) Problem is that now, we need to query for the comments (selecting good
> comment ID) and that it constantly break livestatus module (which doesn't
> always manage to relaunch itself)
>
> 2) We also noted that when we use the Column statement (to select only the
> columns we need), the livestatus module instantly crash
>
> I realise that our problem is a little exotic, as the vast majority of
> livestatus users only use it through Thruk, but if something could be done
> (or as already been done), we would be grateful (our developper mostly :-p)
>
> Denis GERMAIN
>
> PS: An example traceback. I don't have much more yet, we are running
> Shinken on production (I already see admins frowning eyes, but it's stable
> enough if you don't try strange things like us) and my manager don't really
> like us crashing it on purpose. We'll try to reproduce on preproduction
> ASAP, but if this can help pinpointing the issue...
>
> [1319536549] [broker-1] Traceback (most recent call last):
> File
> "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_broker.py",
> line 828, in main
> self.do_main()
> File
> "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_broker.py",
> line 1052, in do_main
> response, keepalive =
> self.livestatus.handle_request(open_connections[socketid]['buffer'])
> File
> "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus.py",
> line 129, in handle_request
> request.parse_input(data)
> File
> "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_request.py",
> line 96, in parse_input
> query.parse_input('\n'.join(query_cmds))
> File
> "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_query.py",
> line 259, in parse_input
> self.filter_stack.put(self.make_filter(operator, attribute, reference))
> File
> "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_query.py",
> line 786, in make_filter
> converter = self.find_converter(attribute)
> File
> "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/livestatus_query.py",
> line 102, in find_converter
> out_map = LSout_map[self.out_map_name]
> KeyError: 'hosts'****
>
> ** **
>
>
> ------------------------------------------------------------------------------
> The demand for IT networking professionals continues to grow, and the
> demand for specialized networking skills is growing even more rapidly.
> Take a complimentary Learning@Cisco Self-Assessment and learn
> about Cisco certifications, training, and career opportunities.
> http://p.sf.net/sfu/cisco-dev2dev
> _______________________________________________
> Shinken-devel mailing list
> Shinken-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/shinken-devel****
>
>
>
>
> ------------------------------------------------------------------------------
> The demand for IT networking professionals continues to grow, and the
> demand for specialized networking skills is growing even more rapidly.
> Take a complimentary Learning@Cisco Self-Assessment and learn
> about Cisco certifications, training, and career opportunities.
> http://p.sf.net/sfu/cisco-dev2dev
> _______________________________________________
> Shinken-devel mailing list
> Shinken-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/shinken-devel****
>
> ** **
>
>
>
> ------------------------------------------------------------------------------
> The demand for IT networking professionals continues to grow, and the
> demand for specialized networking skills is growing even more rapidly.
> Take a complimentary Learning@Cisco Self-Assessment and learn
> about Cisco certifications, training, and career opportunities.
> http://p.sf.net/sfu/cisco-dev2dev
> _______________________________________________
> Shinken-devel mailing list
> Shinken-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/shinken-devel****
>
> ** **
>
>
> ------------------------------------------------------------------------------
> RSA® Conference 2012
> Save $700 by Nov 18
> Register now!
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Shinken-devel mailing list
> Shinken-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/shinken-devel
>
>
------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
Shinken-devel mailing list
Shinken-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shinken-devel
