> -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Joe Williams > Sent: Tuesday, September 07, 2010 9:18 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog stats > > > On Sep 7, 2010, at 11:11 AM, Aaron Wiebe wrote: > > > On Tue, Sep 7, 2010 at 2:07 PM, <[email protected]> wrote: > >> > >> 1. stats that are cheap to gather, generate > >> examples current length of queues, age of oldest message in each > queue, > >> number of messages processed (possibly even per output) > >> 2. stats that are expensive to gather or generate > >> examples: average length of queues, average time to process > messages > > > > Expensive stats shouldn't be the responsibility of rsyslog though - > > trending over time and the like should be the responsibility of a > > third party product (ie, cacti/mrtg). As long as the stats are > easily > > made available through snmp or other methods, the trending should be > > done outside rsyslog. > > > > > > Agreed, rsyslog should just report the current stats and any > trending/analysis should be done externally. No different than running > iostat or other tools. > > One other timing stat I would love to see is the amount of time in each > step of the pipeline (time to read the message from the socket, time > processing it through any rules, time spent in any queues, time spent > writing to disk, etc).
Current system time resolution is probably not good enough to time single steps. In many cases, you would not see any time used at all. But I agree it would be interesting to have an indication. Just one example: the tcp receiver pull a relatively large buffer (64K?) from the socket. So a single receive may obtain 5,000 messages or more with a single receive. Let's assume this takes 0.0001 seconds. Does this mean we need to compute 0.0001/5000 = 0.00000002 seconds as the time required to obtain from the socket? Now think about a busy system processing eg 100,000 msgs per second. Processing involves various steps (more than 20 for a typical plain rsyslog.conf). So we need to measure time at the nanosecond level for each step. Rainer > Haproxy has a good example of this in it's > logging (8.2.3. HTTP log format - > http://haproxy.1wt.eu/download/1.4/doc/configuration.txt). > > -Joe > > > Name: Joseph A. Williams > Email: [email protected] > Blog: http://www.joeandmotorboat.com/ > Twitter: http://twitter.com/williamsjoe > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
