On Thu, 22 Oct 2009, John P. Rouillard wrote: > In message <[email protected]>, > [email protected] writes: >> On Thu, 22 Oct 2009, Clayton Dukes wrote: >>> If you're using the latest version of php-syslog-ng it has a built in >>> deduplication function. >>> http://code.google.com/p/php-syslog-ng >> >> most syslog daemons have the ability to say 'last message repeated X >> times' >> >> rsyslog has an option to include the message that was repeated at the end >> of the 'last message repeated' line > > Yeah I usually disable the "last message repeated X times" first thing > 8-). However if you are getting multiple events/second then it's > usually easy enough to work around. > >> I have rsyslog handling tens of thousands of messages/sec currently with >> fairly low cpu utilization (<20% of one core) and tested it about 6 >> months ago up to ~80K messages/sec sustained, >300K messages/sec peak >> (essentially wire speed of Gig-E). the current dev version has >> significant improvements that should allow this to go significantly >> higher, but the kinks are still being worked out of it. > > Just out of curiosity how many of those messages are being passed to SEC?
I actually have a complex logging infrastructure, I have the syslog messages being sent to different farms of machines for archiving, database inserts (ad-hoc querying), and alerting. on the SEC alerting system I have rsyslog do filtering on programname and then send different programnames to different instances of SEC (so that each instance of SEC only sees fairly relavent logs) I haven't checked recently to see what the total log volume going to SEC is (or what the peak CPU utilization of SEC is). actually, looking at it I do have one instance that is seeing every syslog message, but that instance only has two rules in it. that instance is using about the same CPU as rsyslog is, the next busiest instance is useing about 1/5 of that the peak messages/sec tests mentioned above were to receive and write to disk. I expect that writing to a pipe would be the same, doing more complex things would probably slow it down. David Lang ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
