Excerpts from David Lang's message of 2014-12-16 20:41:49 +0100: > On Tue, 16 Dec 2014, Marc Fournier wrote: > > > Hello, > > > > openldap writes its logs using syslog(3). Three log messages are emitted > > for each ldap search operation. > > > > On a given openldap setup, I reach about 2500 operations per second. The > > simple fact of shutting rsyslog down icreases this number up to 5000 > > (logs are lost obviously). > > > > Writing to a local file, to /dev/null, or to a remote server over UDP > > doesn't significantly change this number. I tried tweaking various > > settings related to the main queue but didn't see any huge difference. > > > > ipmstats shows exactly the same numbers for "imuxsock: submitted" than > > for "main Q: enqueued". All the discarded/full/failed/etc counters have > > a value of 0. > > > > This is rsyslog-8.6.0-2.el6.x86_64 from the adiscon RPM repo (but had > > similar problems with version 5.8.10-9.el6_6 which comes with el6). > > > > Here's the relevant bits of my config: > > > > module(load="impstats" log.syslog="off" log.file="/var/log/impstats.log" > > resetCounters="off" interval="5") > > module(load="imuxsock" SysSock.RateLimit.Interval="0" > > SysSock.Annotate="off" SysSock.FlowControl="off" > > SysSock.IgnoreTimestamp="off") > > > > $MaxMessageSize 2k > > > > $MainMsgQueueDequeueBatchSize 1024 > > $MainMsgQueueType LinkedList > > $MainMsgQueueSize 100000 > > $MainMsgQueueWorkerThreads 1 > > > > $RepeatedMsgReduction off > > $SystemLogRateLimitInterval 0 > > > > $ActionQueueType LinkedList > > $ActionQueueSize 100000 > > $ActionQueueDiscardMark 95000 > > $ActionQueueDiscardSeverity 0 > > $ActionQueueTimeoutEnqueue 0 > > $ActionQueueDequeueSlowdown 1000 > > $ActionQueueWorkerThreads 2 > > $ActionQueueDequeueBatchSize 128 > > $ActionResumeRetryCount -1 > > > > #local4.* -/var/log/ldap/ldap.log > > #local4.* ~ > > local4.* -/dev/null > > #local4.* @localhost > > > > So I'm wondering which one of rsyslog or /dev/log is actually the > > bottleneck here and what can I do about it ? > > /dev/log is just a socket to talk to rsyslog > > One major problem here, why do you have ActionQueueDequeueSlowdown > configured? > that deliberatly pauses before trying to process the next messages. > > you really shouldn't need to have an action queue for writing to a local > file, > you should remove all of the actionqueue parameters (and if you do decide > that > you want a queue, use the new action() syntax, it makes it much clearer > what's > happening)
Ok, right my config is a mess I agree ! Been iterating over it since back when I used 5.8.10. So I started over with a brand new config. The best results I got so far was with these settings: module(load="impstats" log.syslog="off" log.file="/var/log/impstats.log" resetCounters="off" interval="5") module(load="imuxsock" SysSock.RateLimit.Interval="0" SysSock.Annotate="off" SysSock.FlowControl="off" SysSock.IgnoreTimestamp="off") $MaxMessageSize 2k $MainMsgQueueDequeueBatchSize 4096 $MainMsgQueueType LinkedList $MainMsgQueueSize 250000 $MainMsgQueueWorkerThreads 4 $MainMsgQueueWorkerThreadMinimumMessages 100000 $RepeatedMsgReduction off $SystemLogRateLimitInterval 0 *.* ~ The numbers are slightly better now: ~3500 ops/s vs 2500 yesterday. But still a significant difference with the max I reach when running with rsyslogd stopped (~5000 ops/s). Is this an expected compromise ? Or are there possible improvements I can do ? Thanks for sharing your wisdom ! Marc _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
