I forgot to mention that I’m running rsyslog 7.4.6: # rpm -q rsyslog rsyslog-7.4.6-1.el6.x86_64
On Nov 6, 2013, at 12:01 PM, Leggett, Torrance I. <[email protected]> wrote: > I have a central logging server that accepts messages and writes them out - > <year>/<month>/<host>. However, recently I’ve had a problem where the server > stops writing out virtually all such messages and the main message queue > fills and starts writing to the disk asssisted queue. Once it starts doing > this, the DA queue only grows until it hits the max size or the disk fills. > Restarting doesn’t seem to help unless I remove all the files in the DA spool > directory. Below is the server portion of the config. If you need more > configs or debugging, just let me know what. I’m relatively new to debugging > rsyslog issues. > > # cat 99-server.conf > # Switch to server ruleset > $RuleSet server > > $MainMsgQueueFileName mainqueue # unique name prefix for spool files > $MainMsgQueueType LinkedList # main queue should be a dynamic list > in memory > $MainMsgQueueSize 100000 # increase the queue size to handle > the message traffic > $MainMsgQueueHighWatermark 80000 # increase the high water mark to > write messages to disk > $MainMsgQueueLowWatermark 20000 # increase the low water mark to stop > writing to disk > $MainMsgQueueMaxDiskSpace 1g # 1gb disk space limit > $MainMsgQueueSaveOnShutdown off # save messages to disk on shutdown > $MainMsgQueueWorkerThreads 5 # spawn up to 5 threads for queue > processing > $MaxMessageSize 8k # handle larger messages if needed > $RepeatedMsgReduction off # log all messages as they come > > # Load UDP module > $ModLoad imudp > $InputUDPServerBindRuleset server > $UDPServerRun 514 > > # Load TCP module > $ModLoad imtcp > $InputTCPServerBindRuleset server > $InputTCPServerRun 514 > > # Load RELP module > $ModLoad imrelp > $InputRELPServerBindRuleset server > $InputRELPServerRun 20514 > > # Send logs to logstash for indexing > *.* @@127.0.0.1:5544;RSYSLOG_TraditionalForwardFormat > > # Templates > $Template auditFormat,"%MSG%\n" > $Template radiusFormat,"%MSG%\n" > $Template tsmFormat,"%MSG%\n" > $Template dynAuditLog,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/audit.log" > $Template dynAuthLog,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/secure" > $Template dynCronLog,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/cron.log" > $Template > dynDaemonLog,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/daemon.log" > $Template dynDebug,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/debug" > $Template > dynHttpAccess,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/http_access.log" > $Template > dynHttpError,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/http_error.log" > $Template dynKernLog,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/kern.log" > $Template dynMailLog,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/mail.log" > $Template > dynPuppetAgent,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/puppet-agent.log" > $Template > dynPuppetMaster,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/puppet-master.log" > $Template > dynRadiusLog,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/radius.log" > $Template dynSyslog,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/messages" > $Template dynTsmInfo,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/dsmcmd.log" > $Template > dynTsmError,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/dsmerror.log" > $Template dynUserLog,"/var/log/remotes/%$YEAR%/%$MONTH%/%HOSTNAME%/user.log" > > # First capture auditd messages from remotes > # > if $programname == 'auditd' and $syslogfacility-text == 'local6' then > ?dynAuditLog;auditFormat > > # Next capture RADIUS messages from remotes > # > if $programname == 'radiusd' and $syslogfacility-text == 'local6' then > ?dynRadiusLog;radiusFormat > > # Next handle any apache logs and remove them from the stream > # > if $programname == 'httpd' and $syslogfacility-text == 'local6' then { > ?dynHttpAccess > stop > } > if $programname == 'httpd' and $syslogfacility-text == 'local7' then { > ?dynHttpError > stop > } > > # Next handle any nginx logs and remove them from the stream > # > if $programname == 'nginx' and $syslogfacility-text == 'local6' then { > ?dynHttpAccess > stop > } > if $programname == 'nginx' and $syslogfacility-text == 'local7' then { > ?dynHttpError > stop > } > > # Next handle any puppet logs and remove them from the stream > # > if $programname == 'puppet-agent' then { > ?dynPuppetAgent > stop > } > if $programname == 'puppet-master' then { > ?dynPuppetMaster > stop > } > > # Next handle any TSM logs and remove them from the stream > # > if $programname == 'dsmc' and $syslogfacility-text == 'local3' and > $syslogseverity-text == 'info' then ?dynTsmInfo;tsmFormat > if $programname == 'dsmserv' and $syslogfacility-text == 'local3' and > $syslogseverity-text == 'err' then ?dynTsmError;tsmFormat > > # Rules > auth,authpriv.* ?dynAuthLog > *.*;\ > mail.none;\ > cron.none -?dynSyslog > cron.* ?dynCronLog > daemon.* -?dynDaemonLog > kern.* -?dynKernLog > mail.* -?dynMailLog > user.* -?dynUserLog > > # Switch back to default ruleset > $RuleSet RSYSLOG_DefaultRuleset >
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
