any errors :-) go ahead and past pieces inline.
David Lang On Thu, 29 Nov 2012, Luke Marrott wrote:
Which parts of the debug would be the most beneficial? Should I attach it or paste pieces inline? :Luke Marrott On Mon, Nov 19, 2012 at 7:11 PM, David Lang <da...@lang.hm> wrote:On Mon, 19 Nov 2012, Luke Marrott wrote: So I have been trying to figure this out. Went through the config and gotrid of everything that I wasn't using or was commented out from the default template and it's still not getting as much as Splunk is getting so it has to be something with my installation or my configuration. I ran the config check -N 1 and here is the output: [root@nwcacti lmarrott]# /usr/local/sbin/rsyslogd -f /etc/rsyslog.conf -n -N 1 rsyslogd: version 5.8.10, config validation run (level 1), master config /etc/rsyslog.conf rsyslogd: WARNING: rsyslogd is running in compatibility mode. Automatically generated config directives may interfer with your rsyslog.conf settings. We suggest upgrading your config and adding -c5 as the first rsyslogd option. rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad immark rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: MarkMessagePeriod 1200 rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad imuxsock rsyslogd: End of config validation run. Bye. How do I upgrade my config? I also ran a debug and it seems like there are a lot of things it's complaining about. But then again maybe it's normal.start rsyslog with -c5 to avoid this particular error If you can send us the debug log (with the -c5) we can look at the errors that show up, but I suspect that things will work a LOT better for you with the -c5 David Lang:Luke Marrott On Fri, Nov 9, 2012 at 5:02 PM, David Lang <da...@lang.hm> wrote: I'm not sure exactly what will happen, but I suspect that all the logswill end up in all the possible destinations. I don't think rsyslog really will process all the local logs to one set of rules and all the remote logs to another set of rules At least, not unless you are using rulesets, which I am not seeing. a couple thousand log messages/sec should not cause any problems. David Lang On Fri, 9 Nov 2012, Luke Marrott wrote: Date: Fri, 9 Nov 2012 15:14:32 -0700From: Luke Marrott <luke.marr...@gmail.com> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com> To: rsyslog-users <rsyslog@lists.adiscon.com> Subject: Re: [rsyslog] rsyslog dropping logs Only one configuration there. I have all my messages going to directories by host so your method doesn't seem to be working. I did a tcpdump only on port 514 for a few seconds and I had like 2000 messages. :Luke Marrott On Fri, Nov 9, 2012 at 2:48 PM, David Lang <da...@lang.hm> wrote: are these two different configs (the sender and the receiver)?a simple way to see the message rate is to do a cut -f 1 -d ' ' logfiles |sort |uniq -c to look at the timestamps and see how many timestamps you have in a second. David Lang On Fri, 9 Nov 2012, Luke Marrott wrote: Date: Fri, 9 Nov 2012 13:07:02 -0700 From: Luke Marrott <luke.marr...@gmail.com>Reply-To: rsyslog-users <rsyslog@lists.adiscon.com> To: rsyslog-users <rsyslog@lists.adiscon.com> Subject: Re: [rsyslog] rsyslog dropping logs Full configuration: [root@hostname]# cat /etc/rsyslog.conf # if you experience problems, check # http://www.rsyslog.com/******troubleshoot<http://www.rsyslog.com/****troubleshoot> <http://www.**rsyslog.com/**troubleshoot<http://www.rsyslog.com/**troubleshoot><http://www.**rsyslog.com/**troubleshoot<http://rsyslog.com/troubleshoot> <http://www.**rsyslog.com/troubleshoot<http://www.rsyslog.com/troubleshoot>forassistance # rsyslog v3: load input modules # If you do not load inputs, nothing happens! # You may need to set the module load path if modules are not found. $ModLoad immark # provides --MARK-- message capability $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imklog # kernel logging (formerly provided by rklogd) # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! #*.error;mail.none;authpriv.******none;cron.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* -/var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit -/var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log # Remote Logging (we use TCP for reliable delivery) # An on-disk queue is created for this action. If the remote host is # down, messages are spooled to disk and sent when it is up again. #$WorkDirectory /rsyslog/spool # where to place spool files #$ActionQueueFileName uniqName # unique name prefix for spool files #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown #$ActionQueueType LinkedList # run asynchronously #$ActionResumeRetryCount -1 # infinite retries if host is down # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional #*.* @@remote-host:514 # ######### Receiving Messages from Remote Hosts ########## # TCP Syslog Server: # provides TCP syslog reception and GSS-API (if compiled to support it) $ModLoad imtcp.so # load module $InputTCPServerRun 514 # start up TCP listener at port 514 # UDP Syslog Server: $ModLoad imudp.so # provides UDP syslog reception $UDPServerRun 514 # start a UDP syslog server at standard port 514 $template Default,"/data/syslog/%******HOSTNAME%/%HOSTNAME%.log" *.* ?Default [root@hostname]# What's a good way to look at message ratE? :Luke Marrott On Fri, Nov 9, 2012 at 1:03 PM, David Lang <da...@lang.hm> wrote: On Fri, 9 Nov 2012, Luke Marrott wrote:Sorry. I wasn't real clear. The server runs on a big VM in another locationcompletely. No issues with the server during this time. This has been an ongoing thing. I'm running Splunk on the same box and if I turn off rsyslog and turn splunk on the same port it gets all the messages that don't seem to get picked up by rsyslog. Doesn't appear to be any rate limiting configuration. Ok, that is a different situation. In my experience, rsyslog issignicantly better than Splunk at receiving messages. I've testing rsyslog up to 380K messages/sec (gige wire speed) and others have tested rsyslog up to 1M messages/sec, so it's unlikely to be something fundamental to rsyslog, but it could easily be some resource contraint you are running into. can you post your full configuration? what message rate are you seeing? David Lang ______________________________********_________________ rsyslog mailing list http://lists.adiscon.net/********mailman/listinfo/rsyslog<http://lists.adiscon.net/******mailman/listinfo/rsyslog> <http**://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog><http:**//lists.adiscon.net/****mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/**listinfo/rsyslog> <htt**p://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog><http:**//lists.adiscon.net/****mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/**listinfo/rsyslog><htt**p://lists.adiscon.net/mailman/****listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog><htt**p://lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> <htt**p://lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>http://www.rsyslog.com/********professional-services/<http://www.rsyslog.com/******professional-services/> <http://**www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/><http://**www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/><http://**www.rsyslog.com/****professional-**services/<http://www.rsyslog.com/**professional-**services/><http:**//www.rsyslog.com/**professional-**services/<http://www.rsyslog.com/professional-**services/><http:**//www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/>What's up with rsyslog? Follow https://twitter.com/rgerhardsNOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ______________________________******_________________ rsyslog mailing listhttp://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog><http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> <htt**p://lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/><http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/><http://**www.rsyslog.com/**professional-**services/<http://www.rsyslog.com/professional-**services/> <http:**//www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>What's up with rsyslog? Follow https://twitter.com/rgerhardsNOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ______________________________******_________________ rsyslog mailing listhttp://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog><http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> <htt**p://lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/><http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/><http://**www.rsyslog.com/**professional-**services/<http://www.rsyslog.com/professional-**services/> <http:**//www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>What's up with rsyslog? Follow https://twitter.com/rgerhardsNOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ______________________________****_________________rsyslog mailing list http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/>What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ______________________________****_________________rsyslog mailing list http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/>What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ______________________________**_________________rsyslog mailing list http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ______________________________**_________________rsyslog mailing list http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT._______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
