Which parts of the debug would be the most beneficial? Should I attach it or paste pieces inline?
:Luke Marrott On Mon, Nov 19, 2012 at 7:11 PM, David Lang <da...@lang.hm> wrote: > On Mon, 19 Nov 2012, Luke Marrott wrote: > > So I have been trying to figure this out. Went through the config and got >> rid of everything that I wasn't using or was commented out from the >> default >> template and it's still not getting as much as Splunk is getting so it has >> to be something with my installation or my configuration. >> >> I ran the config check -N 1 and here is the output: >> [root@nwcacti lmarrott]# /usr/local/sbin/rsyslogd -f /etc/rsyslog.conf -n >> -N 1 >> rsyslogd: version 5.8.10, config validation run (level 1), master config >> /etc/rsyslog.conf >> rsyslogd: WARNING: rsyslogd is running in compatibility mode. >> Automatically >> generated config directives may interfer with your rsyslog.conf settings. >> We suggest upgrading your config and adding -c5 as the first rsyslogd >> option. >> rsyslogd: Warning: backward compatibility layer added to following >> directive to rsyslog.conf: ModLoad immark >> rsyslogd: Warning: backward compatibility layer added to following >> directive to rsyslog.conf: MarkMessagePeriod 1200 >> rsyslogd: Warning: backward compatibility layer added to following >> directive to rsyslog.conf: ModLoad imuxsock >> rsyslogd: End of config validation run. Bye. >> >> >> How do I upgrade my config? >> >> I also ran a debug and it seems like there are a lot of things it's >> complaining about. But then again maybe it's normal. >> > > start rsyslog with -c5 to avoid this particular error > > If you can send us the debug log (with the -c5) we can look at the errors > that show up, but I suspect that things will work a LOT better for you with > the -c5 > > David Lang > > >> :Luke Marrott >> >> >> >> On Fri, Nov 9, 2012 at 5:02 PM, David Lang <da...@lang.hm> wrote: >> >> I'm not sure exactly what will happen, but I suspect that all the logs >>> will end up in all the possible destinations. I don't think rsyslog >>> really >>> will process all the local logs to one set of rules and all the remote >>> logs >>> to another set of rules >>> >>> >>> At least, not unless you are using rulesets, which I am not seeing. >>> >>> >>> a couple thousand log messages/sec should not cause any problems. >>> >>> >>> David Lang >>> >>> On Fri, 9 Nov 2012, Luke Marrott wrote: >>> >>> Date: Fri, 9 Nov 2012 15:14:32 -0700 >>> >>>> >>>> From: Luke Marrott <luke.marr...@gmail.com> >>>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com> >>>> To: rsyslog-users <rsyslog@lists.adiscon.com> >>>> Subject: Re: [rsyslog] rsyslog dropping logs >>>> >>>> Only one configuration there. >>>> >>>> I have all my messages going to directories by host so your method >>>> doesn't >>>> seem to be working. >>>> >>>> I did a tcpdump only on port 514 for a few seconds and I had like 2000 >>>> messages. >>>> >>>> :Luke Marrott >>>> >>>> >>>> >>>> On Fri, Nov 9, 2012 at 2:48 PM, David Lang <da...@lang.hm> wrote: >>>> >>>> are these two different configs (the sender and the receiver)? >>>> >>>>> >>>>> a simple way to see the message rate is to do a >>>>> cut -f 1 -d ' ' logfiles |sort |uniq -c to look at the timestamps and >>>>> see >>>>> how many timestamps you have in a second. >>>>> >>>>> David Lang >>>>> >>>>> >>>>> On Fri, 9 Nov 2012, Luke Marrott wrote: >>>>> >>>>> Date: Fri, 9 Nov 2012 13:07:02 -0700 >>>>> >>>>> From: Luke Marrott <luke.marr...@gmail.com> >>>>>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com> >>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com> >>>>>> >>>>>> Subject: Re: [rsyslog] rsyslog dropping logs >>>>>> >>>>>> Full configuration: >>>>>> [root@hostname]# cat /etc/rsyslog.conf >>>>>> # if you experience problems, check >>>>>> # >>>>>> http://www.rsyslog.com/******troubleshoot<http://www.rsyslog.com/****troubleshoot> >>>>>> <http://www.**rsyslog.com/**troubleshoot<http://www.rsyslog.com/**troubleshoot> >>>>>> > >>>>>> <http://www.**rsyslog.com/**troubleshoot<http://rsyslog.com/troubleshoot> >>>>>> <http://www.**rsyslog.com/troubleshoot<http://www.rsyslog.com/troubleshoot> >>>>>> >>for >>>>>> >>>>>> assistance >>>>>> >>>>>> >>>>>> # rsyslog v3: load input modules >>>>>> # If you do not load inputs, nothing happens! >>>>>> # You may need to set the module load path if modules are not found. >>>>>> >>>>>> $ModLoad immark # provides --MARK-- message capability >>>>>> $ModLoad imuxsock # provides support for local system logging (e.g. >>>>>> via >>>>>> logger command) >>>>>> $ModLoad imklog # kernel logging (formerly provided by rklogd) >>>>>> >>>>>> # Log all kernel messages to the console. >>>>>> # Logging much else clutters up the screen. >>>>>> #kern.* /dev/console >>>>>> >>>>>> # Log anything (except mail) of level info or higher. >>>>>> # Don't log private authentication messages! >>>>>> #*.error;mail.none;authpriv.******none;cron.none >>>>>> >>>>>> >>>>>> /var/log/messages >>>>>> >>>>>> # The authpriv file has restricted access. >>>>>> authpriv.* >>>>>> /var/log/secure >>>>>> >>>>>> # Log all the mail messages in one place. >>>>>> mail.* >>>>>> -/var/log/maillog >>>>>> >>>>>> >>>>>> # Log cron stuff >>>>>> cron.* -/var/log/cron >>>>>> >>>>>> # Everybody gets emergency messages >>>>>> *.emerg * >>>>>> >>>>>> # Save news errors of level crit and higher in a special file. >>>>>> uucp,news.crit >>>>>> -/var/log/spooler >>>>>> >>>>>> # Save boot messages also to boot.log >>>>>> local7.* >>>>>> /var/log/boot.log >>>>>> >>>>>> # Remote Logging (we use TCP for reliable delivery) >>>>>> # An on-disk queue is created for this action. If the remote host is >>>>>> # down, messages are spooled to disk and sent when it is up again. >>>>>> #$WorkDirectory /rsyslog/spool # where to place spool files >>>>>> #$ActionQueueFileName uniqName # unique name prefix for spool files >>>>>> #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >>>>>> possible) >>>>>> #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown >>>>>> #$ActionQueueType LinkedList # run asynchronously >>>>>> #$ActionResumeRetryCount -1 # infinite retries if host is down >>>>>> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >>>>>> #*.* @@remote-host:514 >>>>>> >>>>>> >>>>>> # ######### Receiving Messages from Remote Hosts ########## >>>>>> # TCP Syslog Server: >>>>>> # provides TCP syslog reception and GSS-API (if compiled to support >>>>>> it) >>>>>> $ModLoad imtcp.so # load module >>>>>> $InputTCPServerRun 514 # start up TCP listener at port 514 >>>>>> >>>>>> # UDP Syslog Server: >>>>>> $ModLoad imudp.so # provides UDP syslog reception >>>>>> $UDPServerRun 514 # start a UDP syslog server at standard port 514 >>>>>> >>>>>> >>>>>> $template Default,"/data/syslog/%******HOSTNAME%/%HOSTNAME%.log" >>>>>> >>>>>> >>>>>> *.* ?Default >>>>>> >>>>>> >>>>>> [root@hostname]# >>>>>> >>>>>> >>>>>> What's a good way to look at message ratE? >>>>>> >>>>>> >>>>>> :Luke Marrott >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Nov 9, 2012 at 1:03 PM, David Lang <da...@lang.hm> wrote: >>>>>> >>>>>> On Fri, 9 Nov 2012, Luke Marrott wrote: >>>>>> >>>>>> >>>>>>> Sorry. I wasn't real clear. The server runs on a big VM in another >>>>>>> >>>>>>> location >>>>>>> >>>>>>>> completely. No issues with the server during this time. This has >>>>>>>> been >>>>>>>> an >>>>>>>> ongoing thing. I'm running Splunk on the same box and if I turn off >>>>>>>> rsyslog >>>>>>>> and turn splunk on the same port it gets all the messages that don't >>>>>>>> seem >>>>>>>> to get picked up by rsyslog. >>>>>>>> >>>>>>>> Doesn't appear to be any rate limiting configuration. >>>>>>>> >>>>>>>> >>>>>>>> Ok, that is a different situation. In my experience, rsyslog is >>>>>>>> >>>>>>> signicantly better than Splunk at receiving messages. I've testing >>>>>>> rsyslog >>>>>>> up to 380K messages/sec (gige wire speed) and others have tested >>>>>>> rsyslog >>>>>>> up >>>>>>> to 1M messages/sec, so it's unlikely to be something fundamental to >>>>>>> rsyslog, but it could easily be some resource contraint you are >>>>>>> running >>>>>>> into. >>>>>>> >>>>>>> can you post your full configuration? >>>>>>> >>>>>>> what message rate are you seeing? >>>>>>> >>>>>>> >>>>>>> David Lang >>>>>>> ______________________________********_________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog<http://lists.adiscon.net/******mailman/listinfo/rsyslog> >>>>>>> <http**://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>>> > >>>>>>> <http:**//lists.adiscon.net/****mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/**listinfo/rsyslog> >>>>>>> <htt**p://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> <http:**//lists.adiscon.net/****mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/**listinfo/rsyslog> >>>>>>> <htt**p://lists.adiscon.net/mailman/****listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> >>>>>>> > >>>>>>> <htt**p://lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> >>>>>>> <htt**p://lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> http://www.rsyslog.com/********professional-services/<http://www.rsyslog.com/******professional-services/> >>>>>>>> <http://**www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>>> > >>>>>>>> >>>>>>> <http://**www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> <http://**www.rsyslog.com/****professional-**services/<http://www.rsyslog.com/**professional-**services/> >>>>>>> <http:**//www.rsyslog.com/**professional-**services/<http://www.rsyslog.com/professional-**services/> >>>>>>> > >>>>>>> <http:**//www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>> > >>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>> myriad >>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>> you >>>>>>> DON'T LIKE THAT. >>>>>>> >>>>>>> ______________________________******_________________ >>>>>>> >>>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>> > >>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> >>>>>> <htt**p://lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>> > >>>>>> >>>>>>> >>>>>>> >>>>>>> http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>> > >>>>>> <http://**www.rsyslog.com/**professional-**services/<http://www.rsyslog.com/professional-**services/> >>>>>> <http:**//www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >>>>>> > >>>>>> >>>>>>> >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>> myriad >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>> DON'T LIKE THAT. >>>>>> >>>>>> ______________________________******_________________ >>>>>> >>>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>> > >>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> >>>>> <htt**p://lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>> > >>>>> >>>>>> >>>>>> >>>>>> http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>> > >>>>> <http://**www.rsyslog.com/**professional-**services/<http://www.rsyslog.com/professional-**services/> >>>>> <http:**//www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >>>>> > >>>>> >>>>>> >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> >>>>> ______________________________****_________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>> > >>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> ______________________________****_________________ >>>> >>> rsyslog mailing list >>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>> > >>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>> > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> > http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
