Re: [rsyslog] dns lookup disabled

2014-01-17 Thread David Lang
On Fri, 17 Jan 2014, robert s wrote: David, thanks for the response, I see, so currently I have a custom /etc/host file with 100+ host names they are formated like this: ipaddres host1.firewall (literraly host1, and so on) ipaddress host2.firewall ipaddress host3.shib ipaddress host4.shib

Re: [rsyslog] dns lookup disabled

2014-01-17 Thread robert s
David, thanks for the response, I see, so currently I have a custom /etc/host file with 100+ host names they are formated like this: ipaddres host1.firewall (literraly host1, and so on) ipaddress host2.firewall ipaddress host3.shib ipaddress host4.shib and I was filtering with ":hostname, con

Re: [rsyslog] Lost messages after server restart? (using RELP)

2014-01-17 Thread Assaf Gordon
Hello David, On 01/17/2014 06:15 PM, David Lang wrote: On Fri, 17 Jan 2014, Assaf Gordon wrote: BUT, If I understood the idea behind "RELP" correctly, the client-side "rsyslogd" should know and detect that those messages were not ACK'd, hence not delivered. your understanding matches mine th

Re: [rsyslog] Lost messages after server restart? (using RELP)

2014-01-17 Thread David Lang
On Fri, 17 Jan 2014, Assaf Gordon wrote: David, Thank you for the detailed explanation. I think I'm in a simpler situation, which does not require the additional protection you've listed. On 01/17/2014 05:48 PM, David Lang wrote: On Fri, 17 Jan 2014, Assaf Gordon wrote: now, with all of th

Re: [rsyslog] Lost messages after server restart? (using RELP)

2014-01-17 Thread Assaf Gordon
David, Thank you for the detailed explanation. I think I'm in a simpler situation, which does not require the additional protection you've listed. On 01/17/2014 05:48 PM, David Lang wrote: On Fri, 17 Jan 2014, Assaf Gordon wrote: now, with all of this said, let's get back to your situation a

Re: [rsyslog] Lost messages after server restart? (using RELP)

2014-01-17 Thread David Lang
On Fri, 17 Jan 2014, Assaf Gordon wrote: Hello, On 01/17/2014 04:35 PM, David Lang wrote: RELP will prevent logs from being lost in transit, but if the log arrived on the server, but was in the queue when rsyslog was killed, and rsyslog didn't have time to write it out, you can still loose thi

Re: [rsyslog] Lost messages after server restart? (using RELP)

2014-01-17 Thread Assaf Gordon
On 01/17/2014 04:57 PM, Radu Gheorghe wrote: I think the most probable cause is that the lost messages were in the in-memory queue, which got wiped during restart. To fix that, you'd have to make it disk-assisted and set queue.saveonshutdown to yes. I think you'll find the details in those two li

Re: [rsyslog] Lost messages after server restart? (using RELP)

2014-01-17 Thread Assaf Gordon
Hello, On 01/17/2014 04:35 PM, David Lang wrote: RELP will prevent logs from being lost in transit, but if the log arrived on the server, but was in the queue when rsyslog was killed, and rsyslog didn't have time to write it out, you can still loose things. how did you kill the rsyslog receiver

Re: [rsyslog] Lost messages after server restart? (using RELP)

2014-01-17 Thread Radu Gheorghe
I think the most probable cause is that the lost messages were in the in-memory queue, which got wiped during restart. To fix that, you'd have to make it disk-assisted and set queue.saveonshutdown to yes. I think you'll find the details in those two links: http://www.rsyslog.com/doc/queues.html ht

Re: [rsyslog] Uncompression of a message failed with return code -3

2014-01-17 Thread Vladimir Marek
> >Or maybe rsyslog supposes that all messages have some sort of date/time > >at the start, so that it can't start with letter 'z'? > > > >>> > Yup - valid syslog messages NEED to start with "<" (actually ). See > RFC5424 & RFC3164. > >>> > >>>Oh, right! Out of your head, do you

Re: [rsyslog] Lost messages after server restart? (using RELP)

2014-01-17 Thread David Lang
RELP will prevent logs from being lost in transit, but if the log arrived on the server, but was in the queue when rsyslog was killed, and rsyslog didn't have time to write it out, you can still loose things. how did you kill the rsyslog receiver? what does your config look like? (how much stu

[rsyslog] Lost messages after server restart? (using RELP)

2014-01-17 Thread Assaf Gordon
Hello, I've setup rsyslogd (version 7.5.8) with RELP and TLS on two machines (one server, one client). For testing purposes, I've sent repeated messages from the client to the server, but killed and restarted the server during those messages. The result is that I've lost some messages, despi

Re: [rsyslog] Main queue settings in new format?

2014-01-17 Thread Radu Gheorghe
Thanks, Rainer! Added to bookmarks :) @James: I'm running on 8.1.4 so I tend to (•_•) ( •_•)>⌐■-■ (⌐■_■) skip the "requires..." part :D 2014/1/16 Boylan, James > Ah! Since I'm actively using 7.4.7 that would explain why I hadn't seen > that setting yet. > > Thanks Rainer. > > -- James > > >

Re: [rsyslog] Queue Alerting?

2014-01-17 Thread David Lang
On Fri, 17 Jan 2014, Radu Gheorghe wrote: I think embedding alerting functionality in rsyslog would be out of its scope. For example, threshold-based alerts are often criticized for being too rigid, so monitoring systems tend to add different algorithms to detect anomalies. My current favorite

Re: [rsyslog] Queue Alerting?

2014-01-17 Thread David Lang
I agree that alerting is outside the scope of rsyslog, but I think that values like this should be exposed to the config (a variation on global variables) At that point, they could be used in filter rules if you wanted to do so, but a better thing would be to output them and let an external too

Re: [rsyslog] Queue Alerting?

2014-01-17 Thread Boylan, James
I agree with Radu. That kind of alerting it well outside the scope of Rsyslog and you would also have to consider the potential performance impact that kind of monitoring would have on the application. -- James -Original Message- From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog

Re: [rsyslog] Queue Alerting?

2014-01-17 Thread Radu Gheorghe
Hi Nick, There's queue.discardmark and queue.discardseverity which helps keep queue sizes in check. Other than that, I guess it's best to monitor queue sizes with impstats and do alerting and actions with your monitoring system. For example Nagios/Shinken can give you those things. WARNING! Opin

Re: [rsyslog] dns lookup disabled

2014-01-17 Thread David Lang
I think you are confusing yourself. the hostname field is not supposed to be set by DNS or /etc/hosts, the fromhost field is. But if the message arrives without a valid hostname field, rsyslog will use fromhost instead. The hostname field is supposed to be set by the machine generating the lo

Re: [rsyslog] Uncompression of a message failed with return code -3

2014-01-17 Thread David Lang
On Fri, 17 Jan 2014, Vladimir Marek wrote: Or maybe rsyslog supposes that all messages have some sort of date/time at the start, so that it can't start with letter 'z'? Yup - valid syslog messages NEED to start with "<" (actually ). See RFC5424 & RFC3164. Oh, right! Out of your head, do yo

[rsyslog] Queue Alerting?

2014-01-17 Thread Nick Syslog
Would it be possible/feasible to consider adding some sort of queue alerting features to the server in order to more closely monitor when queuing could potentially get out of control? (or maybe this has already being suggested...) Something along the lines of the following (parameter options in pa

Re: [rsyslog] dns lookup disabled

2014-01-17 Thread robert s
ok I may be looking at this incorrectly, from other posts on the mailing lists, and responses, rsyslog does not really look at /etc/hosts? from some of the responses that I am reading I see that it may depend on the distro the gethostname() I guess determines the localhost so the question is th

Re: [rsyslog] Uncompression of a message failed with return code -3

2014-01-17 Thread Vladimir Marek
> > > > Or maybe rsyslog supposes that all messages have some sort of date/time > > > > at the start, so that it can't start with letter 'z'? > > > > > > > > > Yup - valid syslog messages NEED to start with "<" (actually ). See > > > RFC5424 & RFC3164. > > > > Oh, right! Out of your head, do you th

Re: [rsyslog] dns lookup disabled

2014-01-17 Thread robert s
it shows that the flag is started ps -fe | grep rsyslog root 8064 1 1 09:05 ?00:00:00 /sbin/rsyslogd -x -i /var/run/syslogd.pid Robert On Fri, Jan 17, 2014 at 9:02 AM, robert s wrote: > I tried changing nsswitch to not do dns, and configured nscd to not do > lookups, and hav

Re: [rsyslog] Uncompression of a message failed with return code -3

2014-01-17 Thread Rainer Gerhards
On Fri, Jan 17, 2014 at 3:01 PM, Vladimir Marek wrote: > > > Or maybe rsyslog supposes that all messages have some sort of date/time > > > at the start, so that it can't start with letter 'z'? > > > > > > Yup - valid syslog messages NEED to start with "<" (actually ). See > > RFC5424 & RFC3164. >

Re: [rsyslog] dns lookup disabled

2014-01-17 Thread robert s
I tried changing nsswitch to not do dns, and configured nscd to not do lookups, and have even disabled nscd, and removing resolve.conf settings Robert On Thu, Jan 16, 2014 at 3:14 PM, robert s wrote: > Thanks for the response Mike, I rsysced the configuration from the > test servers to the prod

Re: [rsyslog] Uncompression of a message failed with return code -3

2014-01-17 Thread Vladimir Marek
> > Or maybe rsyslog supposes that all messages have some sort of date/time > > at the start, so that it can't start with letter 'z'? > > > Yup - valid syslog messages NEED to start with "<" (actually ). See > RFC5424 & RFC3164. Oh, right! Out of your head, do you think that _not_ starting by '<'

Re: [rsyslog] Uncompression of a message failed with return code -3

2014-01-17 Thread Rainer Gerhards
On Fri, Jan 17, 2014 at 2:12 PM, Vladimir Marek wrote: > Hi, > > Recently we hit an issue with rsyslog logging the following message: > > 2014-01-17T12:33:55.563262+00:00 solaris rsyslogd: Uncompression of a > message failed with return code -3 - enable debug logging if you need > further informat

[rsyslog] Uncompression of a message failed with return code -3

2014-01-17 Thread Vladimir Marek
Hi, Recently we hit an issue with rsyslog logging the following message: 2014-01-17T12:33:55.563262+00:00 solaris rsyslogd: Uncompression of a message failed with return code -3 - enable debug logging if you need further information. Message ignored. The problem is that rsyslogd tries to uncom