Re: [rsyslog] Controlling Hostname

[Chris via rsyslog]

 Apologies if this is noise to the list, but I thought maybe someone else may 
find it interesting.
Change the hostname of your Amazon Linux instance - Amazon Elastic Compute Cloud


| 
| 
|  | 
Change the hostname of your Amazon Linux instance - Amazon Elastic Compu...

Set the hostname for your Amazon Linux instance using a dynamic DNS provider.
 |

 |

 |




On Wednesday, March 23, 2022, 04:11:37 PM EDT, David Lang  
wrote:  
 
 managing the hostname in the AWS instance is far better. I don't know the 
details, but there is some ability to run a config script at startup time, you 
could have that set the hostname (say something like 'function-count') and get 
more value from the hostname

David Lang
  
___
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Controlling Hostname

[Chris via rsyslog]

Thank you for your response, I just found that there's likely a more elegant 
solution that requires less configuration. 
There's an AWS config file that allows the dhcp EC2 instances to "preserve 
hostname". 
I'll try that before I try to tackle the template approach. I can provide a 
link to that AWS guidance if anyone happens to be interested.Thanks again David.




   On Wednesday, March 23, 2022, 03:44:14 PM EDT, David Lang  
wrote:  
 
 create a template that has whatever text you want in the hostname field and 
then 
use that when sending a message

on the receiving side (the relay), you can look at fromhost-ip or fromhost and 
then use that in a template while relaying it

David Lang

On Wed, 23 Mar 2022, Chris via rsyslog wrote:

> Date: Wed, 23 Mar 2022 18:08:10 + (UTC)
> From: Chris via rsyslog 
> To: "rsyslog@lists.adiscon.com" 
> Cc: Chris 
> Subject: [rsyslog] Controlling Hostname
> 
> I have several Linux instances in an Amazon VPC. They send UDP 514 to a 
> singular free tier ubuntu server running rsyslog. 
> It aggregates all incoming messages and sends them over TLS to a primary log 
> server running mysql and Loganalyzer on it. 
> Amazon makes controlling the hostname necessary because most hostnames look 
> something like ip-10-0-99-199. 
> I was thinking maybe there was a way I could force the host that is 
> originating a syslog message send it's message as an "IP address" versus the 
> hostname. 
> If that were true, I could likely set any name I wanted in the /etc/hosts on 
> the primary server and then primary server could just resolve the hostname on 
> the primary server. Is that possible? If so how would I accomplish that?
> Thanks in advance,CB
> ___
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
> LIKE THAT.  
___
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Controlling Hostname

[Chris via rsyslog]

I have several Linux instances in an Amazon VPC. They send UDP 514 to a 
singular free tier ubuntu server running rsyslog. 
It aggregates all incoming messages and sends them over TLS to a primary log 
server running mysql and Loganalyzer on it. 
Amazon makes controlling the hostname necessary because most hostnames look 
something like ip-10-0-99-199. 
I was thinking maybe there was a way I could force the host that is originating 
a syslog message send it's message as an "IP address" versus the hostname. 
If that were true, I could likely set any name I wanted in the /etc/hosts on 
the primary server and then primary server could just resolve the hostname on 
the primary server. Is that possible? If so how would I accomplish that?
Thanks in advance,CB
___
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Proven solution

[Chris Bartram via rsyslog]

Everybody advises me this http://northbengalhomestay.com/original.php 
<http://northbengalhomestay.com/original.php/> 

 

 

Chris Bartram

 

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] issues in rsyslog-8.36.0 with systemd service file

[Chris Richmond]

Hi,

   Far from an expert, but I'd built rsyslog-8.36.0 from sources on an RPi
and Centos6.9
boxes so I could enable RELP, and the RPi at some point was having issues
with systemd.
Like this:
Aug 31 04:10:01 pi2a systemd[1]: rsyslog.service start operation timed out.
Terminating.
Aug 31 04:10:02 pi2a systemd[1]: Unit rsyslog.service entered failed state.
Aug 31 04:10:44 pi2a systemd[1]: [/etc/systemd/system/rsyslog.service:8]
Failed to parse service type, ignoring: Simple
Aug 31 04:12:14 pi2a systemd[1]: rsyslog.service start operation timed out.
Terminating.
Aug 31 04:12:14 pi2a systemd[1]: Unit rsyslog.service entered failed state.

And this in /var/log/messages:

Aug 31 04:10:02 pi2a rsyslogd: environment variable TZ is not set, auto
correcting this to TZ=/etc/localtime  [v8.36.0 try
http://www.rsyslog.com/e/2442 ]
Aug 31 04:10:02 pi2a rsyslogd:  [origin software="rsyslogd"
swVersion="8.36.0" x-pid="21681" x-info="http://www.rsyslog.com;] start
Aug 31 04:12:14 pi2a rsyslogd: environment variable TZ is not set, auto
correcting this to TZ=/etc/localtime  [v8.36.0 try
http://www.rsyslog.com/e/2442 ]
Aug 31 04:12:14 pi2a rsyslogd:  [origin software="rsyslogd"
swVersion="8.36.0" x-pid="21736" x-info="http://www.rsyslog.com;] start

It took forever to figure out what was going on, but I could tell from the
systemctl (and the actual things being logged) that
the daemon was coming up just fine and systemd was the thing having the
problem.  In the end I googled for the processes
state (loaded active waiting) and came up with the thing that fixed the
issue, which was changing the Type from =notify to =Simple
I think the deal is that =notify is waiting for to the process to exit and
it doesn't of course, whereas =Simple assumes this is a daemon
and maybe watches for the pid file to update to indicate which process to
track.  Setting the -i seemed to make sense, and the
Resart=on-success made sense at the time since I was trying to stop the
incessant restarts (from system man pages).

pi2a_/home/crichmon/Downloads/rsyslog> diff rsyslog-8.36.0/rsyslog.service
/etc/systemd/system/rsyslog.service 
8,9c8,10
< Type=notify
< ExecStart=/usr/local/sbin/rsyslogd -n -iNONE
---
> Type=Simple
> ExecStart=/usr/local/sbin/rsyslogd -n -i /var/run/syslogd.pid
> PIDFile=/var/run/syslogd.pid
11c12
< Restart=on-failure
---
> Restart=on-success

Still not sure how to get $TZ set or to what value, but that can be done in
the rsyslog.service file as well.
The Centos box (router) logs in UTC, but the RPi logs in local time (I'm in
PST+DST = -7).
Sep  2 15:04:43 router weewx[11873]: reportengine: copied 0 files to
/home/weewx/public_html.weather
Sep  2 08:05:15 pi2a weewx[21909]: manager: Added record 2018-09-02 08:05:00
PDT (1535900700) to database 'weewx.sdb'

Hopefully this will help someone else.

Chris

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Every few minutes rsyslog outputs - rsyslogd: action 'action 3' resumed (module 'builtin:omfile') [v8.32.0 try http://www.rsyslog.com/e/2359 ]

[Chris via rsyslog]

This is on a Ubuntu 18.04.1LTS system that was upgraded last week from
16.04.5LTS. The version of rsyslog installed is:

apt-cache policy rsyslog
rsyslog:
  Installed: 8.32.0-1ubuntu4
  Candidate: 8.32.0-1ubuntu4

The complete output is here:

https://pastebin.com/AxYYQaw5

I went to the links noted. The first one http://www.rsyslog.com/e/2359
from what I can read tells me that whatever action is referenced it was 
resumed. I assume in this case it refers to this "resumed (module 
'builtin:omfile'" The 2nd link http://www.rsyslog.com/e/2007 seems to give me a 
fix for this. I've
looked for what is mentioned in the 2nd link in my /etc/rsyslog.conf
file and in my /etc/rsyslog.d/50-default.conf:

A frequent case for this error message on Debian-based distributions
(like raspbian) is that rsyslog.conf contains the instruction to write
to the xconsole pipe, but this pipe is never read. If so, you can
simply delete these lines to remove the error message. These lines are
usually found at the end of rsyslog.conf.

My current /etc/rsyslog.conf file is here https://pastebin.com/WZVhryNW

If I need to add some lines to the .conf file I'm not sure what they
should be.

Chris

-- 
Chris
KeyID 0xE372A7DA98E6705C
31.11972; -97.90167 (Elev. 1092 ft)
15:32:38 up 2:34, 1 user, load average: 1.55, 1.77, 1.72
Description:Ubuntu 18.04.1 LTS, kernel 4.15.0-32-generic

signature.asc
Description: This is a digitally signed message part
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Rsyslog warning message

[Cheltenham, Chris via rsyslog]

Does anyone know how to clean up this warning message from local
/var/log/messages.

 

May  9 12:22:51 devcas5 rsyslogd: error during parsing file
/etc/rsyslog.conf, on or before line 75: 

parameter 'statefile' deprecated but accepted, consider removing or
replacing it [v8.24.0 try http://www.rsyslog.com/e/2207 ]

 

My rsyslog.conf file has the following ON the client not the server.

 

 

 

input(type="imfile"

  File="/opt/apache-tomcat/logs/catalina.out"

  Tag="devsso-catalina"

  Facility="local6"

  StateFile="/var/spool/rsyslog/catalina.out"

  Severity="info")

 

 

===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

David,
That may be the key this entire issue.
I will try  it on the server side.

However, we are moving to graylog , or possibly and I was trying to filer
it from the client side its being routed to two different servers at the
moment.


===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 

-Original Message-
From: David Lang <da...@lang.hm> 
Sent: Monday, April 30, 2018 5:07 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] excluding ip addresses

On Mon, 30 Apr 2018, Cheltenham, Chris wrote:

> -Original Message-
> From: David Lang <da...@lang.hm>
>
> the thing you do not seem to understand is that you have not been able 
> to show us any log from the source that you are wanting to block.
>
> --  I was showing you the rsyslog data from the client , not the 
> server side

ahh, we were assuming that you were showing us data from the server side,
since it only makes sense to filter on the server side (on the sending
side, fromhost-ip is going to be 127.0.0.1, not the network IP)

> We are trying to help figure out what is happening with the logs, but 
> we don't know your network, so we are trying to help you see what's 
> happening so that you can tell us.
>
> -- I understand I just cannot spend an inordinate amount of time on 
> something that's is really a luxury for us
> -- Maybe I worded that reply and you got the wrong impression

Part of the reason I was spending the time was to teach you the
troubleshooting method :-) As you start using the more advanced features,
there are going to be more times when the result is not what you initially
expect, so the method of looking at the logs in the debug format to see
what you actually have there (as opposed to what you expect to have there)
is required.

good luck, and post again if you have other issues

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

See below 

===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 

-Original Message-
From: David Lang <da...@lang.hm> 
Sent: Monday, April 30, 2018 2:53 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] excluding ip addresses

the thing you do not seem to understand is that you have not been able to
show us any log from the source that you are wanting to block.

--  I was showing you the rsyslog data from the client , not the server
side

This may be because the machine has multiple IP addresses and it's
arriving from a different IP, it may be because you are relaying the
message, so fromhost-ip has the relay IP
 -- Yes it is multi-homed

but from what you have shown, nothing is arriving at the rsyslog machine
from the IP you are wanting to block.

--  I see that as well

We are trying to help figure out what is happening with the logs, but we
don't know your network, so we are trying to help you see what's happening
so that you can tell us.

-- I understand I just cannot spend an inordinate amount of time on
something that's is really a luxury for us
-- Maybe I worded that reply and you got the wrong impression

I'm sorry that you feel that the troubleshooting is too much bother, but
any other syslog daemon is going to have the same problem. If you block by
source IP, but messages aren't arriving from that source IP, they won't be
blocked.

-- It is too much bother when I can grep -v those ip address out when
troubleshooting , it's just not that important to me I have other things
of more importance.


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

Nevermind guys, but thanks.
I don't care anymore.
It's too much of a pain.

I'll use something else.


===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 

-Original Message-
From: David Lang <da...@lang.hm> 
Sent: Thursday, April 26, 2018 2:23 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] excluding ip addresses

you don't need to run in debug mode, just write a file using that template

/var/log/debuglog;RSYSLOG_DebugFormat

will write all logs this way.


On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> Date: Thu, 26 Apr 2018 14:08:10 -0400 (EDT)
> From: "Cheltenham, Chris" <ccheltenham-...@philasd.org>
> To: David Lang <da...@lang.hm>
> Cc: "Cheltenham, Chris via rsyslog" <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] excluding ip addresses
> 
> David,
>
> How do I run in debug mode?
> Is I rsyslog -d ?
>
> I am using CentOS 7 so it would be changed in systemd.
>
>
> ===
>
> Thank You;
>
> Chris Cheltenham
> Technology Services
> The School District of Philadelphia
>
> Work # 215-400-5025
> Cell # 215-301-6571
>
> -Original Message-
> From: David Lang <da...@lang.hm>
> Sent: Thursday, April 26, 2018 2:05 PM
> To: Cheltenham, Chris <ccheltenham-...@philasd.org>
> Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] excluding ip addresses
>
> On Thu, 26 Apr 2018, Cheltenham, Chris wrote:
>
>> David,
>>
>> Thanks for the reply.
>>
>> I used this
>>
>> if $fromhost-ip == '170.235.1.248' then stop if $fromhost-ip == 
>> '170.235.1.249' then stop
>>
>>
>> but it did not work.
>> Is that the correct syntax?
>
> that works
>
> can you log using the format RSYSLOG_DebugFormat and double check that 
> fromhost-ip is being set the way you expect it to be?
>
>> I als restarted rsyslog.
>
> yes, that is needed any time you change the config file.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

Rainer,

I appreciate your assistance.
This rsyslog I configured differently.
I'm out of ideas but if you come up with anything , we'd appreciate it.





=== 

Thank You; 

Chris Cheltenham 
Technology Services 
The School District of Philadelphia 

Work # 215-400-5025 
Cell # 215-301-6571

- Original Message -
From: "Rainer Gerhards" <rgerha...@hq.adiscon.com>
To: "Cheltenham, Chris" <ccheltenham-...@philasd.org>
Cc: "David Lang" <da...@lang.hm>, "rsyslog" <rsyslog@lists.adiscon.com>
Sent: Saturday, April 28, 2018 7:16:04 AM
Subject: Re: [rsyslog] excluding ip addresses

the debuglog does not contain any message from .248, so it does not
help. I would still be interested in seeing the one where the messages
were contained.

Rainer

2018-04-27 15:14 GMT+02:00 Cheltenham, Chris <ccheltenham-...@philasd.org>:
> David ,
>
> In case you wanted to see the debuglog and rsyslog.conf and
> /var/log/messages.
> None of it is very big so you won't have to parse through a ton of stuff.
>
> We push these logs to two place at the moment.
>
> Graylog and rsyslog server.
>
> We are attempting to deprecate the rsyslog server for the fancy outputs
> from Graylog.
>
>
>
>
>
> ===
>
> Thank You;
>
> Chris Cheltenham
> Technology Services
> The School District of Philadelphia
>
> Work # 215-400-5025
> Cell # 215-301-6571
>
>
> -Original Message-
> From: David Lang <da...@lang.hm>
> Sent: Thursday, April 26, 2018 4:29 PM
> To: Cheltenham, Chris <ccheltenham-...@philasd.org>
> Cc: Rainer Gerhards <rgerha...@hq.adiscon.com>; rsyslog-users
> <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] excluding ip addresses
>
> On Thu, 26 Apr 2018, Cheltenham, Chris wrote:
>
>> I tried this as well.
>>
>> This is version 8.24 also.
>>
>>
>>
>> -/etc/rsyslog.conf
>>
>> # Use traditional timestamp format
>>
>> #
>>
>> # DeBugging
>>
>> /var/log/debuglog;RSYSLOG_DebugFormat
>>
>> #
>>
>> :msg, contains, "170.235.1.248" ~
>>
>> :msg, contains, "170.235.1.249" ~
>>
>> #
>>
>>
>>
>>
>>
>>
>>
>> I did get some stuff in the debug logs.
>>
>>
>>
>> msg: 'CLIENT IP ADDRESS: 170.235.1.248'
>>
>> escaped msg: 'CLIENT IP ADDRESS: 170.235.1.248'
>
> As Rainer says, there is a lot of other stuff in that log message (the
> debug format message is 10 lines of output for every log message it
> processes), we need to see the entire message.
>
> If the message is being relayed by some other system, it may not have the
> fromhost-ip that you are expecting. The debug format log messages will
> show you all the details.
>
> David Lang

schooldistrict_phila.tar.gz
Description: application/compressed-tar
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

David ,

In case you wanted to see the debuglog and rsyslog.conf and
/var/log/messages.
None of it is very big so you won't have to parse through a ton of stuff.

We push these logs to two place at the moment.

Graylog and rsyslog server.

We are attempting to deprecate the rsyslog server for the fancy outputs
from Graylog.





===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 


-Original Message-
From: David Lang <da...@lang.hm> 
Sent: Thursday, April 26, 2018 4:29 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: Rainer Gerhards <rgerha...@hq.adiscon.com>; rsyslog-users
<rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] excluding ip addresses

On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> I tried this as well.
>
> This is version 8.24 also.
>
>
>
> -/etc/rsyslog.conf
>
> # Use traditional timestamp format
>
> #
>
> # DeBugging
>
> /var/log/debuglog;RSYSLOG_DebugFormat
>
> #
>
> :msg, contains, "170.235.1.248" ~
>
> :msg, contains, "170.235.1.249" ~
>
> #
>
>
>
>
>
>
>
> I did get some stuff in the debug logs.
>
>
>
> msg: 'CLIENT IP ADDRESS: 170.235.1.248'
>
> escaped msg: 'CLIENT IP ADDRESS: 170.235.1.248'

As Rainer says, there is a lot of other stuff in that log message (the
debug format message is 10 lines of output for every log message it
processes), we need to see the entire message.

If the message is being relayed by some other system, it may not have the
fromhost-ip that you are expecting. The debug format log messages will
show you all the details.

David Lang


rsyslog_sdp.tar.gz
Description: Binary data
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

David,

Would it help to attach the rsyslog.conf and.or the debuglog?


===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 

-Original Message-
From: David Lang <da...@lang.hm> 
Sent: Thursday, April 26, 2018 4:29 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: Rainer Gerhards <rgerha...@hq.adiscon.com>; rsyslog-users
<rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] excluding ip addresses

On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> I tried this as well.
>
> This is version 8.24 also.
>
>
>
> -/etc/rsyslog.conf
>
> # Use traditional timestamp format
>
> #
>
> # DeBugging
>
> /var/log/debuglog;RSYSLOG_DebugFormat
>
> #
>
> :msg, contains, "170.235.1.248" ~
>
> :msg, contains, "170.235.1.249" ~
>
> #
>
>
>
>
>
>
>
> I did get some stuff in the debug logs.
>
>
>
> msg: 'CLIENT IP ADDRESS: 170.235.1.248'
>
> escaped msg: 'CLIENT IP ADDRESS: 170.235.1.248'

As Rainer says, there is a lot of other stuff in that log message (the
debug format message is 10 lines of output for every log message it
processes), we need to see the entire message.

If the message is being relayed by some other system, it may not have the
fromhost-ip that you are expecting. The debug format log messages will
show you all the details.

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

Interesting ..



Thanks



I tried this as well.

This is version 8.24 also.



-/etc/rsyslog.conf

# Use traditional timestamp format

#

# DeBugging

/var/log/debuglog;RSYSLOG_DebugFormat

#

:msg, contains, "170.235.1.248" ~

:msg, contains, "170.235.1.249" ~

#







I did get some stuff in the debug logs.



msg: 'CLIENT IP ADDRESS: 170.235.1.248'

escaped msg: 'CLIENT IP ADDRESS: 170.235.1.248'









===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571

From: Rainer Gerhards <rgerha...@hq.adiscon.com>
Sent: Thursday, April 26, 2018 3:21 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: rsyslog-users <rsyslog@lists.adiscon.com>; David Lang <da...@lang.hm>
Subject: Re: [rsyslog] excluding ip addresses



Sorry to say that, but then it must either really be some other app - or old 
content. Nothing else is possible (that's why I wanted it in the first 
line). Maybe David has some more ideas, but from the developer perspective, 
I don't see anything else that could happen.



Rainer



Sent from phone, thus brief.



Cheltenham, Chris <ccheltenham-...@philasd.org 
<mailto:ccheltenham-...@philasd.org> > schrieb am Do., 26. Apr. 2018, 21:15:

Yes sir,



Here is the top of the rsyslog.conf file.





# Use traditional timestamp format

# DeBugging

#

/var/log/debuglog;RSYSLOG_DebugFormat

#

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat



# Provides kernel logging support (previously done by rklogd)

#$ModLoad imklog

module(load="imklog")



# Provides support for local system logging (e.g. via logger command)

#$ModLoad imuxsock

module(load="imuxsock")



# imput file

#$ModLoad imfile

module(load="imfile")



 4.1.6 of Nessus scan

#$ModLoad imtcp.so

module(load="imtcp.so")

$InputTCPServerRun 514



if $fromhost-ip == '170.235.1.248' then STOP

&~

if $fromhost-ip == '170.235.1.249' then STOP

&~





===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571

From: Rainer Gerhards <rgerha...@hq.adiscon.com 
<mailto:rgerha...@hq.adiscon.com> >
Sent: Thursday, April 26, 2018 3:12 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org 
<mailto:ccheltenham-...@philasd.org> >
Cc: rsyslog-users <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> >; David Lang <da...@lang.hm 
<mailto:da...@lang.hm> >
Subject: Re: [rsyslog] excluding ip addresses



Did you place it in the first line? If so, it records all messages rsyslog 
receives. So if some are in other logs but not this one, someone else is 
writing the other logs.



Rainer

Sent from phone, thus brief.



Cheltenham, Chris <ccheltenham-...@philasd.org 
<mailto:ccheltenham-...@philasd.org> > schrieb am Do., 26. Apr. 2018, 21:04:

Gentlemen,



The log says nothing about those two IP Address.





[root@devsso03 cas]# cd /var/log

[root@devsso03 log]# cat debuglog | grep 249

[root@devsso03 log]# pwd

/var/log

[root@devsso03 log]# cat debuglog | grep 249

[root@devsso03 log]# cat debuglog | grep 248





Yes it is still chattering away in my application logs.



[root@devsso03 cas]# cat cas.log | grep 248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248







Is just not working.



Any other suggestions?





===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571

From: Rainer Gerhards <rgerha...@hq.adiscon.com 
<mailto:rgerha...@hq.adiscon.com> >
Sent: Thursday, April 26, 2018 2:51 PM
To: rsyslog-users <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> >
Cc: David Lang <da...@lang.hm <mailto:da...@lang.hm> >; Cheltenham, Chris 
<ccheltenham-...@philasd.org <mailto:ccheltenham-...@philasd.org> >
Subject: Re: [rsyslog] excluding ip addresses



Place



/var/log/debuglog;RSYSLOG_DebugFormat



And *only* this in the *first* line of rsyslog.conf.



Rainer

Sent from phone, thus brief.



Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> > schrieb am Do., 26. Apr. 2018, 20:48:

David,

I have this is rsyslog.conf

if $fromhost-ip == '170.235.1.248' then
/var/log/debuglog;RSYSLOG_DebugFormat
&~
if $fromhost-ip == '170

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

Yes sir,



Here is the top of the rsyslog.conf file.





# Use traditional timestamp format

# DeBugging

#

/var/log/debuglog;RSYSLOG_DebugFormat

#

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat



# Provides kernel logging support (previously done by rklogd)

#$ModLoad imklog

module(load="imklog")



# Provides support for local system logging (e.g. via logger command)

#$ModLoad imuxsock

module(load="imuxsock")



# imput file

#$ModLoad imfile

module(load="imfile")



 4.1.6 of Nessus scan

#$ModLoad imtcp.so

module(load="imtcp.so")

$InputTCPServerRun 514



if $fromhost-ip == '170.235.1.248' then STOP

&~

if $fromhost-ip == '170.235.1.249' then STOP

&~





===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571

From: Rainer Gerhards <rgerha...@hq.adiscon.com>
Sent: Thursday, April 26, 2018 3:12 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: rsyslog-users <rsyslog@lists.adiscon.com>; David Lang <da...@lang.hm>
Subject: Re: [rsyslog] excluding ip addresses



Did you place it in the first line? If so, it records all messages rsyslog 
receives. So if some are in other logs but not this one, someone else is 
writing the other logs.



Rainer

Sent from phone, thus brief.



Cheltenham, Chris <ccheltenham-...@philasd.org 
<mailto:ccheltenham-...@philasd.org> > schrieb am Do., 26. Apr. 2018, 21:04:

Gentlemen,



The log says nothing about those two IP Address.





[root@devsso03 cas]# cd /var/log

[root@devsso03 log]# cat debuglog | grep 249

[root@devsso03 log]# pwd

/var/log

[root@devsso03 log]# cat debuglog | grep 249

[root@devsso03 log]# cat debuglog | grep 248





Yes it is still chattering away in my application logs.



[root@devsso03 cas]# cat cas.log | grep 248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248







Is just not working.



Any other suggestions?





===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571

From: Rainer Gerhards <rgerha...@hq.adiscon.com 
<mailto:rgerha...@hq.adiscon.com> >
Sent: Thursday, April 26, 2018 2:51 PM
To: rsyslog-users <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> >
Cc: David Lang <da...@lang.hm <mailto:da...@lang.hm> >; Cheltenham, Chris 
<ccheltenham-...@philasd.org <mailto:ccheltenham-...@philasd.org> >
Subject: Re: [rsyslog] excluding ip addresses



Place



/var/log/debuglog;RSYSLOG_DebugFormat



And *only* this in the *first* line of rsyslog.conf.



Rainer

Sent from phone, thus brief.



Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> > schrieb am Do., 26. Apr. 2018, 20:48:

David,

I have this is rsyslog.conf

if $fromhost-ip == '170.235.1.248' then
/var/log/debuglog;RSYSLOG_DebugFormat
&~
if $fromhost-ip == '170.235.1.249' then
/var/log/debuglog;RSYSLOG_DebugFormat
&~


Nothing happens.


===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571


-Original Message-
From: David Lang <da...@lang.hm <mailto:da...@lang.hm> >
Sent: Thursday, April 26, 2018 2:23 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org 
<mailto:ccheltenham-...@philasd.org> >
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> >
Subject: RE: [rsyslog] excluding ip addresses

you don't need to run in debug mode, just write a file using that template

/var/log/debuglog;RSYSLOG_DebugFormat

will write all logs this way.


On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> Date: Thu, 26 Apr 2018 14:08:10 -0400 (EDT)
> From: "Cheltenham, Chris" <ccheltenham-...@philasd.org 
> <mailto:ccheltenham-...@philasd.org> >
> To: David Lang <da...@lang.hm <mailto:da...@lang.hm> >
> Cc: "Cheltenham, Chris via rsyslog" <rsyslog@lists.adiscon.com 
> <mailto:rsyslog@lists.adiscon.com> >
> Subject: RE: [rsyslog] excluding ip addresses
>
> David,
>
> How do I run in debug mode?
> Is I rsyslog -d ?
>
> I am using CentOS 7 so it would be changed in systemd.
>
>
> ===
>
> Thank You;
>
>

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

Gentlemen,



The log says nothing about those two IP Address.





[root@devsso03 cas]# cd /var/log

[root@devsso03 log]# cat debuglog | grep 249

[root@devsso03 log]# pwd

/var/log

[root@devsso03 log]# cat debuglog | grep 249

[root@devsso03 log]# cat debuglog | grep 248





Yes it is still chattering away in my application logs.



[root@devsso03 cas]# cat cas.log | grep 248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248

CLIENT IP ADDRESS: 170.235.1.248







Is just not working.



Any other suggestions?





===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571

From: Rainer Gerhards <rgerha...@hq.adiscon.com>
Sent: Thursday, April 26, 2018 2:51 PM
To: rsyslog-users <rsyslog@lists.adiscon.com>
Cc: David Lang <da...@lang.hm>; Cheltenham, Chris 
<ccheltenham-...@philasd.org>
Subject: Re: [rsyslog] excluding ip addresses



Place



/var/log/debuglog;RSYSLOG_DebugFormat



And *only* this in the *first* line of rsyslog.conf.



Rainer

Sent from phone, thus brief.



Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> > schrieb am Do., 26. Apr. 2018, 20:48:

David,

I have this is rsyslog.conf

if $fromhost-ip == '170.235.1.248' then
/var/log/debuglog;RSYSLOG_DebugFormat
&~
if $fromhost-ip == '170.235.1.249' then
/var/log/debuglog;RSYSLOG_DebugFormat
&~


Nothing happens.


===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571


-Original Message-
From: David Lang <da...@lang.hm <mailto:da...@lang.hm> >
Sent: Thursday, April 26, 2018 2:23 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org 
<mailto:ccheltenham-...@philasd.org> >
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> >
Subject: RE: [rsyslog] excluding ip addresses

you don't need to run in debug mode, just write a file using that template

/var/log/debuglog;RSYSLOG_DebugFormat

will write all logs this way.


On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> Date: Thu, 26 Apr 2018 14:08:10 -0400 (EDT)
> From: "Cheltenham, Chris" <ccheltenham-...@philasd.org 
> <mailto:ccheltenham-...@philasd.org> >
> To: David Lang <da...@lang.hm <mailto:da...@lang.hm> >
> Cc: "Cheltenham, Chris via rsyslog" <rsyslog@lists.adiscon.com 
> <mailto:rsyslog@lists.adiscon.com> >
> Subject: RE: [rsyslog] excluding ip addresses
>
> David,
>
> How do I run in debug mode?
> Is I rsyslog -d ?
>
> I am using CentOS 7 so it would be changed in systemd.
>
>
> ===
>
> Thank You;
>
> Chris Cheltenham
> Technology Services
> The School District of Philadelphia
>
> Work # 215-400-5025
> Cell # 215-301-6571
>
> -Original Message-
> From: David Lang <da...@lang.hm <mailto:da...@lang.hm> >
> Sent: Thursday, April 26, 2018 2:05 PM
> To: Cheltenham, Chris <ccheltenham-...@philasd.org 
> <mailto:ccheltenham-...@philasd.org> >
> Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com 
> <mailto:rsyslog@lists.adiscon.com> >
> Subject: RE: [rsyslog] excluding ip addresses
>
> On Thu, 26 Apr 2018, Cheltenham, Chris wrote:
>
>> David,
>>
>> Thanks for the reply.
>>
>> I used this
>>
>> if $fromhost-ip == '170.235.1.248' then stop if $fromhost-ip ==
>> '170.235.1.249' then stop
>>
>>
>> but it did not work.
>> Is that the correct syntax?
>
> that works
>
> can you log using the format RSYSLOG_DebugFormat and double check that
> fromhost-ip is being set the way you expect it to be?
>
>> I als restarted rsyslog.
>
> yes, that is needed any time you change the config file.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

AH ha, thank you very much.



So now I have stuff in debuglog.



Thank You very much.







===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571

From: Rainer Gerhards <rgerha...@hq.adiscon.com>
Sent: Thursday, April 26, 2018 2:51 PM
To: rsyslog-users <rsyslog@lists.adiscon.com>
Cc: David Lang <da...@lang.hm>; Cheltenham, Chris 
<ccheltenham-...@philasd.org>
Subject: Re: [rsyslog] excluding ip addresses



Place



/var/log/debuglog;RSYSLOG_DebugFormat



And *only* this in the *first* line of rsyslog.conf.



Rainer

Sent from phone, thus brief.



Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> > schrieb am Do., 26. Apr. 2018, 20:48:

David,

I have this is rsyslog.conf

if $fromhost-ip == '170.235.1.248' then
/var/log/debuglog;RSYSLOG_DebugFormat
&~
if $fromhost-ip == '170.235.1.249' then
/var/log/debuglog;RSYSLOG_DebugFormat
&~


Nothing happens.


===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571


-Original Message-
From: David Lang <da...@lang.hm <mailto:da...@lang.hm> >
Sent: Thursday, April 26, 2018 2:23 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org 
<mailto:ccheltenham-...@philasd.org> >
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com 
<mailto:rsyslog@lists.adiscon.com> >
Subject: RE: [rsyslog] excluding ip addresses

you don't need to run in debug mode, just write a file using that template

/var/log/debuglog;RSYSLOG_DebugFormat

will write all logs this way.


On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> Date: Thu, 26 Apr 2018 14:08:10 -0400 (EDT)
> From: "Cheltenham, Chris" <ccheltenham-...@philasd.org 
> <mailto:ccheltenham-...@philasd.org> >
> To: David Lang <da...@lang.hm <mailto:da...@lang.hm> >
> Cc: "Cheltenham, Chris via rsyslog" <rsyslog@lists.adiscon.com 
> <mailto:rsyslog@lists.adiscon.com> >
> Subject: RE: [rsyslog] excluding ip addresses
>
> David,
>
> How do I run in debug mode?
> Is I rsyslog -d ?
>
> I am using CentOS 7 so it would be changed in systemd.
>
>
> ===
>
> Thank You;
>
> Chris Cheltenham
> Technology Services
> The School District of Philadelphia
>
> Work # 215-400-5025
> Cell # 215-301-6571
>
> -Original Message-
> From: David Lang <da...@lang.hm <mailto:da...@lang.hm> >
> Sent: Thursday, April 26, 2018 2:05 PM
> To: Cheltenham, Chris <ccheltenham-...@philasd.org 
> <mailto:ccheltenham-...@philasd.org> >
> Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com 
> <mailto:rsyslog@lists.adiscon.com> >
> Subject: RE: [rsyslog] excluding ip addresses
>
> On Thu, 26 Apr 2018, Cheltenham, Chris wrote:
>
>> David,
>>
>> Thanks for the reply.
>>
>> I used this
>>
>> if $fromhost-ip == '170.235.1.248' then stop if $fromhost-ip ==
>> '170.235.1.249' then stop
>>
>>
>> but it did not work.
>> Is that the correct syntax?
>
> that works
>
> can you log using the format RSYSLOG_DebugFormat and double check that
> fromhost-ip is being set the way you expect it to be?
>
>> I als restarted rsyslog.
>
> yes, that is needed any time you change the config file.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

David,

I have this is rsyslog.conf

if $fromhost-ip == '170.235.1.248' then
/var/log/debuglog;RSYSLOG_DebugFormat
&~
if $fromhost-ip == '170.235.1.249' then
/var/log/debuglog;RSYSLOG_DebugFormat
&~


Nothing happens.


===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 


-Original Message-
From: David Lang <da...@lang.hm> 
Sent: Thursday, April 26, 2018 2:23 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] excluding ip addresses

you don't need to run in debug mode, just write a file using that template

/var/log/debuglog;RSYSLOG_DebugFormat

will write all logs this way.


On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> Date: Thu, 26 Apr 2018 14:08:10 -0400 (EDT)
> From: "Cheltenham, Chris" <ccheltenham-...@philasd.org>
> To: David Lang <da...@lang.hm>
> Cc: "Cheltenham, Chris via rsyslog" <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] excluding ip addresses
> 
> David,
>
> How do I run in debug mode?
> Is I rsyslog -d ?
>
> I am using CentOS 7 so it would be changed in systemd.
>
>
> ===
>
> Thank You;
>
> Chris Cheltenham
> Technology Services
> The School District of Philadelphia
>
> Work # 215-400-5025
> Cell # 215-301-6571
>
> -Original Message-
> From: David Lang <da...@lang.hm>
> Sent: Thursday, April 26, 2018 2:05 PM
> To: Cheltenham, Chris <ccheltenham-...@philasd.org>
> Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] excluding ip addresses
>
> On Thu, 26 Apr 2018, Cheltenham, Chris wrote:
>
>> David,
>>
>> Thanks for the reply.
>>
>> I used this
>>
>> if $fromhost-ip == '170.235.1.248' then stop if $fromhost-ip == 
>> '170.235.1.249' then stop
>>
>>
>> but it did not work.
>> Is that the correct syntax?
>
> that works
>
> can you log using the format RSYSLOG_DebugFormat and double check that 
> fromhost-ip is being set the way you expect it to be?
>
>> I als restarted rsyslog.
>
> yes, that is needed any time you change the config file.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

OK so output to that string in instead of a STOP?

 I.E.
if $fromhost-ip == '170.235.1.248' then
/var/log/debuglog;RSYSLOG_DebugFormat


===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 


-Original Message-
From: David Lang <da...@lang.hm> 
Sent: Thursday, April 26, 2018 2:23 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] excluding ip addresses

you don't need to run in debug mode, just write a file using that template

/var/log/debuglog;RSYSLOG_DebugFormat

will write all logs this way.


On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> Date: Thu, 26 Apr 2018 14:08:10 -0400 (EDT)
> From: "Cheltenham, Chris" <ccheltenham-...@philasd.org>
> To: David Lang <da...@lang.hm>
> Cc: "Cheltenham, Chris via rsyslog" <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] excluding ip addresses
> 
> David,
>
> How do I run in debug mode?
> Is I rsyslog -d ?
>
> I am using CentOS 7 so it would be changed in systemd.
>
>
> ===
>
> Thank You;
>
> Chris Cheltenham
> Technology Services
> The School District of Philadelphia
>
> Work # 215-400-5025
> Cell # 215-301-6571
>
> -Original Message-
> From: David Lang <da...@lang.hm>
> Sent: Thursday, April 26, 2018 2:05 PM
> To: Cheltenham, Chris <ccheltenham-...@philasd.org>
> Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] excluding ip addresses
>
> On Thu, 26 Apr 2018, Cheltenham, Chris wrote:
>
>> David,
>>
>> Thanks for the reply.
>>
>> I used this
>>
>> if $fromhost-ip == '170.235.1.248' then stop if $fromhost-ip == 
>> '170.235.1.249' then stop
>>
>>
>> but it did not work.
>> Is that the correct syntax?
>
> that works
>
> can you log using the format RSYSLOG_DebugFormat and double check that 
> fromhost-ip is being set the way you expect it to be?
>
>> I als restarted rsyslog.
>
> yes, that is needed any time you change the config file.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

David,

I found the service I suppose I just add -d to the Exec Start line?

[Service]
Type=notify
EnvironmentFile=-/etc/sysconfig/rsyslog
ExecStart=/usr/sbin/rsyslogd -n $SYSLOGD_OPTIONS
Restart=on-failure
UMask=0066
StandardOutput=null
Restart=on-failure


===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 


-Original Message-
From: David Lang <da...@lang.hm> 
Sent: Thursday, April 26, 2018 2:05 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] excluding ip addresses

On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> David,
>
> Thanks for the reply.
>
> I used this
>
> if $fromhost-ip == '170.235.1.248' then stop if $fromhost-ip == 
> '170.235.1.249' then stop
>
>
> but it did not work.
> Is that the correct syntax?

that works

can you log using the format RSYSLOG_DebugFormat and double check that
fromhost-ip is being set the way you expect it to be?

> I als restarted rsyslog.

yes, that is needed any time you change the config file.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

David,

How do I run in debug mode?
Is I rsyslog -d ?

I am using CentOS 7 so it would be changed in systemd.
 

===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 

-Original Message-
From: David Lang <da...@lang.hm> 
Sent: Thursday, April 26, 2018 2:05 PM
To: Cheltenham, Chris <ccheltenham-...@philasd.org>
Cc: Cheltenham, Chris via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] excluding ip addresses

On Thu, 26 Apr 2018, Cheltenham, Chris wrote:

> David,
>
> Thanks for the reply.
>
> I used this
>
> if $fromhost-ip == '170.235.1.248' then stop if $fromhost-ip == 
> '170.235.1.249' then stop
>
>
> but it did not work.
> Is that the correct syntax?

that works

can you log using the format RSYSLOG_DebugFormat and double check that
fromhost-ip is being set the way you expect it to be?

> I als restarted rsyslog.

yes, that is needed any time you change the config file.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Excluding IP addresses

[Cheltenham, Chris via rsyslog]

Hello,

 

I am trying to exclude chattiness from my logs.

 

I am not able to get it to work.

 

We have tried -

if $fromhost-ip == '170.235.1.248' then stop

if $fromhost-ip == '170.235.1.249' then stop

 

 

and this -

if $fromhost-ip=='172.16.111.222' then
/dev/null/%FROMHOST-IP%/%syslogfacility-text%.log

 

I also have tried this 

if $fromhost-ip=='172.16.111.222' then /dev/null

 

To no avail.

 

 

Does anyone know what I am doing wrong ?

This is coming from a load balancer.

 

 

 

 

===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Facing issues with rsyslog configuration

[Chris via rsyslog]

Have you tried netstat -n | grep 514 just to make sure netstat is
outputting numbers insted of service names?Worth a look.

On Wed, Apr 25, 2018, at 10:00 PM, eswar472 via rsyslog wrote:
>  Hi,
>
> I am trying to test remote logging between two ubuntu machines. In the> 
> ubuntu machine which i wanted to make it as server, i changed the
> /etc/rsyslog.conf as below. After that i restarted service with "sudo> 
> service rsyslog restart" then checked netstat. I dont see 514 port
> is open.> As per some suggestions in internet i tried with ports 10514 and
> 20514, but> no luck.
>
> Commands I executed after changing configuration file
>
> rreddy@rreddy-node2:~$ sudo service rsyslog restart
> rsyslog stop/waiting
> rsyslog start/running
> rreddy@rreddy-node2:~$ netstat | grep 514
> unix  3  [ ] STREAM CONNECTED 30472
> @/tmp/.ICE-unix/25149
> unix  3  [ ] STREAM CONNECTED 73514
> unix  3  [ ] STREAM CONNECTED 23293
> @/tmp/.ICE-unix/25149
>
>
> Below is the content of my /etc/rsyslog file
>
> #  /etc/rsyslog.confConfiguration file for rsyslog.
> #
> #   For more information see
> #   /usr/share/doc/rsyslog-
> #   doc/html/rsyslog_conf.html> #
> #  Default logging rules can be found in /etc/rsyslog.d/50-
> #  default.conf>
>
> #
>  MODULES 
> #
>
> $ModLoad imuxsock # provides support for local system logging
> $ModLoad imklog   # provides kernel logging support
> #$ModLoad immark  # provides --MARK-- message capability
>
> # provides UDP syslog reception
> $ModLoad imudp
> $UDPServerRun 514
>
> # provides TCP syslog reception
> $ModLoad imtcp
> $InputTCPServerRun 514
>
> $AllowedSender TCP, 127.0.0.1, 10.22.42.115
> $template Incoming-logs,"/var/log/test.log"
>
> ###
>  GLOBAL DIRECTIVES 
> ###
>
> #
> # Use traditional timestamp format.
> # To enable high precision timestamps, comment out the following line.> #
> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
>
> # Filter duplicated messages
> $RepeatedMsgReduction on
>
> #
> # Set the default permissions for all log files.
> #
> $FileOwner syslog
> $FileGroup adm
> $FileCreateMode 0640
> $DirCreateMode 0755
> $Umask 0022
> $PrivDropToUser syslog
> $PrivDropToGroup syslog
>
> #
> # Where to place spool and state files
> #
> $WorkDirectory /var/spool/rsyslog
>
> #
> # Include all config files in /etc/rsyslog.d/
> #
> $IncludeConfig /etc/rsyslog.d/*.conf
>
>
> Can you help me in finding what is wrong with this configuration.
>
> Thank you,
> Eshwar
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by
> a myriad> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if 
> you> DON'T LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] excluding ip addresses

[Cheltenham, Chris via rsyslog]

Hello,

 

I am using 8.24 in CentOS 7.

 

How do I exclude ip addresses from being logged?

 

I tried this ..

 

#Ignore the 170.235.1.248 and 170.235.1.249 A 10 Load balancer health
checks

if $fromhost-ip=='170.235.1.248' then
/dev/null/%FROMHOST-IP%/%syslogfacility-text%.log-I-I/%FROMHOST

if $fromhost-ip=='170.235.1.249' then
/dev/null/%FROMHOST-IP%/%syslogfacility-text%.log-I-I/%FROMHOST

 

and

 

#Ignore the 170.235.1.248 and 170.235.1.249 A 10 Load balancer health
checks

if $fromhost-ip=='170.235.1.248' then /dev/null/

if $fromhost-ip=='170.235.1.249' then /dev/null/

 

 

to no avail.

 

These are load balancer health checks clogging up my logs.

 

 

 

===

Thank You;

Chris Cheltenham
Technology Services
The School District of Philadelphia

Work # 215-400-5025
Cell # 215-301-6571 

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] TCP Stops Local Logging

[Chris]

On Fri, Mar 24, 2017, at 09:50 PM, David Lang wrote:
> On Fri, 24 Mar 2017, Chris wrote:
> 
> > On RH 6 systems running rsyslog 5.8.10 we noticed that if we setup a
> > client system to use TCP to log to a remote server:
> > *.*   @@192.168.1.2
> >
> > If the remote log server is not reachable for some reason no logging
> > takes place, not even local logging to the local system log files.
> > When the log server is available and rsyslog is restarted  both local
> > logging and remote logging work.  Is this a known issue or is there some
> > way to ensure that local logging still occurs when  the TCP remote
> > server is down?
> 
> This is working as designed (for the config you specified), if a message
> cannot 
> be delivered to one destination, and you don't have rsyslog configured to
> throw 
> it away, it is not able to finish processing that log message and start
> work on 
> the next one.
> 
> You can create an action queue for the delivery to a remote system, and
> until 
> that queue fills up, other log processing will continue.
> 
> You really should move to at least v7, if not v8, a lot of things have
> changed, 
> especially the available syntax for specifying queues.
> 
> David Lang

Thank you for the response.  Unfortunately we need to stay with the
version that came with the OS for now.  Internal compliance
requirements.  

What I'd like to do is setup all the clients to log to both  the remote
server (TCP) and the local logs.  When the remote TCP server is not
available, I want it to continue to log to the local logs.   Pretty new
to more advanced rsyslog configurations, we've always just done the
basic.

Can you point me in the right direction on how to go about this?  
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] TCP Stops Local Logging

[Chris]

On RH 6 systems running rsyslog 5.8.10 we noticed that if we setup a
client system to use TCP to log to a remote server:
*.*   @@192.168.1.2

If the remote log server is not reachable for some reason no logging
takes place, not even local logging to the local system log files.  
When the log server is available and rsyslog is restarted  both local
logging and remote logging work.  Is this a known issue or is there some
way to ensure that local logging still occurs when  the TCP remote
server is down?
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Forwarding Events

[Chris Schafer]

The UDPSpoof module, and filter conditions are your friends. I can't help
write your filter conditions if I don't know what kind of event you're
looking for though.

Also, what version of rsyslog are you running?

On Fri, Aug 12, 2016 at 10:52 AM William Ryals  wrote:

> > Question,
> >
> > I have the need to capture only a certain "heartbeat" event coming into
> my says log farm and forward to another remote server. I get billions of
> events daily and this is a small amount. I need to maintain the source
> ip/hostname when forwarding the events so the remote server will think the
> events came from the original source. I know how to send all, but getting
> it to send only specific ones that match a regex is not happening. I am
> sure this is a simple task and I am overthinking it.
> >
> > Thanks in advance!
> >
> > Bill
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Why don't we trim msg per default?

[Chris Schafer]

I can confirm this - I also would have parsers that'd break if we did this.
If we're going to start using auto trim, I'd prefer we initially start with
a second token ($msgTrim or something) to ease the migration.

On Sun, Jan 24, 2016 at 2:36 PM David Lang  wrote:

> On Sun, 24 Jan 2016, Thomas D. wrote:
>
> > Hi,
> >
> > today I converted a configuration into the modern syntax and run into
> > the problem that most msg values seems to start with a space character
> > which I didn't covered in my "startwith" value:
> >
> >> Debug line with all properties:
> >> FROMHOST: 'srv42', fromhost-ip: '127.0.0.1', HOSTNAME: 'srv42', PRI: 22,
> >> syslogtag 'dovecot:', programname: 'dovecot', APP-NAME: 'dovecot',
> PROCID: '-', MSGID: '-',
> >> TIMESTAMP: 'Jan 24 21:18:17', STRUCTURED-DATA: '-',
> >> msg: ' pop3-login: Disconnected (tried to use disallowed plaintext
> auth): user=<>, rip=1.2.3.4, lip=9.8.7.6'
> >> escaped msg: ' pop3-login: Disconnected (tried to use disallowed
> plaintext auth): user=<>, rip=1.2.3.4, lip=9.8.7.6'
> >> inputname: imuxsock rawmsg: '<22>Jan 24 21:18:17 dovecot: pop3-login:
> Disconnected (tried to use disallowed plaintext auth): user=<>,
> rip=1.2.3.4, lip=9.8.7.6'
> >> $!:
> >> $.:
> >> $/:
> >
> > Is this normal? Why don't we trim messages per default (we still have
> > rawmsg for people who don't like that)?
>
> because the standard doesn't require such a space, and parsers written
> over the
> years include the space in them, so changing this will break lots of
> existing
> stuff.
>
> David Lang
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] liblognorm tokenize issue

[Chris Schafer]

I wrote string-to and completely believe it should be optimized. I am not
great with C, I just needed the function at the time.
On Mon, Jun 1, 2015 at 9:00 PM David Lang da...@lang.hm wrote:

 On Tue, 2 Jun 2015, singh.janmejay wrote:

  Also, you probably want to add some tests.

 I couldn;t find any existing tests for these parsers, and couldn't figure
 out
 how to run the testbench from liblognorm. What did I miss?

 David Lang


  On Tue, Jun 2, 2015 at 2:56 AM, David Lang da...@lang.hm wrote:
  On Fri, 29 May 2015, David Lang wrote:
 
  attached is a patch that lets you specify multiple characters for
 char-to
  and char-sep, any one of the characters will work, so with the example
 above
 
  rule=:%foo:tokenize::char-sep: % c
 
  # echo 'ab c' |./lognormalizer -r del -e json
 
  you get
 
  { foo: [ a, b ] }
 
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] liblognorm tokenize issue

[Chris Schafer]

I'm currently being affected by this same issue, and would love to see a
resolution!
Either this (having tokenized auto-separate the strings) or being able to
specify multiple characters the char-to would stop at (so stop at the first
occurance of  OR | with char-to:|)

On Fri, May 29, 2015 at 10:26 AM David Lang da...@lang.hm wrote:

 this doesn't solve the problem because it can only return a string.

 I want to tokenize and then use something more complex (json,
 name-value-list,
 iptables, cef, etc)

 David Lang

 On Fri, 29 May 2015, singh.janmejay wrote:

  Should we have an optional argument in word: except.
 
  Eg.
 
  %foo:word:%%bar:word%
 
  Given bazquux will give us:
 
  {foo : baz, bar: quux}
 
  If we take multiple chars (allow escaped unicode sequences), we can
  say the default value of this field is 'space' and 'tab'.
 
  On Fri, May 29, 2015 at 1:38 AM, David Lang da...@lang.hm wrote:
  I think that the config
 
  rule=:%foo:tokenized::word%
 
  against the string
 
  123
 
  should return
  { foo: [ 1, 2, 3 ] }
 
  but instead it returns
 
  { foo: [ 123 ] }
 
  because 'word' is applied before the split of tokenized.
 
  If I change 'word' to 'number' it performs as expected (returning three
  values)
 
  this can be worked around by doing
 
  rule=:%foo:tokenized::char-sep:%
 
  but this is ugly, and it prevents doing anything smarter (such as a
 descent
  or recursive that would be able to split a name-value pair)
 
  https://github.com/rsyslog/liblognorm/issues/64
 
  filed for this.
 
  David Lang
 
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards
  NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
 myriad of
  sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T
  LIKE THAT.
 
 
 
 
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] mmnormalize thoughts

[Chris Schafer]

David,
As far as docs go, when i went into documentation for liblognorm.com, i
found
http://www.liblognorm.com/files/manual/index.html

Which includes string-to. That said, I know it's there because I put the
function in, and if you have a suggestion as to better document the
functions, that could lead to a wider acceptance of libnorm.
t

On Thu, Mar 12, 2015 at 1:36 AM singh.janmejay singh.janme...@gmail.com
wrote:

 It never goes back up because if any other rule was going to match the
 current line, it would be a subtree of the current node (this is an
 invariant).

 It does try all sub-trees from any node before giving up. It first
 tries all field-nodes, then appropriate literal-node.

 In this case anything at the end will be matched by rest, the only
 thing that rest will not match is string with 0 length, which the next
 rule won't match anyway.

 About 0-length suffix, I want to think a bit about how to support it
 with descent. As of now it expects a remaining-text field.

 Im unsure if this answers your question though.

 On Thu, Mar 12, 2015 at 1:05 PM, David Lang da...@lang.hm wrote:
  On Thu, 12 Mar 2015, singh.janmejay wrote:
 
  On Thu, Mar 12, 2015 at 9:19 AM, David Lang da...@lang.hm wrote:
 
  On Thu, 12 Mar 2015, singh.janmejay wrote:
 
  Tried re-ordering it? Put the one with /port first?
 
 
 
  no, lognorm rules are not supposed to be order dependent, so I didn't
 try
  that (especially after finding things failing to parse with rsyslog
 that
  worked manually)
 
 
  In case of input strings being matching-rule-wise disjoint, you are
  right, order won't matter. But when they are not disjoint, order does
  matter, because the first one to match the string wins.
 
  Consider this rulebase:
  rule=:%ip:ipv4%%last:rest%
  rule=:%ip:ipv4%%junk:char-sep:/%/%port:number%
 
  If you write it the way I have above, you'll end up matching first
  rule for input 10.20.30.40/5
 
 
  but when it can't find a match for / and has to undo the match and go
 back
  up the tree, why doesn't it try the next possible match? (repeating as
  needed until it has tried all possible branches of the tree)
 
  David Lang
 
 
  But if you write it this way:
  rule=:%ip:ipv4%%junk:char-sep:/%/%port:number%
  rule=:%ip:ipv4%%last:rest%
 
  You'll end up matching the first one.
 
  I know it appears order independent for your original rulebase, but
  that is because fields are always tried first(in preference to
  subtrees hanging off literals), and rest is a field, while '/' creates
  a  litteral-subtree.
 
 
  Yes, rest must get atleast one char to succeed. I'll create some new
  tests without rest-capture (and see what fails).
 
 
 
  Ok, this can be worked around (but it's a bit ugly), any reason why
 rest
  has
  to get at least one character?
 
 
  Yep, its annoying, it happens only for last token.
 
  The reason is, parsed-fragment length = input-string is used as a
  termination condition for ln_normalize recursion (see ln_normalizeRec)
  and the last token identified when recursion terminates is not the
  terminal-node, so its not considered a complete match(one that goes
  till leaf of ptree).
 
 
  David Lang
 
 
  On Thu, Mar 12, 2015 at 1:09 AM, David Lang da...@lang.hm wrote:
 
 
  I just upgraded to liblognorm 1.1.1 (unfortunantly I didn't get a
  chance
  to
  compile it myself and test it earlier)
 
  I ran into two problems
 
  first, %last:rest% does not match if there is nothing left on the
 line
 
  i.e. a line that ends with an IP address will not match
  rule=:%ip:ipv4%%last:rest%
 
  secondly, liblognorm is selecting the rule that matches the least
  amount
  of
  the message.
 
  so with these two rules
 
  rule=:%ip:ipv4%%last:rest%
  rule=:%ip:ipv4%/%port:number%%last:rest%
 
 
  I guess the hack I proposed above (using char-sep) can unblock you for
  now, unless you hate its aesthetics too much :-).
 
 
  192.168.1.1/5 will get matched by the first rule, with '/5' in last,
  even
  though the second rule would match it. If I remove the first rule,
 the
  second rule does match and the parse succeeds.
 
  David Lang
 
 
  On Fri, 6 Feb 2015, David Lang wrote:
 
  While I'm working to build packages of this to test with, what
 happens
  if
  you descend into a ruleset like the following
 
  rule=:%ip:ipv4%%last:rest%
  rule=:%ip:ipv4%/%port:number%%last:rest%
 
  will it work to find the match that has the least left in last?
 
  David Lang
 
 
  On Fri, 6 Feb 2015, singh.janmejay wrote:
 
  It's going to be in the coming release, just master build for now.
 
  --
  Regards,
  Janmejay
 
  PS: Please blame the typos in this mail on my phone's uncivilized
  soft
  keyboard sporting it's not-so-smart-assist technology.
 
  On Feb 6, 2015 6:37 AM, David Lang da...@lang.hm wrote:
 
  On Wed, 4 Feb 2015, singh.janmejay wrote:
 
   On Wed, Feb 4, 2015 at 6:22 PM, David Lang da...@lang.hm
 wrote:
 
 
 
 
   On Wed, 4 Feb 2015, singh.janmejay wrote:
 
 
 
 
   On Wed, Feb 4, 2015 at 7:17 AM, 

Re: [rsyslog] Docs fail to compile in liblognorm 1.1.0 (and shameless plug for my pull request)

[Chris Schafer]

Sweet.
To make it easier on you, I synced in my travis things to my main repo
(cherry-picked only the thing that mattered), it's just changing one line
in the travis config really. Will let you know if the docs are failing to
build before you release a version update again.
Chris

On Mon Feb 02 2015 at 11:57:57 PM Rainer Gerhards rgerha...@hq.adiscon.com
wrote:

 2015-02-03 1:49 GMT+01:00 Chris Schafer chrisp.scha...@gmail.com:

  So, liblognorm 1.1.0 fails to compile docs due to a tiny underlining
 error.
  Total bummer, and can be fixed by adding a single # under alpha in
  there...
  There's also the option of fast forwarding to my pull request (#20) which
  would sync everything up, and fix the bug.


 Thanks for the reminder. As usual, I got side-stepped while working on the
 PR. That sometimes happens when I can't finish work when I have time to do
 it. But you'll notice that everything decent (and yours sure is) gets
 merged soon enough before a release (or explicitely pushed back for later
 review if it is quite complex).


  Additionally, you may want to
  look at my travis-test-docs branch, which has travis test the docs
  config/make process as well.
 

 Will try.

 Rainer

  Chris
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards
  NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
  of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
  DON'T LIKE THAT.
 
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Docs fail to compile in liblognorm 1.1.0 (and shameless plug for my pull request)

[Chris Schafer]

Sent a PR.
On Tue, Feb 3, 2015 at 12:55 AM Rainer Gerhards rgerha...@hq.adiscon.com
wrote:

 2015-02-03 9:41 GMT+01:00 Chris Schafer chrisp.scha...@gmail.com:

  Sweet.
  To make it easier on you, I synced in my travis things to my main repo
  (cherry-picked only the thing that mattered), it's just changing one line
  in the travis config really.


 great!

 Would you mind doing the PR? I'd like to get to a PR-based workflow (not
 the least because of travis). I know it's brain-dead in this case, but I
 myself try to get used to it. If you don't like to bother, I'll just cherry
 pick from your repo.

 Rainer


  Will let you know if the docs are failing to
  build before you release a version update again.
  Chris
 
  On Mon Feb 02 2015 at 11:57:57 PM Rainer Gerhards 
  rgerha...@hq.adiscon.com
  wrote:
 
   2015-02-03 1:49 GMT+01:00 Chris Schafer chrisp.scha...@gmail.com:
  
So, liblognorm 1.1.0 fails to compile docs due to a tiny underlining
   error.
Total bummer, and can be fixed by adding a single # under alpha
 in
there...
There's also the option of fast forwarding to my pull request (#20)
  which
would sync everything up, and fix the bug.
  
  
   Thanks for the reminder. As usual, I got side-stepped while working on
  the
   PR. That sometimes happens when I can't finish work when I have time to
  do
   it. But you'll notice that everything decent (and yours sure is) gets
   merged soon enough before a release (or explicitely pushed back for
 later
   review if it is quite complex).
  
  
Additionally, you may want to
look at my travis-test-docs branch, which has travis test the docs
config/make process as well.
   
  
   Will try.
  
   Rainer
  
Chris
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
  myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
 you
DON'T LIKE THAT.
   
   ___
   rsyslog mailing list
   http://lists.adiscon.net/mailman/listinfo/rsyslog
   http://www.rsyslog.com/professional-services/
   What's up with rsyslog? Follow https://twitter.com/rgerhards
   NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
 myriad
   of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
   DON'T LIKE THAT.
  
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards
  NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
  of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
  DON'T LIKE THAT.
 
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Docs fail to compile in liblognorm 1.1.0 (and shameless plug for my pull request)

[Chris Schafer]

So, liblognorm 1.1.0 fails to compile docs due to a tiny underlining error.
Total bummer, and can be fixed by adding a single # under alpha in
there...
There's also the option of fast forwarding to my pull request (#20) which
would sync everything up, and fix the bug. Additionally, you may want to
look at my travis-test-docs branch, which has travis test the docs
config/make process as well.
Chris
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

[Chris Schafer]

I like the nullmarker idea a lot, since that's one of the most common
issue. Also, it solves it pretty efficiently. I think it needs to be in the
rulebase, or liblognorm is tied to being only a part of rsyslog.
Chris

On Tue Jan 27 2015 at 10:27:42 PM singh.janmejay singh.janme...@gmail.com
wrote:

 I see what you are thinking of, but somethings that may be worth thinking
 about before we decide:

 - Does it make sense for users to pack unrelated samples in the same
 rulebase?

   There are 3 problems with this:
  * The tree will become large, and back-tracking several unrelated
 branches will be wasteful (a condition in ruleset which calls the action
 will be much more efficient assuming tests is not very complex)

  * The rulebase will be composed of several unrelated rules, making it
 harder to read

  * Multiple parse-trees may have to be maintained in order to satisfy
 all combinations of nullMarker (eg. a non-leaf field, marked for
 null-handling in one sample, but not marked for it in the other) (so
 matching will become O(n) in number of combinations). So it is some
 dev-work and little bit of perf-overhead.

 - The alternative is to set nullMarker at top level in a rulebase (instead
 of being able to change it for every sample).

   But then the flexibility is slightly lowered.

 - If we go with action level param, its useful in cases where one has
 standard access-log format but load-balancer level always have some fields
 (say upstream latency or upstream-ip) which app-layer access logs will not
 have.

   This can use the same rulebase with nullMarker in one case, and without
 it in another.

 Thoughts?

 On Wed, Jan 28, 2015 at 11:13 AM, David Lang da...@lang.hm wrote:

  I'm thinking that it needs to only apply to part of a ruleset. I can't
 see
  why you would use the same rulebase with different values overall, but I
  can easily see a rulebase that covers more than one type of logs needing
  different values for the different types of logs.
 
  remember that liblognorm is most effictive if it has one ruleset to cover
  everything you are looking at rather than doing other conditionals and
 then
  picking which rulset to use.
 
  David Lang
 
 
  On Wed, 28 Jan 2015, singh.janmejay wrote:
 
   I think action parameter is the most flexible place to have it at.
 Because
  same rulebase can be used with different values.
 
  Either module or rulebase level param will be less flexible compared to
  this.
 
  --
  Regards,
  Janmejay
 
  PS: Please blame the typos in this mail on my phone's uncivilized soft
  keyboard sporting it's not-so-smart-assist technology.
 
  On Jan 28, 2015 10:48 AM, David Lang da...@lang.hm wrote:
 
   On Wed, 28 Jan 2015, singh.janmejay wrote:
 
   Ok, one way I can think of doing it: expose a parameter at
 action/module
 
  level which turns on defaulting and picks a default string.
 
  Eg.
 
  action(type=mmnormalize   nullMarker=-)
 
  Where nullMarker is a string (not a char).
 
  Whenever a - is encountered and a field is expected, it should skip
  the
  key(the key will not be present at all) and continue matching next
 token
  onwards.
 
  Thoughts?
 
 
  This needs to be something in the liblognorm config, not in rsyslog.
  different types of logs would have different nullMarker strings.
 
  with that adjustment, I think it's a good idea.
 
  David Lang
 
   --
 
  Regards,
  Janmejay
 
  PS: Please blame the typos in this mail on my phone's uncivilized soft
  keyboard sporting it's not-so-smart-assist technology.
 
  On Jan 28, 2015 6:38 AM, David Lang da...@lang.hm wrote:
 
   On Wed, 28 Jan 2015, singh.janmejay wrote:
 
 
   May be it'll be useful to discuss what you want to achieve with such
 
   representations of sample. I mean if possible, take a few samples
 from
  your
  existing rulebase which you think highlight the problem(s) you are
  facing.
 
 
   I think the example is the Apache logs, where Apache either puts a
  value,
  or it puts a placeholder '-'
 
  if you want to capture a specific type (number or ip address for
  example),
  you won't match a log entry that has a - in that field.
 
  If there are only a couple fields that are like this, you can list
 all
  the
  combinations in the ruleset, but if you have a lot of fields like
 this,
  the
  combinatorial explosion would make for a LOT of rules.
 
  So I don't think he really needs a generic 'or' allowing any types to
  be
  combined as much as a way to say this field could be this type or
 this
  constant
 
  David Lang
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards
  NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
  myriad
  of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
 you
  DON'T LIKE

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

[Chris Schafer]

@Janmejay:
I'll be honest - I strikedon't/strike didn't know if it'll handle
escape sequences. I didn't test it earlier, just tested it now. Totally
worked, woo!
I did put in documentation - you can check the file. Actually included a
little bugfix on existing documentation that kept it from compiling as well.
I'm not against putting tests in at all, though I didn't because I didn't
see any tests for the non-special functions, only regex and tokenization. I
can throw them in. What I did do is test this against a couple thousand log
lines that I actually needed to parse, just to make sure it worked.

On Mon Jan 26 2015 at 10:01:21 PM Kendall Green kendallar...@gmail.com
wrote:

 I like the 'or' option, precisely for doing type check condition when a
 whole lot of fields exists in records. This is currently cumbersome and
 quickly becomes a daunting mess of a Cartesian Product set rule base for
 all the combination of fields that could have single values unquoted, or
 possibly quoted. Not to mention how this use case caries over to other
 scenarios where an or operation would be invaluable to type casting.

 %tag:type:or:type%
 could be very useful, not just to solve the issue of which behavior should
 be default, as it would be set by the syntax.

 For example, if type quoted-string is set first, then should check without
 quotes up to space.
 Wouldn't the default be for what the type is, so with quoted-string, then
 it's quoted, unless an 'or' condition exists for an alternate expected data
 type.

 With so very many fields in verbose messages, it is great to have a single
 rule which would otherwise be an exponentially lengthy ruleset to
 accommodate all the possible known type setting combinations.
 %Description:quoted-string:or:word%

 An ''type:or:type option could also be useful in other cases where
 unpopulated fields exists with a default type value which doesn't match the
 field when populated with specific typed value.

 %IP Address:ipv4:or:word%
 The IP Address is provided, or a hyphen exists in the field when
 unpopulated. In this scenario more specific literal matching would also be
 nice option, which please correct me if literals already exists beyond
 annotations. Having a char type match as char-sep somewhat resembles, where
 field extraction only when the literal matches. The difference being that
 the literal would be matched for field value not just up to that position.
 To give a more strict rule:
 %IP Address:ipv4:or:char:\x2d%

 Similarly, it would be good to have string type, like described for the
 purposed char type above, but for capturing the string literal instead of
 only the literal char. Rulebase could use string parse
 enhancement with capture of literal string at specific field start
 position within rulebase, since existing features could likely be used like
 annotation fields. Additionally, please inform of any contributions for
 the discussion regarding data type of fields to match string as a
 string-to, as char-to / char-sep feature of char
 separator on string, like the function, field($!path, string-or-char).   So
 please also elaborate on what has already been done for rulebase matching
 string literals. Thanks!

 -Kendall



 On Mon, Jan 26, 2015 at 5:49 PM, David Lang da...@lang.hm wrote:

  I don't like the or option as I think it makes the rules harder to
 read.
  unless you are doing this on a lot of fields in a line, just make a new
  line with the different type.
 
  We need feedback from others, but at the very least I think making this
 an
  option to the standard quoted-string type would be better than a new type
  (the question is if this should be enabled by default or disabled by
  default)
 
 
  David Lang
 
  On Tue, 27 Jan 2015, Chris Schafer wrote:
 
   It comes back as a full fail. I thought about modifying that, but I
 didn't
  want to wreck anything currently in place.
  A coworker of mine had a great idea for an or ability, going
  %tag:or:quoted-string:word% where i attempts the first, and if that
 fails,
  goes to the second. However, that's not going to be easy, and I wanted
 to
  push this change before you guys got too many commits ahead.
 
  On Mon Jan 26 2015 at 4:43:02 PM David Lang da...@lang.hm wrote:
 
   hmm, I'm wondering if we should do this for the normal quoted type? If
  you
  say
  quoted string and there isn't a quote does it just not match?
 
  David Lang
 
  On Tue, 27 Jan 2015, Chris Schafer wrote:
 
   This only handles  because that's what the current quoted string
 does.
  If it doesn't start with , it implements the word functionality
 
  (which I
 
  shamelessly copied). The idea is to capture inputs where the source
 
  system
 
  only quotes it if it contains a space, but leaves it unquoted
 otherwise.
  Example:
  No data = -
  One Word = word
  Two words+ = Two Words
 
  The function should handle all three.
  Chris
 
  On Mon Jan 26 2015 at 4:36:25 PM David Lang da...@lang.hm wrote:
 
   does this handle embedded

[rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

[Chris Schafer]

Just submitted the following pull request:
https://github.com/rsyslog/liblognorm/pull/20
And I believe it could solve a lot of issues (at least, it solves a lot of
mine) surrounding mmnormalize parsing in rsyslog. I'm looking for
comments/issues/holy-crap-you-can't-code-what-are-you-doing, if you guys
have any. This is my first time submitting a patch to a large project (or
at least one where I didn't know the maintainer personally), so be gentle
please :)

Chris
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] New Pull request for liblognorm - additional mmnormalize functionality

[Chris Schafer]

It comes back as a full fail. I thought about modifying that, but I didn't
want to wreck anything currently in place.
A coworker of mine had a great idea for an or ability, going
%tag:or:quoted-string:word% where i attempts the first, and if that fails,
goes to the second. However, that's not going to be easy, and I wanted to
push this change before you guys got too many commits ahead.

On Mon Jan 26 2015 at 4:43:02 PM David Lang da...@lang.hm wrote:

 hmm, I'm wondering if we should do this for the normal quoted type? If you
 say
 quoted string and there isn't a quote does it just not match?

 David Lang

 On Tue, 27 Jan 2015, Chris Schafer wrote:

  This only handles  because that's what the current quoted string does.
  If it doesn't start with , it implements the word functionality
 (which I
  shamelessly copied). The idea is to capture inputs where the source
 system
  only quotes it if it contains a space, but leaves it unquoted otherwise.
  Example:
  No data = -
  One Word = word
  Two words+ = Two Words
 
  The function should handle all three.
  Chris
 
  On Mon Jan 26 2015 at 4:36:25 PM David Lang da...@lang.hm wrote:
 
  does this handle embedded quotes in the string? and do you handle
 strings
  starting with ' and  or just one of them?
 
  David Lang
 
  On Tue, 27 Jan 2015, Chris Schafer wrote:
 
  Date: Tue, 27 Jan 2015 00:30:54 +
  From: Chris Schafer chrisp.scha...@gmail.com
  Reply-To: rsyslog-users rsyslog@lists.adiscon.com
  To: rsyslog@lists.adiscon.com
  Subject: [rsyslog] New Pull request for liblognorm - additional
  mmnormalize
  functionality
 
  Just submitted the following pull request:
  https://github.com/rsyslog/liblognorm/pull/20
  And I believe it could solve a lot of issues (at least, it solves a lot
  of
  mine) surrounding mmnormalize parsing in rsyslog. I'm looking for
  comments/issues/holy-crap-you-can't-code-what-are-you-doing, if you
 guys
  have any. This is my first time submitting a patch to a large project
 (or
  at least one where I didn't know the maintainer personally), so be
 gentle
  please :)
 
  Chris
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards
  NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
 myriad
  of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
  DON'T LIKE THAT.
 
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards
  NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
  of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
  DON'T LIKE THAT.
 
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards
  NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.
 
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Clustered servers - client-config suggestions

[Chris Bartram]

We have a setup where our rsyslog servers are a RHEL cluster;  shared 
virtual-IP that is owned by the active member. That virtual IP name is what 
all clients will send traffic to.

Our clients are RHEL 6 systems (and use the standard rsyslog version that comes 
with RHEL6). They will be sending relatively high volumes of data (auditd is 
being setup to use rsyslog on all the clients). Currently all clients are being 
setup to use tcp transport - though we will probably look into RELP later. I'm 
using the r7 stable version of rsyslog on the servers if that matters.

Are there any specific directives I should use on the client side to ensure a 
smooth and quick failover should the servers failover? 

Thanks,
Chris Bartram


The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Config errors v7.6.3

[Chris Bartram]

Was trying to fill out some options that were listed in the online docs but 
flag errors when included in the configs...

rsyslogd: [origin software=rsyslogd swVersion=7.6.3 x-pid=67677 
x-info=http://www.rsyslog.com;] start

input(type=imudp

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
54: parameter 'batchSize' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
54: parameter 'TimeRequery' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

input(type=imptcp

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
64: parameter 'ServerNotifyOnConnectionClose' not known -- typo in config file? 
[try http://www.rsyslog.com/e/2207 ]

input(type=imrelp

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
76: parameter 'KeepAlive.Time' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
76: parameter 'KeepAlive.Interval' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
76: parameter 'KeepAlive.Probes' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
76: parameter 'KeepAlive' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

input(type=imtcp

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
91: parameter 'MaxSessions' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
91: parameter 'MaxListeners' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
91: parameter 'FlowControl' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
91: parameter 'KeepAlive' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]

rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 
91: parameter 'NotifyOnConnectionClose' not known -- typo in config file? [try 
http://www.rsyslog.com/e/2207 ]


-Chris Bartram
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Failover destination doesn't work if TCP connection not closed properly?

[Chris 'Chipper' Chiapusio]

are you using '-j REJECT --reject-with icmp-port-unreachable'

or just the default '-j DROP'?


DROP tells iptables to drop the packet on the floor like it never existed,
giving the sending host no indication as to what happened.  REJECT will
respond with something and you can specify what that something is.

Chip

On Tue, Jul 01, 2014 at 11:38:21AM +, Max Williams wrote:

Hi,
I am trying to get reliable failover logging to 2 remote hosts using this 
config:

*.* @@remote1:514
$ActionExecOnlyWhenPreviousIsSuspended on
 @@remote2:514
$ActionExecOnlyWhenPreviousIsSuspended off

This works fine if I stop syslog on the remote1 host, the rsyslog client host fails 
over and fails back with no problems. But if I use iptables to drop TCP/514 on 
remote1 server then on the client host the TCP connection goes to CLOSE_WAIT and 
then to LAST_ACK  SYN_SENT and finally to just SYN_SENT. It then just stays as 
SYN_SENT indefinitely and rsyslog does not failover to the second destination:
tcp0  1 client host:40416  remote1:514
SYN_SENT3393/rsyslogd

I've read Rainer's blog 
posthttp://blog.gerhards.net/2011/03/using-failover-and-asynchornous-actions.html and I 
do not have $ActionQueueType LinkedList set.

Is there some configuration I am missing? I'm using version 5.8.10, I know it's 
old.

Thanks,
Max



The London Metal Exchange is a company incorporated in England and Wales with 
registered number 02128666, VAT registered number GB 918 4582 96 and having its 
registered office at 56 Leadenhall Street, London EC3A 2DX.

LME Clear Limited is a company incorporated in England and Wales with 
registered number 07611628, VAT registered number GB 918 4582 96 and having its 
registered office at 56 Leadenhall Street, London EC3A 2DX.

The London Metal Exchange is a recognised investment exchange, supervised by 
the Financial Conduct Authority (FCA).

This email may have been sent on behalf of The London Metal Exchange, LME Clear 
Limited, or jointly on behalf of both.

Please note that this message is intended for the named recipient(s) only. Its 
contents may be confidential or subject to professional privilege.  If you are 
not an intended recipient, you may not disclose, copy or use in any way the 
information contained in it; please delete it and notify lmehelpd...@lme.com 
immediately and delete it from your system.

Unless expressly attributed, the views expressed in this email do not 
necessarily represent the views of the London Metal Exchange or LME Clear 
Limited.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


--
--
 Warning 
This e-mail message, without warrant or warning, and despite US law as set
forth in the Foreign Intelligence Surveillance Act of 1978, may be subject
to monitoring by the United States National Security Agency and/or the
Department of Defense. Information contained in this message may be used
against any senders or recipients, now or in the future, in a public trial
or secret tribunal.
  Please encrypt anything important.
   PGP Key: http://wwwkeys.pgp.net:11371/pks/lookup?op=getsearch=0x6CFA486D
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] user-space kmsg logging issues with rsyslog

[Chris J Arges]

Hi,
I've noticed that linux kernels before this commit behave differently in
rsyslog:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=7ff9554bb578ba02166071d2d487b7fc7d860d62

What I've observed is if I do something like the following in kernels
before this patch:
# echo test  /dev/kmsg

This will show up in kern.log with something as simple as:
kern.*  /var/log/kern.log

However kernels after that patch no longer show up in kern.log with the
same rule. What I've noticed is the default userspace kmsg priority
level is different (observed via dmesg -r):

Before that patch if we echo something into /dev/kmsg we get:
4[ 35.084348] before

If we do it on or after that patch we get:
12[ 71.091005] after

According to this documentation:
http://www.mjmwired.net/kernel/Documentation/ABI/testing/dev-kmsg

The N value is both the priority and facility combined (after that
patch was introduced).

Is there a way to specify kernel priority/facility levels greater than 7
in order to log userspace generated kmsg entries?

Thanks,
--chris j arges
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] user-space kmsg logging issues with rsyslog

[Chris J Arges]

On 06/30/2014 03:02 PM, David Lang wrote:
 On Mon, 30 Jun 2014, Chris J Arges wrote:
 
 Hi,
 I've noticed that linux kernels before this commit behave differently in
 rsyslog:
 http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/patch/?id=7ff9554bb578ba02166071d2d487b7fc7d860d62


 What I've observed is if I do something like the following in kernels
 before this patch:
 # echo test  /dev/kmsg

 This will show up in kern.log with something as simple as:
 kern.*  /var/log/kern.log

 However kernels after that patch no longer show up in kern.log with the
 same rule. What I've noticed is the default userspace kmsg priority
 level is different (observed via dmesg -r):

 Before that patch if we echo something into /dev/kmsg we get:
 4[ 35.084348] before

 If we do it on or after that patch we get:
 12[ 71.091005] after

 According to this documentation:
 http://www.mjmwired.net/kernel/Documentation/ABI/testing/dev-kmsg

 The N value is both the priority and facility combined (after that
 patch was introduced).

 Is there a way to specify kernel priority/facility levels greater than 7
 in order to log userspace generated kmsg entries?
 
 nothing in rsyslog limits these values. what value are you trying to use?
 

So, I've looked here:
http://www.rsyslog.com/doc/imklog.html

I've added this option to /etc/rsyslog.conf:
$ConsoleLogLevel 14

And reloaded/restarted rsyslogd and they still don't seem to show kernel
messages in /var/log/kern.log

I did '# echo test  /dev/kmsg', and nothing shows up in kern.log/syslog.

--chris j arges


 David Lang
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.
 
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Problem with rsyslog deleating

[Chris 'Chipper' Chiapusio]

On Tue, Jun 03, 2014 at 11:09:53AM -0700, David Lang wrote:

On Tue, 3 Jun 2014, Duarte Silva wrote:


From the appliance configuration, they are using syslog only as a
transport for the messages. The messages can then be XML or JSON. I don't
think I will have any luck in trying to swing the appliance maker to make
the messages a one liner. I will try to home brew something out.


They may be using the syslog port, but this isn't syslog transport.

is this being sent of TCP or UDP? can you send us a short tcpdump of the
messages?


It can be sent over TCP or UDP (the example I gave was TCP, check the tcpdump
command line). Not really, sorry.


if UDP, are they sending one message per packet? or can one message span
multiple packets? if one message can span multiple packets, then they are in
deep trouble because UDP is unreliable delivery and packets can get lost or
arrive out of order.


Yes, one of the problems I noticed was that the UDP notification wasn't
contiguous (spanned throughout multiple packets), hence the switch to TCP in
the appliance configuration.


If this is TCP, then a parser module could read the stream and treat each
complete JSON object as a separate message. this would require a custom
module.

What appliance is this?


Malware related, their logging is crap (for example they don't even allow a
Rsyslsog server port change in the configuration).


wow this is broken. It would be nice to know the vendor name, so that 
we could pass the word to avoid this vendor. Security devices that 
can't log sanely are a major problem.


But this looks like something that could be dealt with using the tcp 
transport, but it would be a custom input module.


It is broken but if its the vendor I think it is you do not want to
avoid them. I've sent a private message to Duarte and if its the vendor I'm
recognizing we will work together to get this fixed.

Chip





Compared to what I'm sure you spent on the appliances, paying for a custom
module to receive these messages will be pretty cheap, talk with Rainer off
of the main list to get a quote for this. I've done it in the past. It's
much nicer to throw a little money at Adiscon and have it be part of the
core rsyslog than to hack something up and have to maintain it for future
versions.


I decided to drop Rsyslog and went to Logstash. Not using the appliance
Rsyslog notifications capabilities though. Used the appliance HTTP notifications
instead (sends a POST with the JSON encoded notification using CURL).


Ok, good luck.

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


--
--
 Warning 
This e-mail message, without warrant or warning, and despite US law as set
forth in the Foreign Intelligence Surveillance Act of 1978, may be subject
to monitoring by the United States National Security Agency and/or the
Department of Defense. Information contained in this message may be used
against any senders or recipients, now or in the future, in a public trial
or secret tribunal.
  Please encrypt anything important.
   PGP Key: http://wwwkeys.pgp.net:11371/pks/lookup?op=getsearch=0x6CFA486D
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] could not load module '/lib64/rsyslog/imrelp.so (version 7.6.0)

[Chris 'Chipper' Chiapusio]

This is what RedHat just dropped:

https://rhn.redhat.com/errata/RHSA-2014-0247.html
gnutls-1.4.1-14.el5_10.x86_64.rpm

https://rhn.redhat.com/errata/RHSA-2014-0246.html
gnutls-2.8.5-13.el6_5.x86_64.rpm

Chip

On Thu, Mar 06, 2014 at 09:39:32AM +0200, Radu Gheorghe wrote:

The current is RHEL 6. The latest is 6.5. Anwar has 6.3, but the same
problem is with 6.5.

The old one is v5. Which is still supported, by the way :)

On Thu, Mar 6, 2014 at 8:31 AM, Rainer Gerhards
rgerha...@hq.adiscon.com wrote:

Wait... is thst the current RHEL? I always thought it was the outdated
one...

Rainer

Sent from phone, thus brief.
Am 05.03.2014 22:07 schrieb Radu Gheorghe radu.gheor...@sematext.com:


On Wed, Mar 5, 2014 at 6:44 PM, Rainer Gerhards
rgerha...@hq.adiscon.com wrote:
 As far as I remember, building a recent GnuTLS on that old platform is a
 lot of hassle. That, plus David's info on the vuln probably means we
won't
 go through the hassle.

My $0.02 is that going through the hassle would be nice once the
vulnerability is fixed. I've gone through that and got it working in a
day (or half a day, I don't remember). And I'm very much a newbie with
regards to compiling stuff in general and gnutls in particular.


 @Andre: I still wonder if the dependency for the relp package is not
 correct. Should it specify the newer GnutTLS version? If it doesn't, relp
 won't work in any case, right? So if that's the case (and RH does not
 backport a newer version), that probably means we should officially cease
 relp support for that old version (as far as rsyslog's own packages are
 concerned).

 Comments?

Dropping support for RHEL/CentOS 6, while 7 is still in beta is a bit
too much, IMO. Maybe it's just the QA in me saying that. Still, this
would make RPMs pretty much useless, wouldn't it?

But let's get constructive. So let me switch to that mode:
- does relp really depend on gnutls? maybe it shouldn't, unless you
want to use RELP+TLS. Otherwise, the same would apply to omfwd. If I
want to use TCP+TLS, I need gnutls, right? Wait, that should work with
the rsyslog-gnutls package. What does that actually provide? I see
./lib64/rsyslog/lmnsd_gtls.so, but this doesn't say much to me.
- I think it's highly desired to have an easy way for people using
RHEL/CentOS to get [all the features of] rsyslog installed without
going through the hassle of compiling a new gnutls. Disclosure: I'm
one of those people.

It sounds like the way to go is to provide a gnutls package in the
rsyslog RPMS (rsyslog-gnutls or a new package?). And a few of us who
are interested can join an effort that wouldn't have to be duplicated
by lots of other people using RHEL/CentOS/Scientific Linux/etc/etc.
I'm willing to join that effort.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.



--
--
 Warning 
This e-mail message, without warrant or warning, and despite US law as set
forth in the Foreign Intelligence Surveillance Act of 1978, may be subject
to monitoring by the United States National Security Agency and/or the
Department of Defense. Information contained in this message may be used
against any senders or recipients, now or in the future, in a public trial
or secret tribunal.
  Please encrypt anything important.
   PGP Key: http://wwwkeys.pgp.net:11371/pks/lookup?op=getsearch=0x6CFA486D
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Sending a custom log to a remote server

[Chris Mann]

Update: 

I got it working, the changes I made in the config file that I posted here 
worked. I just had to comment out the drop privs part of the config file on the 
client server. Life is good.

Silly Ubuntu.

Thanks for all your help David and Rainer!


On Feb 19, 2014, at 10:02 AM, David Lang da...@lang.hm wrote:

 On Wed, 19 Feb 2014, Rainer Gerhards wrote:
 
 are you on ubuntu? Their defaut config drops privileges, but the file
 system has wrong perms. Suggest to trx runniung as root, at least for a try.
 
 good point, is this something we can fix in the PPA?
 
 David Lang
 
 
 Rainer
 
 
 On Wed, Feb 19, 2014 at 3:30 PM, Chris Mann ch...@walkingthumbs.com wrote:
 
 
 On Feb 19, 2014, at 8:33 AM, David Lang da...@lang.hm wrote:
 
 On Wed, 19 Feb 2014, Chris Mann wrote:
 
 On Feb 18, 2014, at 8:08 PM, David Lang da...@lang.hm wrote:
 
 On Tue, 18 Feb 2014, Chris Mann wrote:
 
 Hello all,
 
 I'm trying to send a custom log file that our program generates to
 the remote rsyslog server, with little to no luck. Ideally, I'd like to
 have that log sent to it's own file and not mixed in with the syslog
 traffic.
 
 We're using Ubuntu 12.04LTS
 
 So, if you are using the default version of rsyslog, this is old
 enough that it's unsupported by the community (but your issue is probably
 not version dependant), what version is running?
 
 I'm running v7 stable from the adiscon apt-get repo.
 
 Ok, that helps
 
 
 Server rsyslog server config:
 
 $ModLoad imuxsock # provides support for local system logging
 $ModLoad imklog   # provides kernel logging support (previously done
 by rklogd)
 $ModLoad immark  # provides --MARK-- message capability
 
 # provides UDP syslog reception
 #$ModLoad imudp
 #$UDPServerRun 514
 
 # provides TCP syslog reception
 $ModLoad imtcp
 $InputTCPServerRun 10514
 
 why use an odd port like this instead of using the standard 514 port?
 
 Just preference and as Rainer said, 514 is used by something else :).
 
 
 $template DynaFile,/var/log/remote/%HOSTNAME%.log
 *.* -?DynaFile
 
 ok, this logs everything into per hostname files, with no filtering
 ahead of it.
 
 ###
  GLOBAL DIRECTIVES 
 ###
 
 #
 # Use traditional timestamp format.
 # To enable high precision timestamps, comment out the following line.
 #
 $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
 
 # Filter duplicated messages
 $RepeatedMsgReduction on
 
 #
 # Set the default permissions for all log files.
 
 $FileOwner syslog
 $FileGroup adm
 $FileCreateMode 0640
 $DirCreateMode 0755
 $Umask 0022
 $PrivDropToUser syslog
 $PrivDropToGroup adm
 
 #
 # Where to place spool files
 #
 $WorkDirectory /var/spool/rsyslog
 
 #
 # Include all config files in /etc/rsyslog.d/
 #
 $IncludeConfig /etc/rsyslog.d/*.conf
 
 # This one is the template to generate the log filename dynamically,
 depending on the client's IP address.
 $template
 %RemoteHost,,/var/log/remote/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/syslog.log
 
 this template is by hostname, not client IP, you would use
 %fromhost-ip% instead of %hostname% if you want it by IP
 
 but it really doesn't matter since you don't have anything that uses
 this template. I also think that you can't use % in a tempate name, and
 should only have one ,
 
 as a result, I'm pretty sure that you get errors about being unable to
 parse the config file when you startup.
 
 Actually, I'm not getting any errors on start up. rsyslog starts up
 just fine.
 
 are you shure? double check that it's not logging anything about errors
 at startup time. that line just doesn't look right. I also don't see any
 place that you are trying to use this template.
 
 Nothing in the log, honest:
 
 Feb 19 14:25:10 bundt rsyslogd: [origin software=rsyslogd
 swVersion=7.4.10 x-pid=31532 x-info=http://www.rsyslog.com;] start
 Feb 19 14:25:10 bundt rsyslogd: rsyslogd's groupid changed to 4
 Feb 19 14:25:10 bundt rsyslogd: rsyslogd's userid changed to 101
 
 
 
 
 Client rsyslog config:
 
 # $ModLoad imfile
 $ModLoad imuxsock # provides support for local system logging
 $ModLoad imklog   # provides kernel logging support (previously done
 by rklogd)
 # $ModLoad immark  # provides --MARK-- message capability
 
 # Watch /var/log/ejabberd/ejabberd.log
 module(load=imfile PollingInterval=10)
 input(type=imfile
 File=/var/log/ejabberd/ejabberd.log
 Tag=ejabberd:
 StateFile=state-ejabberd
 Severity=info
 Facility=local6
 )
 
 # Provides UDP forwarding. The IP is the server's IP address
 # *.* @54.227.155.34:514
 
 # Provides TCP forwarding. But the current server runs on UDP
 *.* @@devil.walkingservers.net:10514
 
 # provides UDP syslog reception
 #$ModLoad imudp
 #$UDPServerRun 514
 
 # provides TCP syslog reception
 #$ModLoad imtcp
 #$InputTCPServerRun 514
 
 
 ###
  GLOBAL DIRECTIVES 
 ###
 
 #
 # Use traditional timestamp format.
 # To enable high precision timestamps, comment out

Re: [rsyslog] Sending a custom log to a remote server

[Chris Mann]

On Feb 18, 2014, at 8:08 PM, David Lang da...@lang.hm wrote:

 On Tue, 18 Feb 2014, Chris Mann wrote:
 
 Hello all,
 
 I’m trying to send a custom log file that our program generates to the 
 remote rsyslog server, with little to no luck. Ideally, I’d like to have 
 that log sent to it’s own file and not mixed in with the syslog traffic.
 
 We’re using Ubuntu 12.04LTS
 
 So, if you are using the default version of rsyslog, this is old enough that 
 it's unsupported by the community (but your issue is probably not version 
 dependant), what version is running? 

I’m running v7 stable from the adiscon apt-get repo.

 
 Server rsyslog server config:
 
 $ModLoad imuxsock # provides support for local system logging
 $ModLoad imklog   # provides kernel logging support (previously done by 
 rklogd)
 $ModLoad immark  # provides --MARK-- message capability
 
 # provides UDP syslog reception
 #$ModLoad imudp
 #$UDPServerRun 514
 
 # provides TCP syslog reception
 $ModLoad imtcp
 $InputTCPServerRun 10514
 
 why use an odd port like this instead of using the standard 514 port?

Just preference and as Rainer said, 514 is used by something else :).

 
 $template DynaFile,/var/log/remote/%HOSTNAME%.log
 *.* -?DynaFile
 
 ok, this logs everything into per hostname files, with no filtering ahead of 
 it.
 
 ###
  GLOBAL DIRECTIVES 
 ###
 
 #
 # Use traditional timestamp format.
 # To enable high precision timestamps, comment out the following line.
 #
 $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
 
 # Filter duplicated messages
 $RepeatedMsgReduction on
 
 #
 # Set the default permissions for all log files.
 
 $FileOwner syslog
 $FileGroup adm
 $FileCreateMode 0640
 $DirCreateMode 0755
 $Umask 0022
 $PrivDropToUser syslog
 $PrivDropToGroup adm
 
 #
 # Where to place spool files
 #
 $WorkDirectory /var/spool/rsyslog
 
 #
 # Include all config files in /etc/rsyslog.d/
 #
 $IncludeConfig /etc/rsyslog.d/*.conf
 
 # This one is the template to generate the log filename dynamically, 
 depending on the client's IP address.
 $template 
 %RemoteHost,,/var/log/remote/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/syslog.log
 
 this template is by hostname, not client IP, you would use %fromhost-ip% 
 instead of %hostname% if you want it by IP
 
 but it really doesn't matter since you don't have anything that uses this 
 template. I also think that you can't use % in a tempate name, and should 
 only have one ,
 
 as a result, I'm pretty sure that you get errors about being unable to parse 
 the config file when you startup.

Actually, I’m not getting any errors on start up. rsyslog starts up just fine. 

 
 
 Client rsyslog config:
 
 # $ModLoad imfile
 $ModLoad imuxsock # provides support for local system logging
 $ModLoad imklog   # provides kernel logging support (previously done by 
 rklogd)
 # $ModLoad immark  # provides --MARK-- message capability
 
 # Watch /var/log/ejabberd/ejabberd.log
 module(load=imfile PollingInterval=10)
 input(type=imfile
   File=/var/log/ejabberd/ejabberd.log
   Tag=ejabberd:
   StateFile=state-ejabberd
   Severity=info
   Facility=local6
   )
 
 # Provides UDP forwarding. The IP is the server's IP address
 # *.* @54.227.155.34:514
 
 # Provides TCP forwarding. But the current server runs on UDP
 *.* @@devil.walkingservers.net:10514
 
 # provides UDP syslog reception
 #$ModLoad imudp
 #$UDPServerRun 514
 
 # provides TCP syslog reception
 #$ModLoad imtcp
 #$InputTCPServerRun 514
 
 
 ###
  GLOBAL DIRECTIVES 
 ###
 
 #
 # Use traditional timestamp format.
 # To enable high precision timestamps, comment out the following line.
 #
 $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
 
 # Filter duplicated messages
 $RepeatedMsgReduction on
 
 #
 # Set the default permissions for all log files.
 #
 $FileOwner syslog
 $FileGroup adm
 $FileCreateMode 0640
 $DirCreateMode 0755
 $Umask 0022
 $PrivDropToUser syslog
 $PrivDropToGroup adm
 
 #
 # Where to place spool files
 #
 $WorkDirectory /var/spool/rsyslog
 
 #
 $IncludeConfig /etc/rsyslog.d/*.conf
 
 
 Can someone kick me in the direction of where I’m screwing up?
 
 In general, you should put global directives before any output. I don't know 
 if that matters or not
 
 I don't know of there is anything being added by the include lines.
 
 
 so, this sends logs from the client to the server, using the default format 
 (because you haven't specified anything), and the server then writes them to 
 /var/log/hostname.log files
 
 now, you do set the logs you read from the file to the facility local6, so 
 you could filter on that on the server if you want them written separately
 
 but, what is it that you think should be happening with this config? and what 
 is actually happening?

Long story short, I’d like the ejabberd.log file to go to 
/var/log/remotes/$hostname/ejabberd.log as well as have the remote syslog file

Re: [rsyslog] Sending a custom log to a remote server

[Chris Mann]

On Feb 19, 2014, at 8:33 AM, David Lang da...@lang.hm wrote:

 On Wed, 19 Feb 2014, Chris Mann wrote:
 
 On Feb 18, 2014, at 8:08 PM, David Lang da...@lang.hm wrote:
 
 On Tue, 18 Feb 2014, Chris Mann wrote:
 
 Hello all,
 
 I’m trying to send a custom log file that our program generates to the 
 remote rsyslog server, with little to no luck. Ideally, I’d like to have 
 that log sent to it’s own file and not mixed in with the syslog traffic.
 
 We’re using Ubuntu 12.04LTS
 
 So, if you are using the default version of rsyslog, this is old enough 
 that it's unsupported by the community (but your issue is probably not 
 version dependant), what version is running?
 
 I’m running v7 stable from the adiscon apt-get repo.
 
 Ok, that helps
 
 
 Server rsyslog server config:
 
 $ModLoad imuxsock # provides support for local system logging
 $ModLoad imklog   # provides kernel logging support (previously done by 
 rklogd)
 $ModLoad immark  # provides --MARK-- message capability
 
 # provides UDP syslog reception
 #$ModLoad imudp
 #$UDPServerRun 514
 
 # provides TCP syslog reception
 $ModLoad imtcp
 $InputTCPServerRun 10514
 
 why use an odd port like this instead of using the standard 514 port?
 
 Just preference and as Rainer said, 514 is used by something else :).
 
 
 $template DynaFile,/var/log/remote/%HOSTNAME%.log
 *.* -?DynaFile
 
 ok, this logs everything into per hostname files, with no filtering ahead 
 of it.
 
 ###
  GLOBAL DIRECTIVES 
 ###
 
 #
 # Use traditional timestamp format.
 # To enable high precision timestamps, comment out the following line.
 #
 $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
 
 # Filter duplicated messages
 $RepeatedMsgReduction on
 
 #
 # Set the default permissions for all log files.
 
 $FileOwner syslog
 $FileGroup adm
 $FileCreateMode 0640
 $DirCreateMode 0755
 $Umask 0022
 $PrivDropToUser syslog
 $PrivDropToGroup adm
 
 #
 # Where to place spool files
 #
 $WorkDirectory /var/spool/rsyslog
 
 #
 # Include all config files in /etc/rsyslog.d/
 #
 $IncludeConfig /etc/rsyslog.d/*.conf
 
 # This one is the template to generate the log filename dynamically, 
 depending on the client's IP address.
 $template 
 %RemoteHost,,/var/log/remote/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/syslog.log
 
 this template is by hostname, not client IP, you would use %fromhost-ip% 
 instead of %hostname% if you want it by IP
 
 but it really doesn't matter since you don't have anything that uses this 
 template. I also think that you can't use % in a tempate name, and should 
 only have one ,
 
 as a result, I'm pretty sure that you get errors about being unable to 
 parse the config file when you startup.
 
 Actually, I’m not getting any errors on start up. rsyslog starts up just 
 fine.
 
 are you shure? double check that it's not logging anything about errors at 
 startup time. that line just doesn't look right. I also don't see any place 
 that you are trying to use this template.

Nothing in the log, honest:

Feb 19 14:25:10 bundt rsyslogd: [origin software=rsyslogd swVersion=7.4.10 
x-pid=31532 x-info=http://www.rsyslog.com;] start
Feb 19 14:25:10 bundt rsyslogd: rsyslogd's groupid changed to 4
Feb 19 14:25:10 bundt rsyslogd: rsyslogd's userid changed to 101

 
 
 
 Client rsyslog config:
 
 # $ModLoad imfile
 $ModLoad imuxsock # provides support for local system logging
 $ModLoad imklog   # provides kernel logging support (previously done by 
 rklogd)
 # $ModLoad immark  # provides --MARK-- message capability
 
 # Watch /var/log/ejabberd/ejabberd.log
 module(load=imfile PollingInterval=10)
 input(type=imfile
  File=/var/log/ejabberd/ejabberd.log
  Tag=ejabberd:
  StateFile=state-ejabberd
  Severity=info
  Facility=local6
  )
 
 # Provides UDP forwarding. The IP is the server's IP address
 # *.* @54.227.155.34:514
 
 # Provides TCP forwarding. But the current server runs on UDP
 *.* @@devil.walkingservers.net:10514
 
 # provides UDP syslog reception
 #$ModLoad imudp
 #$UDPServerRun 514
 
 # provides TCP syslog reception
 #$ModLoad imtcp
 #$InputTCPServerRun 514
 
 
 ###
  GLOBAL DIRECTIVES 
 ###
 
 #
 # Use traditional timestamp format.
 # To enable high precision timestamps, comment out the following line.
 #
 $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
 
 # Filter duplicated messages
 $RepeatedMsgReduction on
 
 #
 # Set the default permissions for all log files.
 #
 $FileOwner syslog
 $FileGroup adm
 $FileCreateMode 0640
 $DirCreateMode 0755
 $Umask 0022
 $PrivDropToUser syslog
 $PrivDropToGroup adm
 
 #
 # Where to place spool files
 #
 $WorkDirectory /var/spool/rsyslog
 
 #
 $IncludeConfig /etc/rsyslog.d/*.conf
 
 
 Can someone kick me in the direction of where I’m screwing up?
 
 In general, you should put global directives before any output. I don't 
 know if that matters or not
 
 I don't know of there is anything being added

[rsyslog] Sending a custom log to a remote server

[Chris Mann]

Hello all,

I’m trying to send a custom log file that our program generates to the remote 
rsyslog server, with little to no luck. Ideally, I’d like to have that log sent 
to it’s own file and not mixed in with the syslog traffic.

We’re using Ubuntu 12.04LTS

Server rsyslog server config:

$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)
$ModLoad immark  # provides --MARK-- message capability

# provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 10514

$template DynaFile,/var/log/remote/%HOSTNAME%.log
*.* -?DynaFile
###
 GLOBAL DIRECTIVES 
###

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Filter duplicated messages
$RepeatedMsgReduction on

#
# Set the default permissions for all log files.

$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup adm

#
# Where to place spool files
#
$WorkDirectory /var/spool/rsyslog

#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf

# This one is the template to generate the log filename dynamically, depending 
on the client's IP address.
$template 
%RemoteHost,,/var/log/remote/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/syslog.log


Client rsyslog config:

# $ModLoad imfile
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)
# $ModLoad immark  # provides --MARK-- message capability

# Watch /var/log/ejabberd/ejabberd.log
module(load=imfile PollingInterval=10)
input(type=imfile
File=/var/log/ejabberd/ejabberd.log
Tag=ejabberd:
StateFile=state-ejabberd
Severity=info
Facility=local6
)

# Provides UDP forwarding. The IP is the server's IP address
# *.* @54.227.155.34:514

# Provides TCP forwarding. But the current server runs on UDP
*.* @@devil.walkingservers.net:10514

# provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514


###
 GLOBAL DIRECTIVES 
###

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Filter duplicated messages
$RepeatedMsgReduction on

#
# Set the default permissions for all log files.
#
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup adm

#
# Where to place spool files
#
$WorkDirectory /var/spool/rsyslog

#
$IncludeConfig /etc/rsyslog.d/*.conf


Can someone kick me in the direction of where I’m screwing up?
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] rsyslog 7.6.0 (v7-stable) released

[Chris 'Chipper' Chiapusio]

The supporting libraries need to be published to the v7-stable tree, but for
those that can't wait you can probably grab them from the v7-devel tree.

Chip

On Thu, Feb 13, 2014 at 11:11:49AM -0700, Kendall Green wrote:

RPMs are out available for 7.6, but want to mention that rsyslog-relp-7.6.0
package has issues resolving required libraries, librelp  1.1.1.
The number of bits in RainerScript integers, 32 limitation to resolve with
json-c update, and any fixed for including with rsyslog7.6, so please can
you package the available (librelp 1.2.2 and json-c 0.11) dependencies?

Thanks,
-Kg


On Thu, Feb 13, 2014 at 1:23 AM, Andre Lorbach alorb...@adiscon.com wrote:


Hi all,

the WAIT is over ;).
RPM's for RSyslog V7.6.0 Stable are online now.

Best regards,
Andre Lorbach

 -Original Message-
 From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
 boun...@lists.adiscon.com] On Behalf Of Boylan, James
 Sent: Wednesday, February 12, 2014 9:18 PM
 To: rsyslog-users
 Subject: Re: [rsyslog] rsyslog 7.6.0 (v7-stable) released

 Best news I've heard all day. Time to build out the new RPMs. :)

 -- James

 -Original Message-
 From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
 boun...@lists.adiscon.com] On Behalf Of Rainer Gerhards
 Sent: Wednesday, February 12, 2014 1:45 PM
 To: rsyslog-users
 Subject: Re: [rsyslog] rsyslog 7.6.0 (v7-stable) released

 We didn't manage today, as it looks... for sure tomorrow.

 Sent from phone, thus brief.
 Am 12.02.2014 20:43 schrieb Xuri Nagarin secs...@gmail.com:

  +1 for the RPM release, hitting yum update every 2 seconds :)
 
 
  On Wed, Feb 12, 2014 at 9:51 AM, Nick Syslog rsys...@nanoscopic.net
  wrote:
   Anxiously anticipating the RHEL/CentOS RPMs for 7.6 :o)
  
   Hooray for pstats!
  
  
   On Wed, Feb 12, 2014 at 8:32 AM, Florian Riedl fri...@adiscon.com
  wrote:
  
   Hi everyone.
  
   This is the first release of rsyslog 7.6 in the v7-stable branch.
  
   Since 7.4 a lot of new functions have found their way into rsyslog.
   With
   7.6 being the successor of the 7.5 development branch, everything
   that
  has
   been added there has now found its way into the stable version. The
  major
   additions consist of
  
  - imrelp/omrelp now support TLS  (zip) compression
  - impstats is now emitting resource usage counters, can directly
emit
  delta values and can now be bound to a ruleset
  - mmpstrucdata is a new module to parse RFC5424 structured data
into
  JSON message properties
  - mmutf8fix is a new module to fix invalid UTF-8 sequences
  - mmsequence is a new module that helps with action load
balancing
  - new defaults for main/ruleset queues to be more
   enterprise-like
  
   Also the new stable version has undergone a lot of bug fixes,
  performance
   improvements and optimizations that make rsyslog 7.6 a lot more
   reliable and performing than before.
  
   Also, requirements have changed a little. For rsyslog 7.6 you now
  require
   librelp 1.1.4 and libestr 0.1.7 due to major fixes.
  
   More detailed information is available in the ChangeLog.
  
   ChangeLog:
  
   http://www.rsyslog.com/changelog-for-7-6-0-v7-stable/
  
   Download:
  
   http://www.rsyslog.com/rsyslog-7-6-0-v7-stable/
  
   We have also released version 7.4.10 with some late crucial fixes.
   This
  is
   the definitive last release of 7.4 with 7.6 now succeeding it.
  
   As always, feedback is appreciated.
  
   Best regards,
   Florian Riedl

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


--
--
 Warning 
This e-mail message, without warrant or warning, and despite US law as set
forth in the Foreign Intelligence Surveillance Act of 1978, may be subject
to monitoring by the United States National Security Agency and/or the
Department of Defense. Information contained in this message may be used
against any senders or recipients, now or in the future, in a public trial
or secret tribunal.
  Please encrypt anything important.
   PGP Key: http://wwwkeys.pgp.net:11371/pks/lookup?op=getsearch=0x6CFA486D
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] v7.4.7 epel-6 RPMs

[Chris 'Chipper' Chiapusio]

I'm wondering if there is a problem generating the epel6 packages or if it
just got missed since the epel5 packages have been on the site for a couple
days.

Thanks,
Chip



--
--
 Warning 
This e-mail message, without warrant or warning, and despite US law as set
forth in the Foreign Intelligence Surveillance Act of 1978, may be subject
to monitoring by the United States National Security Agency and/or the
Department of Defense. Information contained in this message may be used
against any senders or recipients, now or in the future, in a public trial
or secret tribunal.
  Please encrypt anything important.
   PGP Key: http://wwwkeys.pgp.net:11371/pks/lookup?op=getsearch=0x6CFA486D
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] regex filter syntax for v7

[Chris Bartram]

Below are a couple sanitized examples of the debug format.
So I want to drop all messages where programname=kernel and msg regex ^ 
type=\d+ audit\(.*\)
(type= values vary)

What's the proper R7 syntax for that (including what needs to be escaped)? *I 
have some other similar filters I want to implement so REALLY want to get the 
regex syntax down.

Thanks.
 -Chris Bartram

Debug line with all properties:
FROMHOST: snip, PRI: 5,
syslogtag 'kernel:', programname: 'kernel', APP-NAME: 'kernel', PROCID: '-', 
MSGID: '-',
TIMESTAMP: 'Dec  3 17:18:38', STRUCTURED-DATA: '-',
msg: ' type=1302 audit(1386109118.424:31333674): item=2 name=/usr/xyz 
inode=138597 dev=fd:06 mode=0100640 ouid=000 ogid=000 rdev=00:00'
escaped msg: ' type=1302 audit(1386109118.424:31333674): item=2 name=/usr/xyz 
inode=138597 dev=fd:06 mode=0100640 ouid=000 ogid=000 rdev=00:00'
inputname: imudp rawmsg: '5Dec  3 17:18:38 host kernel: type=1302 
audit(1386109118.424:31333674): item=2 name=/usr/xyz  inode=138597 dev=fd:06 
mode=0100640 ouid=000 ogid=000 rdev=00:00'

Debug line with all properties:
FROMHOST: snip, PRI: 5,
syslogtag 'kernel:', programname: 'kernel', APP-NAME: 'kernel', PROCID: '-', 
MSGID: '-',
TIMESTAMP: 'Dec  3 17:18:38', STRUCTURED-DATA: '-',
msg: ' type=1302 audit(1386109118.424:31333674): item=3 
name=/usr/xyz/agent/agent_inst/sysman/emd/agntstmp.txt.bak inode=138597 
dev=fd:06 mode=0100640 ouid=000 ogid=000 rdev=00:00'
escaped msg: ' type=1302 audit(1386109118.424:31333674): item=3 name=/usr/xyz 
inode=138597 dev=fd:06 mode=0100640 ouid=000 ogid=000 rdev=00:00'
inputname: imudp rawmsg: '5Dec  3 17:18:38 host kernel: type=1302 
audit(1386109118.424:31333674): item=3 name=/usr/xyz  inode=138597 dev=fd:06 
mode=0100640 ouid=000 ogid=000 rdev=00:00'
The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)


On Mon, 12/2/13, David Lang da...@lang.hm wrote:

 Subject: Re: [rsyslog] regex filter syntax for v7
 To: rsyslog-users rsyslog@lists.adiscon.com
 Date: Monday, December 2, 2013, 11:57 PM
 
 as I said earlier, I think this is
 because kernel: is the programname, it's not 
 part of the message, so when you look for it in msg, you
 aren't ever going to 
 find it.
 
 output some of the logs with the format RSYSLOG_DebugFormat
 and look at what 
 gets put into each of the variables, it will help a lot when
 you run into issues 
 like this.
 
 David Lang
 
 On Mon, 2 Dec 2013, Chris Bartram wrote:
 
  Tried the script with my example and it didn't indicate
 I needed to escape anything; ^kernel: type=[0-9]+ audit
 
  Yet when I tried the following in my .conf file it
 didn't catch (suppress) any records.
 
  :msg, regex, ^kernel: type=[0-9]+ audit stop
 
  -Chris Bartram
 
  The purpose of life is not to be happy. It is to be
 useful, to be honorable, to be compassionate, to have it
 make some difference that you have lived and lived well.
 (Ralph Waldo Emerson)
 
  
  On Mon, 12/2/13, Rainer Gerhards rgerha...@hq.adiscon.com
 wrote:
 
  Subject: Re: [rsyslog] regex filter syntax for v7
  To: rsyslog-users rsyslog@lists.adiscon.com
  Date: Monday, December 2, 2013, 11:04 AM
 
  On Mon, Dec 2, 2013 at 3:28 PM,
  Rainer Gerhards rgerha...@hq.adiscon.comwrote:
 
   On Mon, Dec 2, 2013 at 1:39 PM, Chris Bartram
 chrisrbart...@yahoo.comwrote:
  
   Still looking for help on this. As I said I
 need
  REGEX syntax (including
   characters that might need escaping) and
 didn't see
  anything helpful in the
   online docs.
  
  
   Well, basically you need to know how to form your
 POSIX
  ERE regexp. Once
   you have this string, you need to include it in a
  proper constant. For
   example a backslash is escape character, so you
 need to
  escape it by using
   two backslashes (that's the same in any
 programming and
  config language,
   it's not rsyslog-specific...).
  
   Let me see if we can do a quick online tool for
 the
  escaping...
  
 
  I have written a small escaper. It's available at:
 
  http://www.rsyslog.com/rainerscript-constant-string-escaper/
 
  Not 100% perfect yet, but I think it escapes
 everything
  correctly (but I
  need to verify it against rsyslog code, not happen
 today).
  If you have
  problems, let me know.
 
  Rainer
 
 
  
   Rainer
  
  
   Thanks,
    Chris Bartram
  
  
   The purpose of life is not to be happy. It is
 to
  be useful, to be
   honorable, to be compassionate, to have it
 make
  some difference that you
   have lived and lived well. (Ralph Waldo
 Emerson)
  
   
   On Wed, 11/27/13, Chris Bartram chrisrbart...@yahoo.com
  wrote:
  
    Subject: [rsyslog] regex filter syntax for
  v7
    To: rsyslog-users rsyslog@lists.adiscon.com
    Date: Wednesday, November 27, 2013, 12:24
 AM
  
  
    Can someone provide me an example

Re: [rsyslog] regex filter syntax for v7

[Chris Bartram]

Tried the script with my example and it didn't indicate I needed to escape 
anything; ^kernel: type=[0-9]+ audit

Yet when I tried the following in my .conf file it didn't catch (suppress) any 
records.

:msg, regex, ^kernel: type=[0-9]+ audit stop

-Chris Bartram

The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)


On Mon, 12/2/13, Rainer Gerhards rgerha...@hq.adiscon.com wrote:

 Subject: Re: [rsyslog] regex filter syntax for v7
 To: rsyslog-users rsyslog@lists.adiscon.com
 Date: Monday, December 2, 2013, 11:04 AM
 
 On Mon, Dec 2, 2013 at 3:28 PM,
 Rainer Gerhards rgerha...@hq.adiscon.comwrote:
 
  On Mon, Dec 2, 2013 at 1:39 PM, Chris Bartram chrisrbart...@yahoo.comwrote:
 
  Still looking for help on this. As I said I need
 REGEX syntax (including
  characters that might need escaping) and didn't see
 anything helpful in the
  online docs.
 
 
  Well, basically you need to know how to form your POSIX
 ERE regexp. Once
  you have this string, you need to include it in a
 proper constant. For
  example a backslash is escape character, so you need to
 escape it by using
  two backslashes (that's the same in any programming and
 config language,
  it's not rsyslog-specific...).
 
  Let me see if we can do a quick online tool for the
 escaping...
 
 
 I have written a small escaper. It's available at:
 
 http://www.rsyslog.com/rainerscript-constant-string-escaper/
 
 Not 100% perfect yet, but I think it escapes everything
 correctly (but I
 need to verify it against rsyslog code, not happen today).
 If you have
 problems, let me know.
 
 Rainer
 
 
 
  Rainer
 
 
  Thanks,
   Chris Bartram
 
 
  The purpose of life is not to be happy. It is to
 be useful, to be
  honorable, to be compassionate, to have it make
 some difference that you
  have lived and lived well. (Ralph Waldo Emerson)
 
  
  On Wed, 11/27/13, Chris Bartram chrisrbart...@yahoo.com
 wrote:
 
   Subject: [rsyslog] regex filter syntax for
 v7
   To: rsyslog-users rsyslog@lists.adiscon.com
   Date: Wednesday, November 27, 2013, 12:24 AM
 
 
   Can someone provide me an example of a
 working regex (has to
   be regex) filter I can use in my v7
 rsyslog.conf on a RHEL5
   server to ignore/drop messages meeting a
 specific
   expression?
 
   Examples I've tried didn't work; and I see
 notes in other
   forums about needing to double-escape
 characters in the
   regex?
 
   **It would be extra helpful if the regex
 example could use
   perl-like syntax? something like 
 ^kernel\[\d+\] XYZ
 
   Thanks!
   -Chris Bartram
 
 
   The purpose of life is not to be happy. It
 is to be useful,
   to be honorable, to be compassionate, to have
 it make some
   difference that you have lived and lived
 well. (Ralph Waldo
   Emerson)
  
 ___
   rsyslog mailing list
   http://lists.adiscon.net/mailman/listinfo/rsyslog
   http://www.rsyslog.com/professional-services/
   What's up with rsyslog? Follow https://twitter.com/rgerhards
   NOTE WELL: This is a PUBLIC mailing list,
 posts are ARCHIVED
   by a myriad of sites beyond our control.
 PLEASE UNSUBSCRIBE
   and DO NOT POST if you DON'T LIKE THAT.
 
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards
  NOTE WELL: This is a PUBLIC mailing list, posts are
 ARCHIVED by a myriad
  of sites beyond our control. PLEASE UNSUBSCRIBE and
 DO NOT POST if you
  DON'T LIKE THAT.
 
 
 
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED
 by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE
 and DO NOT POST if you DON'T LIKE THAT.
 
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] regex filter syntax for v7

[Chris Bartram]

Can someone provide me an example of a working regex (has to be regex) filter I 
can use in my v7 rsyslog.conf on a RHEL5 server to ignore/drop messages meeting 
a specific expression?

Examples I've tried didn't work; and I see notes in other forums about needing 
to double-escape characters in the regex?

**It would be extra helpful if the regex example could use perl-like syntax? 
something like  ^kernel\[\d+\] XYZ

Thanks!
-Chris Bartram 


The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] v7.4.6 severe backlogs; need tuning help

[Chris Bartram]

The if statement below didn't work either?  Still getting flooded with those 
messages and others that I definitely need a regex to identify.  Any examples 
of a working regex filter in v7 format?

Many thanks for all the help! 
Chris Bartram

Sent from Yahoo Mail on Android

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] v7.4.6 severe backlogs; need tuning help

[Chris Bartram]

Rsyslog v7.4.6 on RHEL5 (VM): pipe (disk assist?) files continue to build up.

More benchmarks today; added detailed (millisecond level) timing to my script 
output to track down what’s slowing it down. 

Most of the time the script completes in less than 0.0001 seconds (time from 
the point where a record is read until the time the script goes back to read 
another record); the worst times I see are around 0.0135 seconds (typically due 
to dns lookups).
 
On the other hand, in a period of 30,000 incoming records I monitored, there 
were 118 instances where the actual file read took 30 seconds to complete?? And 
ALL of these instances were ALMOST EXACTLY 30 seconds (some examples):
 
(took 29.0241)
(took 29.2514)
(took 29.7580)
(took 28.9838)
(took 29.3149)
(took 28.6892)
(took 29.0497)
(took 28.9364)
(took 29.5044)
(took 28.9323)
(took 28.7323)
(took 29.1876)
(took 28.9036)
(took 29.5737)
(took 29.2888)
(took 29.0551)
(took 29.4591)
(took 28.6651)
(took 29.0516)
(took 29.3968)
(took 29.2382)
(took 29.1401)
(took 29.6804)
(took 28.3885)

This looks suspiciously like a timeout somewhere; I have no timeouts configured 
in my (Perl) script code so it’s something external.

Iostat reports all along show not much pressure at that level. Top on the host 
shows average cpu utilization under 10%.

My code is in a “while (INPUTFILE) {}” loop – so nothing fancy. So where is 
the 30 second timeout coming from??

-Chris Bartram

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] v7.4.6 severe backlogs; need tuning help

[Chris Bartram]

 rsyslog_pipe_kern.0018
-rw--- 1 root root 1049022 Nov 21 12:29 rsyslog_pipe_kern.0019
-rw--- 1 root root 1048956 Nov 21 12:29 rsyslog_pipe_kern.0020
-rw--- 1 root root 1049084 Nov 21 12:29 rsyslog_pipe_kern.0021
-rw--- 1 root root 1048605 Nov 21 12:29 rsyslog_pipe_kern.0022
-rw--- 1 root root  686242 Nov 21 12:29 rsyslog_pipe_kern.0023

-Chris Bartram
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] v7.4.6 severe backlogs; need tuning help

[Chris Bartram]

Agreed.

I am still confused as to why the script keeps getting fed current syslog 
records rather than FIFO though? Even if rsyslog has to start pushing data to 
disk shouldn't my script be forced to ingest the oldest data before being 
handed current data? Or is there some config option to force that which I'm 
missing.

I am examining the script to try and determine where it's getting bogged down; 
I'm also starting to filter some of the recently added (floods) of data we 
don't care about (at least on the alerting server) in the rsyslog configuration 
file. To that end;

I'm trying to drop/ignore incoming records like this:

kernel: type=1123 audit(1385078725.944:14351983): user pid=32142 uid=0 
auid=1101 ses=168513 msg='cwd=/tmp 
cmd=64636C69202D6C207266F74202D320646D303463656C303107073202D6566207C206772657063656C6C737276207C267726570202D762027677265705C7C7374617427
 (terminal=? res=success)'

Trying this:

:msg, regex, ^kernel: type=[0-9]+ audit stop

No syntax error but not working. I saw a post on a Redhat forum that noted you 
need to escape some characters (they only mentioned the + sign) with 
*double* slashes? Couldn't find any complete example on rsyslog.com though...?

-Chris Bartram

The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)


On Thu, 11/21/13, Dave Caplinger davecaplin...@solutionary.com wrote:

 Subject: Re: [rsyslog] v7.4.6 severe backlogs; need tuning help
 To: rsyslog-users rsyslog@lists.adiscon.com
 Date: Thursday, November 21, 2013, 4:36 PM
 
 On Nov 21, 2013, at 2:42 PM, David
 Lang da...@lang.hm
 wrote:
 
  As long as your scripts are unable to process messages
 anywhere close to the
  rate that they are arriving, you will fall behind, and
 you will end up spilling
  to disk and never catching up.
  
  ...
  
  It looks as if your scripts can handle ~4000 messages/5
 min or around 13
  messages/sec. Anything more than that just builds up
 and ends up spilling to
  disk.
  
  David Lang
 
 Chris,
 
 Perhaps the attached will help visualize what is going on
 (assuming it makes it through the mailing list).  We
 all agree that the problem is definitely that your scripts
 that read from the FIFOs are not processing the data quickly
 enough.  My own experience with converting from
 syslog-ng to rsyslog has been similar -- rsyslog has been
 quite a bit faster for me, so this may explain why your
 scripts worked in the past: syslog-ng simply wasn't going
 this fast.
 
 Everything coming in on the inputs (imudp, imtcp, imuxsock)
 is getting enqueued into the Main Q without problem. 
 The Main Q's size remains 0 (and maxqsize stays low),
 indicating that everything that enters the Main Q leaves it
 promptly.
 
 Action 2 and 4 are the ones that are getting data too fast
 and entering DA mode once they exceed 80,000 messages in
 length (your highwatermark setting).  (Which is another
 way of saying that they cannot dequeue the messages quickly
 enough.)  FIFOs will block writers if the reader hasn't
 emptied the buffer yet, and that's exactly what is happening
 here.
 
 The end result is that you are falling behind at a rate of
 around 5,100 messages per 5 minutes.  See what you can
 do to reduce the script's processing time (such as
 processing the incoming data in batches rather than
 per-line?).
 
 - Dave Caplinger
 
 
 -Inline Attachment Follows-
 
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED
 by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE
 and DO NOT POST if you DON'T LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] v7.4.6 severe backlogs; need tuning help

[Chris Bartram]

On Wed, 11/20/13, Rainer Gerhards rgerha...@hq.adiscon.com wrote:

 Subject: Re: [rsyslog] v7.4.6 severe backlogs; need tuning help
 To: rsyslog-users rsyslog@lists.adiscon.com
 Date: Wednesday, November 20, 2013, 10:01 AM
 
 On Wed, Nov 20, 2013 at 3:59 PM,
 Chris Bartram chrisrbart...@yahoo.comwrote:
 
  Since I last restarted yesterday afternoon I currently
 have over 600
  rsyslog_pipe_kern.0650 files in the rsyslog
 working directory...
  numbered .0020 thru 0642.
 
  Oddly there are no files for any of the other queues;
 before I added the
  watermark and batchsize options I was also seeing a
 bunch of
  rsyslog_pipe_other and rsyslog_pipe_cron files
 being created as well.
 
 
 This sounds like you have found the script that is too slow
 to catch up
 (rsyslog_pipe_kern). Again, impstats will show more
 details.
 
 Rainer
 
 **Yes, but based on the work the script has to do there will always be cases 
where traffic comes in faster than the script can process; I just need a 
reliable way to temporarily queue that burst traffic until the script catches 
up (which it always does eventually). I suspect the bursts may be too large for 
memory-resident structures though and I worry about blocking further incoming 
traffic while the script is catching up?

-Chris Bartram

 
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] regex in new template format

[Chris 'Chipper' Chiapusio]

I'm trying to upgrade my template configurations to the new style and I'm not
seeing any examples of how to set up a regex on a field in the new format.

I'm trying to migrate this:

$template LogHostFix,%timegenerated% 
%fromhost:R,ERE,1,FIELD:^(.*)\.(domain.com|domain2.com|domain3.com)$--end% %hostname% 
%syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n

to this:

template(name=LogHostFix2 type=list) {
constant(value=)
property(name=pri)
constant(value=)
property(name=timestamp dateFormat=rfc3339)
constant(value= )
property(name=fromhost)
constant(value= )
property(name=hostname)
constant(value= )
property(name=syslogtag)
property(name=msg spifno1stsp=on)
}


The original syslog packet is missing the hostname, so I'm injecting that and
rebuilding the rest of the syslog structure based on debug output.  I just
don't want the FQDN :)


Thanks,
Chip


--
--
 Warning 
This e-mail message, without warrant or warning, and despite US law as set
forth in the Foreign Intelligence Surveillance Act of 1978, may be subject
to monitoring by the United States National Security Agency and/or the
Department of Defense. Information contained in this message may be used
against any senders or recipients, now or in the future, in a public trial
or secret tribunal.
  Please encrypt anything important.
   PGP Key: http://wwwkeys.pgp.net:11371/pks/lookup?op=getsearch=0x6CFA486D
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] v7.4.6 severe backlogs; need tuning help

[Chris Bartram]

Rainer;

  I tried adding the:
$actionqueuehighwatermark 8
$actionqueuelowwatermark 7

  as well as David's suggestion of increasing the BatchSize and things 
immediately got much better - but not fixed.

  And more interesting - I started monitoring the real-time output of my 
scripts as they read and processed the data they are being fed. Even as rsyslog 
is creating (and not deleting) what I assume are disc-assist files in the 
rsyslog-work-directory all the 5 script processes were processing records in 
real-time - never getting more than a few seconds behind. True to the increased 
batch size I could see occasional large bursts of records coming in to each 
script - which each processed quickly then sat there idle occasionally waiting 
for the next batch... I watched this for several hours and every one of the 
scripts was seeing real-time data... The scripts all log the timestamp from the 
syslog record they are reading as well as the current wall time (so I could 
monitor throughput and make adjustments if there started to be large gaps in 
the timestamps).

  Which leads me to conclude that either;
 1) some data is getting missed or processed out of order? Though the impstats 
state nothing has been discarded anywhere - or -
 2) data really is flowing to the scripts at speed, but for some reason rsyslog 
isn't cleaning up the disc files it's creating. It DOES delete some - I watched 
file lists and there would be 001-010 or similar, and sometime later files 
003-0022 or something... Always creating more than it deleted though.

With the increases above new files were being added much more slowly than 
before (earlier these files would start getting created within seconds of 
restarting rsyslog; after the changes it was over 30 minutes before I saw a 
single file created - and by the time left work several hours later there were 
about 90 (large) files in the working directory... Before the changes in the 
same time period there were several hundred.

I also ran the iostat command several times through the day. Not sure what 
reasonable numbers are, but the write Kb/s column would hit 100-140 as it was 
running. Otherwise the numbers didn't seem outrageous (I forgot to send myself 
any samples to include but I'll do that tomorrow if needed.

 -Chris Bartram 

The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)


On Tue, 11/19/13, Rainer Gerhards rgerha...@hq.adiscon.com wrote:

 Subject: Re: [rsyslog] v7.4.6 severe backlogs; need tuning help
 To: rsyslog-users rsyslog@lists.adiscon.com
 Date: Tuesday, November 19, 2013, 10:06 AM
 
 On Tue, Nov 19, 2013 at 1:27 PM,
 Chris Bartram chrisrbart...@yahoo.comwrote:
 
  Running Rsyslog 7.4.6 on a RHEL5 system.
 
  Since update to 7.4.6 (and possibly related to higher
 incoming traffic
  levels as well) the scripts that process incoming
 messages per action queue
  rapidly fall further behind; yet they seem to not be
 getting passed data
  frequently and CPU usage on the server is averaging
 less than 10%.It
  appears to me that incoming traffic is being processed
 at full-speed and
  data seems to pour into the action queue work files,
 but isn't getting sent
  out to the pipe files very quickly.
 
  Action queue files (in the “$WorkDirectory”) are
 building up so rapidly
  that the file system has filled up once on me already.
 “ps” commands always
  only show one rsyslogd process; perhaps I need to set
 it up somehow to use
  a process per action queue?
 
  I have this system setup so that it splits incoming
 streams by category
  and pipes each stream to a script so I can get some
 parallel processing.
  The scripts decide if the message is action-worthy and
 if so generate email
  alerts as applicable. The process has been running
 pretty well, average
  load is about 2M messages/day so far and the script has
 been keeping up
  with traffic (most messages processed within 1 second
 of arrival)..I'm
  guessing that I'm single-threading somewhere?
 
  Recording impstats hourly; last 2 hours below:
 
  Mon Nov 18 11:41:53 2013: imuxsock: submitted=1143
 ratelimit.discarded=0
  ratelimit.numratelimiters=269
  Mon Nov 18 11:41:53 2013: action 1: processed=353723
 failed=0
  Mon Nov 18 11:41:53 2013: action 2: processed=91661
 failed=0
  Mon Nov 18 11:41:53 2013: action 3: processed=39894
 failed=0
  Mon Nov 18 11:41:53 2013: action 4: processed=105490
 failed=0
  Mon Nov 18 11:41:53 2013: action 5: processed=4129
 failed=0
  Mon Nov 18 11:41:53 2013: action 6: processed=112549
 failed=0
  Mon Nov 18 11:41:53 2013: imudp(*:514): submitted=0
  Mon Nov 18 11:41:53 2013: imudp(*:514):
 submitted=272787
  Mon Nov 18 11:41:53 2013: imptcp(*/514/IPv6):
 submitted=0
  Mon Nov 18 11:41:53 2013: imptcp(*/514/IPv4):
 submitted=79790
  Mon Nov 18 11:41:53 2013: action 2 queue[DA

[rsyslog] Relp/tls setup in v7.4.6

[Chris Bartram]

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Relp/tls setup in v7.4.6

[Chris Bartram]

Sorry -yahoo/phone problems apparently;

  Trying to get relp/tls working but hitting some errors. Version is 7.4.6 and 
I’m getting the following errors when I start up rsyslog. Am I out of luck with 
relp/tls on this version? This was thee latest stable release yum found for 
my RHEL5 box when pointed at the rsyslog repo.
 
error during parsing file /etc/rsyslog.conf, on or before line 66: parameter 
'tls' not known -- typo in config file?
error during parsing file /etc/rsyslog.conf, on or before line 66: parameter 
'tls.authMode' not known -- typo in config file?
 

Below is the relevant part of the config file. Line 66 points to the closing 
“)” after the “input” line below:
 
module(load=imrelp# provides RELP (Reliable 
Extended Logging Protocol) support
  )

input(type=imrelp  # Setup RELP (tls) server on 
TCP/20514
  port=20514
  tls=on
  tls.authMode=name
  )

Thanks,
 Chris Bartram
 
The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)


On Thu, 11/14/13, David Lang da...@lang.hm wrote:

 Subject: Re: [rsyslog] Relp/tls setup in v7.4.6
 To: chrisrbart...@yahoo.com
 Date: Thursday, November 14, 2013, 8:46 PM
 
 There was no text in this post.
 
 David Lang
 
 On Thu, 14 Nov 2013, Chris Bartram wrote:
 
  Date: Thu, 14 Nov 2013 10:02:13 -0800 (PST)
  From: Chris Bartram chrisrbart...@yahoo.com
  Reply-To: rsyslog-users rsyslog@lists.adiscon.com
  To: rsyslog@lists adiscon. com rsyslog@lists.adiscon.com
  Subject: [rsyslog] Relp/tls setup in v7.4.6
  
 
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards
  NOTE WELL: This is a PUBLIC mailing list, posts are
 ARCHIVED by a myriad of sites beyond our control. PLEASE
 UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
 
 
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] embed missing HOSTNAME in syslog

[Chris 'Chipper' Chiapusio]

The application is delivering the logs to localhost on port 514/udp.

I've figured out how to get the logs right on the receiving rsyslog server,
but I do need to send to one other destination that I do not have control of
and would like to send them a properly formated log.

Is it possible to utilize system shell environment variables or shell execs
to acquire the local hostname (hostname -s) for use in a %HOSTNAME:::%
substitution?

Chip


On Tue, Oct 08, 2013 at 08:45:25AM -0700, David Lang wrote:
how are you getting the logs into rsyslog? is your app sending them 
to localhost port 514 UDP? writing them to /dev/log? something else?


David Lang

On Sat, 5 Oct 2013, Chris 'Chipper' Chiapusio wrote:



rsyslog is not inserting a hostname, the central log server 
(rsyslog V7) is using the

first word as the hostname (and creating fun dynamic directories with them)

Chip

On Fri, Oct 04, 2013 at 04:52:24PM -0700, David Lang wrote:
When rsyslog sends it out, it will send it with a hostname in the 
message. What arrives on the remote machine if you don't do 
anything, just send it?


David Lang

On Fri, 4 Oct 2013, Chris 'Chipper' Chiapusio wrote:

I have an application that can send syslog, however it does not 
include the
hostname in the syslog message.  I am sending the syslog to 
localhost running

rsyslog 3.22.1 (RHEL5.x stock) and want to embed the hostname into the log
messages prior to forwarding them on to their final destination.

I'm just not clear on how to format the property replacer, or 
if there is a
built-in variable I can use to stuff the hostname into the 
property replacer.



debug log demonstrating the missing hostname data:

6698.153096000:imudp.c: Listening on UDP syslogd socket 4 (IPv4/port 514).
6698.15310:imudp.c: imUDP calling select, active 
file descriptors

(max 4): 4
6698.15316:main queue:Reg/w0: main queue: entering rate limiter
6698.153178000:main queue:Reg/w0: main queue: entry deleted, state 0, size
now 0 entries
6698.153186000:main queue:Reg/w0: Called action, logging to builtin-fwd
6698.153193000:main queue:Reg/w0: action 9 queue: entry added, size now 1
entries
6698.153202000:main queue:Reg/w0: wtpAdviseMaxWorkers signals busy
6698.153209000:main queue:Reg/w0: action 9 queue: EnqueueMsg 
advised worker

start
6698.153215000:main queue:Reg/w0: Called action, logging to builtin-file
6698.153228000:main queue:Reg/w0:  (/var/log/local6)
6698.15324:action 9 queue:Reg/w0: action 9 queue: entering 
rate limiter
6698.153251000:main queue:Reg/w0: Called action, logging to 
builtin-discard

6698.153265000:main queue:Reg/w0:
6698.153271000:main queue:Reg/w0: main queue: entering rate limiter
6698.153276000:main queue:Reg/w0: main queue:Reg/w0: worker 
IDLE, waiting for

work.
6698.15330:action 9 queue:Reg/w0: action 9 queue: entry 
deleted, state 0,

size now 0 entries
6698.153324000:action 9 queue:Reg/w0:  mxloghost
6698.15333:action 9 queue:Reg/w0:  mxloghost:514/tcp
6698.153342000:action 9 queue:Reg/w0: TCP sent 78 bytes, requested 78
6698.15335:action 9 queue:Reg/w0: action 9 queue: entering 
rate limiter

6698.153356000:action 9 queue:Reg/w0: action 9 queue:Reg/w0: worker IDLE,
waiting for work.
6698.154521000:imudp.c: Message from inetd socket: #4, host:
localhost.localdomain
6698.154538000:imudp.c: logmsg: flags 0, from 'localhost.localdomain', msg
Oct  4 19:58:18 filter_instance1 debg s=1ey2g78qfq mod=session cmd=macros
data=j duration=0.000
6698.154543000:imudp.c: Message has legacy syslog format.
6698.15455:imudp.c: main queue: entry added, size now 1 entries
6698.154564000:imudp.c: wtpAdviseMaxWorkers signals busy
6698.15457:imudp.c: main queue: EnqueueMsg advised worker start


Thanks,
Chip


--
--
 Warning 
This e-mail message, without warrant or warning, and despite US law as set
forth in the Foreign Intelligence Surveillance Act of 1978, may be subject
to monitoring by the United States National Security Agency and/or the
Department of Defense. Information contained in this message may be used
against any senders or recipients, now or in the future, in a public trial
or secret tribunal.
  Please encrypt anything important.
   PGP Key: http://wwwkeys.pgp.net:11371/pks/lookup?op=getsearch=0x6CFA486D
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] embed missing HOSTNAME in syslog

[Chris 'Chipper' Chiapusio]

I have an application that can send syslog, however it does not include the
hostname in the syslog message.  I am sending the syslog to localhost running
rsyslog 3.22.1 (RHEL5.x stock) and want to embed the hostname into the log
messages prior to forwarding them on to their final destination.

I'm just not clear on how to format the property replacer, or if there is a
built-in variable I can use to stuff the hostname into the property replacer.


debug log demonstrating the missing hostname data:

6698.153096000:imudp.c: Listening on UDP syslogd socket 4 (IPv4/port 514).
6698.15310:imudp.c: imUDP calling select, active file descriptors
(max 4): 4
6698.15316:main queue:Reg/w0: main queue: entering rate limiter
6698.153178000:main queue:Reg/w0: main queue: entry deleted, state 0, size
now 0 entries
6698.153186000:main queue:Reg/w0: Called action, logging to builtin-fwd
6698.153193000:main queue:Reg/w0: action 9 queue: entry added, size now 1
entries
6698.153202000:main queue:Reg/w0: wtpAdviseMaxWorkers signals busy
6698.153209000:main queue:Reg/w0: action 9 queue: EnqueueMsg advised worker
start
6698.153215000:main queue:Reg/w0: Called action, logging to builtin-file
6698.153228000:main queue:Reg/w0:  (/var/log/local6)
6698.15324:action 9 queue:Reg/w0: action 9 queue: entering rate limiter
6698.153251000:main queue:Reg/w0: Called action, logging to builtin-discard
6698.153265000:main queue:Reg/w0:
6698.153271000:main queue:Reg/w0: main queue: entering rate limiter
6698.153276000:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for
work.
6698.15330:action 9 queue:Reg/w0: action 9 queue: entry deleted, state 0,
size now 0 entries
6698.153324000:action 9 queue:Reg/w0:  mxloghost
6698.15333:action 9 queue:Reg/w0:  mxloghost:514/tcp
6698.153342000:action 9 queue:Reg/w0: TCP sent 78 bytes, requested 78
6698.15335:action 9 queue:Reg/w0: action 9 queue: entering rate limiter
6698.153356000:action 9 queue:Reg/w0: action 9 queue:Reg/w0: worker IDLE,
waiting for work.
6698.154521000:imudp.c: Message from inetd socket: #4, host:
localhost.localdomain
6698.154538000:imudp.c: logmsg: flags 0, from 'localhost.localdomain', msg
Oct  4 19:58:18 filter_instance1 debg s=1ey2g78qfq mod=session cmd=macros
data=j duration=0.000
6698.154543000:imudp.c: Message has legacy syslog format.
6698.15455:imudp.c: main queue: entry added, size now 1 entries
6698.154564000:imudp.c: wtpAdviseMaxWorkers signals busy
6698.15457:imudp.c: main queue: EnqueueMsg advised worker start


Thanks,
Chip


--
--
 Warning 
This e-mail message, without warrant or warning, and despite US law as set
forth in the Foreign Intelligence Surveillance Act of 1978, may be subject
to monitoring by the United States National Security Agency and/or the
Department of Defense. Information contained in this message may be used
against any senders or recipients, now or in the future, in a public trial
or secret tribunal.
  Please encrypt anything important.
   PGP Key: http://wwwkeys.pgp.net:11371/pks/lookup?op=getsearch=0x6CFA486D
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] embed missing HOSTNAME in syslog

[Chris 'Chipper' Chiapusio]

rsyslog is not inserting a hostname, the central log server (rsyslog V7) is 
using the
first word as the hostname (and creating fun dynamic directories with them)

Chip

On Fri, Oct 04, 2013 at 04:52:24PM -0700, David Lang wrote:
When rsyslog sends it out, it will send it with a hostname in the 
message. What arrives on the remote machine if you don't do anything, 
just send it?


David Lang

On Fri, 4 Oct 2013, Chris 'Chipper' Chiapusio wrote:


I have an application that can send syslog, however it does not include the
hostname in the syslog message.  I am sending the syslog to localhost running
rsyslog 3.22.1 (RHEL5.x stock) and want to embed the hostname into the log
messages prior to forwarding them on to their final destination.

I'm just not clear on how to format the property replacer, or if there is a
built-in variable I can use to stuff the hostname into the property replacer.


debug log demonstrating the missing hostname data:

6698.153096000:imudp.c: Listening on UDP syslogd socket 4 (IPv4/port 514).
6698.15310:imudp.c: imUDP calling select, active file descriptors
(max 4): 4
6698.15316:main queue:Reg/w0: main queue: entering rate limiter
6698.153178000:main queue:Reg/w0: main queue: entry deleted, state 0, size
now 0 entries
6698.153186000:main queue:Reg/w0: Called action, logging to builtin-fwd
6698.153193000:main queue:Reg/w0: action 9 queue: entry added, size now 1
entries
6698.153202000:main queue:Reg/w0: wtpAdviseMaxWorkers signals busy
6698.153209000:main queue:Reg/w0: action 9 queue: EnqueueMsg advised worker
start
6698.153215000:main queue:Reg/w0: Called action, logging to builtin-file
6698.153228000:main queue:Reg/w0:  (/var/log/local6)
6698.15324:action 9 queue:Reg/w0: action 9 queue: entering rate limiter
6698.153251000:main queue:Reg/w0: Called action, logging to builtin-discard
6698.153265000:main queue:Reg/w0:
6698.153271000:main queue:Reg/w0: main queue: entering rate limiter
6698.153276000:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for
work.
6698.15330:action 9 queue:Reg/w0: action 9 queue: entry deleted, state 0,
size now 0 entries
6698.153324000:action 9 queue:Reg/w0:  mxloghost
6698.15333:action 9 queue:Reg/w0:  mxloghost:514/tcp
6698.153342000:action 9 queue:Reg/w0: TCP sent 78 bytes, requested 78
6698.15335:action 9 queue:Reg/w0: action 9 queue: entering rate limiter
6698.153356000:action 9 queue:Reg/w0: action 9 queue:Reg/w0: worker IDLE,
waiting for work.
6698.154521000:imudp.c: Message from inetd socket: #4, host:
localhost.localdomain
6698.154538000:imudp.c: logmsg: flags 0, from 'localhost.localdomain', msg
Oct  4 19:58:18 filter_instance1 debg s=1ey2g78qfq mod=session cmd=macros
data=j duration=0.000
6698.154543000:imudp.c: Message has legacy syslog format.
6698.15455:imudp.c: main queue: entry added, size now 1 entries
6698.154564000:imudp.c: wtpAdviseMaxWorkers signals busy
6698.15457:imudp.c: main queue: EnqueueMsg advised worker start


Thanks,
Chip




___


--
--
 Warning 
This e-mail message, without warrant or warning, and despite US law as set
forth in the Foreign Intelligence Surveillance Act of 1978, may be subject
to monitoring by the United States National Security Agency and/or the
Department of Defense. Information contained in this message may be used
against any senders or recipients, now or in the future, in a public trial
or secret tribunal.
  Please encrypt anything important.
   PGP Key: http://wwwkeys.pgp.net:11371/pks/lookup?op=getsearch=0x6CFA486D
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Very high throughput options

[Chris Bartram]

We are in the planning stages of setting up a rsyslog server pool to 
accommodate syslog streams from a couple thousand *nix servers; including 
auditd type data and potentially some application logs (so it's going to be a 
VERY high volume of data) and we're looking to archive this data somewhere.We 
have a 10Gb network infrastructure, and I can throw as many RHEL machines at it 
as needed (as well as F5 load balancers in front).

Eventually the data may need to be searched, but highest priority is getting it 
written somewhere quickly (and reliably - we need to minimize any possible data 
loss so our archives can stand up to auditing requirements). In that regard, 
any suggestions on file systems that can handle that kind of load? Ideally we 
want all the log files written to the same storage somewhere - i.e. we don't 
want to have to consolidate files from separate locations to search all the log 
files for some specific host. On the other hand we can split up load by subnet 
sources perhaps and route specific machines to specific rsyslog clusters to 
ease the load on any one cluster (though our larger subnets still may have 
around 1,000 systems reporting); as long as it's easy to identify where to look 
for data from a given host.


I welcome any advice on setups that allow multiple concurrent (active) rsyslog 
servers writing to a common-ish file system as well as any gotchas or 
performance benchmarks we can use to help plan the system.

Thanks,
 Chris Bartram
 
The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] trouble adding relp to existing server

[Chris Bartram]

Wow. Thanks all. Sad that the official RHEL repository is so far behind... 

I'll see about linking to the rsyslog repository.

-Chris Bartram


 
The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)



 From: Rainer Gerhards rgerha...@hq.adiscon.com
To: rsyslog-users rsyslog@lists.adiscon.com 
Sent: Wednesday, April 3, 2013 4:33 AM
Subject: Re: [rsyslog] trouble adding relp to existing server
 


 -Original Message-
 From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
 boun...@lists.adiscon.com] On Behalf Of David Lang
 Sent: Wednesday, April 03, 2013 10:06 AM
 To: Chris Bartram; rsyslog-users
 Subject: Re: [rsyslog] trouble adding relp to existing server
 
 On Tue, 2 Apr 2013, Chris Bartram wrote:
 
  On a RHEL 5 system I have an existing server where I have basic UDP and
 encrypted tls transports setup. I'm now trying to add RELP but even after
 adding the librelp packages I get an error from rsyslog complaining that it
 can't open imrelp.so. In fact there is no imrelp.so* anywhere on the
 system?.
 
  Sticking to standard yum install packages, since although this is the 
  server,
 I'm going to need to setup RELP clients on 500+ systems, and I need this to 
 be
 as standardized as possible.
 
 rsyslog 3.22 is downright ancient

Oops... I overlooked that. Yeah, could very probably be no relp in that 
version.

Rainer
(7.4 is due to be released in a week or so).
 You really should go with newer packages (It's very possible that RHEL5
 packages don't include relp support)
 
 I believe that in RHEL5.9 or 5.10 they added a new, optional rsyslog package
 that is 5.x, still old, but much better than 3.22
 
 In addition to that option, there are CentOS/RHEL packages at
 http://www.rsyslog.com/rhelcentos-rpms/ Add the appropriate repository
 here to your yum configuration and you can then essentually forget that
 these aren't in the base RHEL repository.
 
 David Lang
 
 
  uname -a
  Linux hostname 2.6.18-308.24.1.el5 #1 SMP Wed Nov 21 11:42:14 EST 2012
  x86_64 x86_64 x86_64 GNU/Linux
 
  yum list | grep \(rsyslog\|relp\|tls\)
  gnutls.x86_64   1.4.1-7.el5_8.2   installed
  gnutls-utils.x86_64 1.4.1-7.el5_8.2   installed
  librelp.i386    0.1.1-2.el5   installed
  librelp.x86_64  0.1.1-2.el5   installed
  librelp-devel.i386  0.1.1-2.el5   installed
  librelp-devel.x86_64    0.1.1-2.el5   installed
  rsyslog.x86_64  3.22.1-7.el5  installed
  rsyslog-gnutls.x86_64   3.22.1-7.el5  installed
 
 
  Rsyslog restart:
 
  rsyslogd: [origin software=rsyslogd swVersion=3.22.1 x-pid=16187
  x-info=http://www.rsyslog.com;] (re)start
 
  rsyslogd-2066:could not load module '/lib64/rsyslog/imrelp.so',
  dlopen: /lib64/rsyslog/imrelp.so: cannot open shared object file: No
  such file or directory [try http://www.rsyslog.com/e/2066 ]
 
  Thanks in advance,
 
   -Chris Bartram
 
 
  The purpose of life is not to be happy. It is to be useful, to be
  honorable, to be compassionate, to have it make some difference that
  you have lived and lived well. (Ralph Waldo Emerson)
  ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
  WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
  sites
 beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
 LIKE THAT.
 
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.



___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] trouble adding relp to existing server

[Chris Bartram]

On a RHEL 5 system I have an existing server where I have basic UDP and 
encrypted tls transports setup. I'm now trying to add RELP but even after 
adding the librelp packages I get an error from rsyslog complaining that it 
can't open imrelp.so. In fact there is no imrelp.so* anywhere on the system?.

Sticking to standard yum install packages, since although this is the server, 
I'm going to need to setup RELP clients on 500+ systems, and I need this to be 
as standardized as possible.

uname -a
Linux hostname 2.6.18-308.24.1.el5 #1 SMP Wed Nov 21 11:42:14 EST 2012 x86_64 
x86_64 x86_64 GNU/Linux

yum list | grep \(rsyslog\|relp\|tls\)
gnutls.x86_64   1.4.1-7.el5_8.2   installed
gnutls-utils.x86_64 1.4.1-7.el5_8.2   installed
librelp.i386    0.1.1-2.el5   installed
librelp.x86_64  0.1.1-2.el5   installed
librelp-devel.i386  0.1.1-2.el5   installed
librelp-devel.x86_64    0.1.1-2.el5   installed
rsyslog.x86_64  3.22.1-7.el5  installed
rsyslog-gnutls.x86_64   3.22.1-7.el5  installed


Rsyslog restart:

rsyslogd: [origin software=rsyslogd swVersion=3.22.1 x-pid=16187 
x-info=http://www.rsyslog.com;] (re)start

rsyslogd-2066:could not load module '/lib64/rsyslog/imrelp.so', dlopen: 
/lib64/rsyslog/imrelp.so: cannot open shared object file: No such file or 
directory [try http://www.rsyslog.com/e/2066 ]

Thanks in advance, 

 -Chris Bartram

 
The purpose of life is not to be happy. It is to be useful, to be honorable, 
to be compassionate, to have it make some difference that you have lived and 
lived well. (Ralph Waldo Emerson)
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] MongoDB PHP Driver Extensions is not installed

[Chris Roberts]

Andre,

Thanks again for your help. It looks like I edited the php.ini under
/etc/php5/cli/php.ini instead of the one in the path you specified. I added
extension=mongo.so under the Dynamic Extensions section of the file, saved
it, then restarted apache2 and it works!


Chris

On Wed, Mar 20, 2013 at 6:04 AM, Andre Lorbach alorb...@ro1.adiscon.comwrote:

 Perhaps you got the wrong php.ini? There is one for apache only usually
 located at /etc/php5/apache2/php.ini

 Best regards,
 Andre Lorbach

  -Original Message-
  From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
  boun...@lists.adiscon.com] On Behalf Of Chris Roberts
  Sent: Dienstag, 19. März 2013 15:36
  To: rsyslog@lists.adiscon.com
  Subject: [rsyslog] MongoDB PHP Driver Extensions is not installed
 
  Hello,
 
  I have finished the configuration of my syslog server, but after going
 through
  the loganalyzer setup, I receive the message:
 
  Error, MongoDB PHP Driver Extensions is not installed! Please see
 *website*
 
  I did perform (sudo pecl install mongo) and added extension=mongo.so
  under the Dynamic Extensions in the file php.ini, but I'm still
 receiving the
  message.
 
 
  Is there a step that I'm missing?
 
 
 
  --
  Chris Roberts
  IT Professional
  Budd Baer, Inc
  71 Murtland Ave
  Washington, PA 15301
  Phone: 724-222-0700 Ext: 6601
  Fax: 724-914-6633
  http://www.buddbaer.com/
 
  This message and any attachments are intended only for the use of the
  addressee and may contain information that is privileged and
 confidential. If
  the reader of the message is not the intended recipient or an authorized
  representative of the intended recipient, you are hereby notified that
 any
  dissemination of this communication is strictly prohibited. If you have
  received this communication in error, please notify us immediately by
 e-mail
  and delete the message and any attachments from your system.
 
  Think before you print. Please consider the environment before printing
 this
  e-mail ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL:
  This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
 beyond
  our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.




-- 
Chris Roberts
IT Professional
Budd Baer, Inc
71 Murtland Ave
Washington, PA 15301
Phone: 724-222-0700 Ext: 6601
Fax: 724-914-6633
http://www.buddbaer.com/

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

Think before you print. Please consider the environment before printing this 
e-mail

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] MongoDB PHP Driver Extensions is not installed

[Chris Roberts]

Hello,

I have finished the configuration of my syslog server, but after going
through the loganalyzer setup, I receive the message:

Error, MongoDB PHP Driver Extensions is not installed! Please see *website*

I did perform (sudo pecl install mongo) and added extension=mongo.so under
the Dynamic Extensions in the file php.ini, but I'm still receiving the
message.


Is there a step that I'm missing?



-- 
Chris Roberts
IT Professional
Budd Baer, Inc
71 Murtland Ave
Washington, PA 15301
Phone: 724-222-0700 Ext: 6601
Fax: 724-914-6633
http://www.buddbaer.com/

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

Think before you print. Please consider the environment before printing this 
e-mail
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Ubuntu 12.04 LTS rsyslog + mongodb + loganalyzer install (Need peer review!)

[Chris Roberts]

It's the bothersome IT guy again!

Anyways, I finally was able to get loganalyzer to work with rsyslog  mongo
DB on Ubuntu 12.04 LTS. I would like to get a peer review of the steps I
took to see if there is a better way or if I am missing something. So
here's the code:


1. Install Ubuntu 12.04 Server

2. Configure static IP (sudo nano /etc/network/interfaces)

3. sudo nano /etc/apt/sources.list (uncomment deb and deb-src for extras
and partners repositories)

4. sudo apt-get update

5. sudo apt-get upgrade

6. sudo apt-get dist-upgrade

7. Packages needed (use apt-get install for these):

pkg-config build-essential autoconf uuid uuid-dev libgtk2.0-dev libperl-dev
mongodb mongodb-server php-pear apache2 php5

8. sudo nano /etc/apache2/conf.d/fqdn (add Servername localhost)

9. sudo /etc/init.d/apache2 restart

10. Edit php.ini. This was found in /etc/php5/cli/php.ini. Under Dynamic
Extensions, create extension=mongo.so

11. wget libestr.adiscon.com/files/download/libestr-0.1.4.tar.gz

12. tar xzvf libestr-0.1.4.tar.gz -C /tmp/

13. cd /tmp/libestr-0.1.4

14. ./configure --libdir=/usr/lib --includedir=/usr/include --prefix=/usr

15. make

16. sudo make install

17. cd ~

18. clear

19. wget http://www.libee.org/download/files/download/libee-0.4.1.tar.gz

20. tar xzvf libee-0.4.1.tar.gz -C /tmp/

21. cd /tmp/libee-0.4.1

22. ./configure --libdir=/usr/lib --includedir=/usr/include --prefix=/usr

23. make

24. make install

25. cd ~

26. clear

27. wget http://www.liblognorm.com/files/download/liblognorm-0.3.5.tar.gz

28. tar xzvf liblognorm-0.3.5.tar.gz -C /tmp/

29. cd /tmp/liblognorm-0.3.5

30. ./configure --libdir=/usr/lib --includedir=/usr/include --prefix=/usr

31. make

32. sudo make install

33. cd ~

34. clear

35. wget https://github.com/downloads/json-c/json-c/json-c-0.10.tar.gz

36. tar xzvf json-c-0.10.tar.gz -C /tmp/

37. cd /tmp/json-c-0.10

38. ./autogen.sh

39. ./configure --libdir=/usr/lib --includedir=/usr/include
--sbindir=/usr/sbin --prefix=/usr

40. make

41. sudo make install

42. cp -vvv /tmp/json-c-0.10/json_object_iterator.h /usr/include/json

43. cd ~

44. clear

45. wget
http://archive.ubuntu.com/ubuntu/pool/universe/libm/libmongo-client/libmongo-client_0.1.5.orig.tar.gz

46. tar xzvf libmongo-client_0.1.5.orig.tar.gz-C /tmp/

47. cd /tmp/libmongo-client-0.1.5

48. ./autogen.sh

49. ./configure --libdir=/usr/lib --includedir=/usr/include --prefix=/usr

50. make

51. sudo make install

52. cd ~

53. clear

54. wget http://www.rsyslog.com/files/download/rsyslog/rsyslog-7.2.6.tar.gz

55. tar xzvf rsyslog-7.2.6.tar.gz -C /tmp/

56. cd /tmp/rsyslog-7.2.6

57. ./configure  --prefix=/usr --enable-imtcp --enable-mmjsonparse
--enable-ommongodb

58. make

59. sudo make install

60. cd ~

61. clear

62. wget http://download.adiscon.com/loganalyzer/loganalyzer-3.6.3.tar.gz

63. tar xzvf loganalyzer-3.6.3.tar.gz -C /tmp/

64. cd /tmp/loganalyzer-3.6.3

65. sudo mkdir -p /var/www/html/loganalyzer

66. sudo cp -R src/* /var/www/html/loganalyzer

67. sudo cp -R contrib/* /var/www/html/loganalyzer

68. cd /var/www/loganalyzer

69. sudo chmod +x configure.sh secure.sh

70. sudo ./configure.sh

71. cd ~

72. clear

73. Open web browser and go to server-ip/loganalyzer to complete the setup!


I apologize for such a long e-mail. Like always, any feedback is greatly
appreciated!


Thanks,
-- 
Chris Roberts
IT Professional
Budd Baer, Inc
71 Murtland Ave
Washington, PA 15301
Phone: 724-222-0700 Ext: 6601
Fax: 724-914-6633
http://www.buddbaer.com/

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

Think before you print. Please consider the environment before printing this 
e-mail
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] LogAnalyzer install

[Chris Roberts]

I'm almost done with a complete installation how-to for Ubuntu 12.04 Server
and need some help regarding the INSTALL file instructions in Step 1 where
it states:

1. Upload all files from the loganalyzer/src/ folder to you webserver. The
other files are not need on the webserver

I have installed Apache2  php5 on the Ubuntu server I'm building, but I'm
still a bit new to webservers. Does the first instruction indicate that if
the webserver is on the same machine as the syslog server, do I just need
to run ./configure with the appropriate options?

Also, once I figure that part out, I'll provide everyone with the doc I've
built over the past few days. It's pretty helpful for those new to Linux
(including myself)




Thanks,


-- 
Chris Roberts
IT Professional
Budd Baer, Inc
71 Murtland Ave
Washington, PA 15301
Phone: 724-222-0700 Ext: 6601
Fax: 724-914-6633
http://www.buddbaer.com/

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

Think before you print. Please consider the environment before printing this 
e-mail
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] libestr version requirements not met

[Chris Roberts]

Andre,

I just ran ./configure without any options.

On Wed, Mar 6, 2013 at 3:33 AM, Andre Lorbach alorb...@ro1.adiscon.comwrote:

 Have you configured with libee with --prefix=/usr as well?

 Regards,
 Andre

  -Original Message-
  From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
  boun...@lists.adiscon.com] On Behalf Of Chris Roberts
  Sent: Dienstag, 5. März 2013 20:32
  To: rsyslog@lists.adiscon.com
  Subject: [rsyslog] libestr version requirements not met
 
  Hello again!
 
  I'm running into a problem when I run ./configure --prefix=/usr --enable-
  imtcp --enable-mmjsonparse --enable-ommongodb while in the directory
  user@server:/tmp/rsyslog-7.2.6 (where I extracted the tar.gz).
  Anyways, When I run the command, it will generate
 
  error: Package requirements (libestr = 0.1.2) were not met:
 
  Requested 'libestr = 0.1.2' but version of libestr is 0.1.1
 
  After receiving that error, I proceeded to wget the latest version of
 libestr at
  libestr.adiscon.com/files/download/libestr-0.1.4.tar.gz,
  extract it to the /tmp directory and run ./configure from there. No error
  messages were generated when I did that. Although I acquired the latest
  version of libestr, it is still showing that my installed version is
 0.1.1.
 
 
  Any ideas as to what I may have done wrong?
 
  --
  Chris Roberts
  IT Professional
  Budd Baer, Inc
  71 Murtland Ave
  Washington, PA 15301
  Phone: 724-222-0700 Ext: 6601
  Fax: 724-914-6633
  http://www.buddbaer.com/
 
  This message and any attachments are intended only for the use of the
  addressee and may contain information that is privileged and
 confidential. If
  the reader of the message is not the intended recipient or an authorized
  representative of the intended recipient, you are hereby notified that
 any
  dissemination of this communication is strictly prohibited. If you have
  received this communication in error, please notify us immediately by
 e-mail
  and delete the message and any attachments from your system.
 
  Think before you print. Please consider the environment before printing
 this
  e-mail ___
  rsyslog mailing list
  http://lists.adiscon.net/mailman/listinfo/rsyslog
  http://www.rsyslog.com/professional-services/
  What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL:
  This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
 beyond
  our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.




-- 
Chris Roberts
IT Professional
Budd Baer, Inc
71 Murtland Ave
Washington, PA 15301
Phone: 724-222-0700 Ext: 6601
Fax: 724-914-6633
http://www.buddbaer.com/

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

Think before you print. Please consider the environment before printing this 
e-mail

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] libestr version requirements not met

[Chris Roberts]

Hello again!

I'm running into a problem when I run ./configure --prefix=/usr
--enable-imtcp --enable-mmjsonparse --enable-ommongodb while in the
directory user@server:/tmp/rsyslog-7.2.6 (where I extracted the tar.gz).
Anyways, When I run the command, it will generate

error: Package requirements (libestr = 0.1.2) were not met:

Requested 'libestr = 0.1.2' but version of libestr is 0.1.1

After receiving that error, I proceeded to wget the latest version of
libestr at libestr.adiscon.com/files/download/libestr-0.1.4.tar.gz,
extract it to the /tmp directory and run ./configure from there. No error
messages were generated when I did that. Although I acquired the latest
version of libestr, it is still showing that my installed version is 0.1.1.


Any ideas as to what I may have done wrong?

-- 
Chris Roberts
IT Professional
Budd Baer, Inc
71 Murtland Ave
Washington, PA 15301
Phone: 724-222-0700 Ext: 6601
Fax: 724-914-6633
http://www.buddbaer.com/

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

Think before you print. Please consider the environment before printing this 
e-mail
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Help with configuration of rsyslog on Ubuntu 12.04 LTS server

[Chris Roberts]

Hello!

I'm having some difficulties with adding 2 modules to rsyslog using Ubuntu
12.04 LTS. I am trying to follow the installation guide here:
http://loganalyzer.adiscon.com/articles/using-mongodb-with-rsyslog-and-loganalyzer,
but I am getting stuck when it tells me to add the modules mmjsonparse 
ommongodb using ./configure. Since rsyslog is already installed, is there
another way to add those modules without the ./configure method?

Any help would be appreciated!


Thanks,

-- 
Chris Roberts
IT Professional
Budd Baer, Inc
71 Murtland Ave
Washington, PA 15301
Phone: 724-222-0700 Ext: 6601
Fax: 724-914-6633
http://www.buddbaer.com/

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

Think before you print. Please consider the environment before printing this 
e-mail
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Help with configuration of rsyslog on Ubuntu 12.04 LTS server

[Chris Roberts]

If rsyslog is currently installed on Ubuntu Server by default, how would I
compile it again?

On Mon, Mar 4, 2013 at 2:26 PM, David Lang da...@lang.hm wrote:

 On Mon, 4 Mar 2013, Chris Roberts wrote:

  Hello!

 I'm having some difficulties with adding 2 modules to rsyslog using Ubuntu
 12.04 LTS. I am trying to follow the installation guide here:
 http://loganalyzer.adiscon.**com/articles/using-mongodb-**
 with-rsyslog-and-loganalyzerhttp://loganalyzer.adiscon.com/articles/using-mongodb-with-rsyslog-and-loganalyzer
 ,
 but I am getting stuck when it tells me to add the modules mmjsonparse 
 ommongodb using ./configure. Since rsyslog is already installed, is there
 another way to add those modules without the ./configure method?

 Any help would be appreciated!


 you will have to either compile rsyslog yourself, or install non-ubuntu
 packages that include the modules you need.

 David Lang
 __**_
 rsyslog mailing list
 http://lists.adiscon.net/**mailman/listinfo/rsysloghttp://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/**professional-services/http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
 of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
 DON'T LIKE THAT.




-- 
Chris Roberts
IT Professional
Budd Baer, Inc
71 Murtland Ave
Washington, PA 15301
Phone: 724-222-0700 Ext: 6601
Fax: 724-914-6633
http://www.buddbaer.com/

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

Think before you print. Please consider the environment before printing this 
e-mail
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Empty hostname field not matching

[Chris Picton]

Hi

I am using rsyslog 5.8.12, and I have templates as follows:

$template ServerMessages,/srv/log/Machines/%HOSTNAME%/%$YEAR%-%$MONTH
%-%$DAY%/messages.gz


Some of my equipment is not sending a hostname, so the logs go to
/srv/log/Machines/YY-MM-DD/messages.gz

In the above case, I would like to match a missing hostname and instead
use %fromhost-ip% instead for the folder name.


I have tried a few things to catch the logs with a missing hostname, so
I can categorise them correctly, but none of the following work:

:HOSTNAME, isequal,  ?TemplateDebug2;DebugFormat
:hostname, regex, ^\s*$ ?TemplateDebug2;DebugFormat
:hostname, ereregex, ^\s*$ ?TemplateDebug2;DebugFormat
if $hostname == ' then ?TemplateDebug2;DebugFormat

If, however, I match a specific source ip, the resulting log it written,
with a blank hostname:


$template TemplateDebug2,/srv/log/DEBUG2/%fromhost-ip%.gz
$template TemplateDebug3,/srv/log/DEBUG2/CUSTOM-%fromhost-ip%.gz

$template DebugFormat,ts=%TIMESTAMP% ip=%FROMHOST-IP% host=%HOSTNAME%
tag=%syslogtag% MSG=@%rawmsg%@\n


:fromhost, contains, 10.20.240.3 ?TemplateDebug3;DebugFormat

This results in:
ts=Sep 20 08:19:16 ip=10.20.240.30 host= tag=: MSG=@141: 2012 Sep 20
08:19:16.279 SAST: last message repeated 1 time@



I have attached a pcap of a similar message so you can see the exact
message being sent.



What is the best way to log to hostname based folders, falling back to
ip based folders if the hostname is not set?

Regards
Chris


syslog.pcap
Description: application/vnd.tcpdump.pcap
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Empty hostname field not matching

[Chris Picton]

Hi Rainer

I have set up a test server to extract a clean debug log, and am seeing
behaviour differences between the live and test servers

On the test server, the empty hostname fields are being matched
correctly, and logs are written where I expect them.  For this server, I
have copied the configs from the live server, and set up two devices to
log to both live and test.

On the live server when I have a lot more clients, the empty hostnames
are not always/reliably matched.

Should I send you the large debug file from the live server off list?

Chris

On Thu, 2012-09-20 at 06:33 +, Rainer Gerhards wrote:
 
  -Original Message-
  From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
  boun...@lists.adiscon.com] On Behalf Of Chris Picton
  Sent: Thursday, September 20, 2012 8:32 AM
  To: rsyslog@lists.adiscon.com
  Subject: [rsyslog] Empty hostname field not matching
  
  Hi
  
  I am using rsyslog 5.8.12, and I have templates as follows:
  
  $template ServerMessages,/srv/log/Machines/%HOSTNAME%/%$YEAR%-%$MONTH
  %-%$DAY%/messages.gz
  
  
  Some of my equipment is not sending a hostname, so the logs go to
  /srv/log/Machines/YY-MM-DD/messages.gz
  
  In the above case, I would like to match a missing hostname and instead
  use %fromhost-ip% instead for the folder name.
  
  
  I have tried a few things to catch the logs with a missing hostname, so
  I can categorise them correctly, but none of the following work:
  
  :HOSTNAME, isequal,  ?TemplateDebug2;DebugFormat
  :hostname, regex, ^\s*$ ?TemplateDebug2;DebugFormat
  :hostname, ereregex, ^\s*$ ?TemplateDebug2;DebugFormat
  if $hostname == ' then ?TemplateDebug2;DebugFormat
 
 Pls post debug log so that we can see if the hostname is actually empty.
 Rainer

OK - I will set up a test server quickly 

  
  If, however, I match a specific source ip, the resulting log it
  written,
  with a blank hostname:
  
  
  $template TemplateDebug2,/srv/log/DEBUG2/%fromhost-ip%.gz
  $template TemplateDebug3,/srv/log/DEBUG2/CUSTOM-%fromhost-ip%.gz
  
  $template DebugFormat,ts=%TIMESTAMP% ip=%FROMHOST-IP% host=%HOSTNAME%
  tag=%syslogtag% MSG=@%rawmsg%@\n
  
  
  :fromhost, contains, 10.20.240.3 ?TemplateDebug3;DebugFormat
  
  This results in:
  ts=Sep 20 08:19:16 ip=10.20.240.30 host= tag=: MSG=@141: 2012 Sep 20
  08:19:16.279 SAST: last message repeated 1 time@
  
  
  
  I have attached a pcap of a similar message so you can see the exact
  message being sent.
  
  
  
  What is the best way to log to hostname based folders, falling back to
  ip based folders if the hostname is not set?
  
  Regards
  Chris
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
 sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
 LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] rsyslog stops

[Chris Cheltenham (External - Capgemini America, Inc.)]

WE ARE USING RH 5.X.

After a log rotate, he syslogd stops.
Has anyone had this issue?
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

Re: [rsyslog] could not load module '/usr/local/lib/rsyslog/ommail.so

[Chris McCraw]

On Fri, May 4, 2012 at 11:25 AM, Jo Rhett jrh...@netconsonance.com wrote:
 What versions does your yum repo have?

 If you need a spec file, I can send you the spec file I have for EL5 which 
 will build the RPMs for 5.8.11 for you.

Any chance we could get that spec file into the contrib/ directory?  I
spent a fair amount of time finding a spec file for even vaguely
recent rsyslog that wasn't inextricably bound up with features that
EL5 didn't have, and ripping out all the stuff that referenced the new
init replacement.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

Re: [rsyslog] rsyslogd 5.8.5 + heavy message load + compression -failure mode?

[Chris McCraw]

On Thu, Apr 19, 2012 at 12:02 AM, Rainer Gerhards
rgerha...@hq.adiscon.com wrote:
 -Original Message-
 From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
 boun...@lists.adiscon.com] On Behalf Of Chris McCraw
 Sent: Thursday, April 19, 2012 2:18 AM
 To: rsyslog-users
 Subject: [rsyslog] rsyslogd 5.8.5 + heavy message load + compression -
 failure mode?

 Hi folks,

 I probably missed it, but after awhile searching the docs fruitlessly,
 I decided I'd ask the experts.

 We have an rsyslog server that handles a couple million messages (from
 a single remote server) per minute.  It logs these to several logfiles
 with a cpu load of about 40% of a single CPU core.  logrotate
 currently takes about 12 hours to single-threadedly, consecutively,
 compress and rotate these logs.  It's been suggested that we take
 logrotate out of the loop and just have rsyslog write compressed
 files.  This seems like a great idea, but I'm curious about how it
 scales, since I don't have a good test environment (just some
 underpowered VM's which don't seem to generate comparable load no
 matter how I try) to work with.

 Suppose that rsyslog needs more than a core's worth of CPU to do the
 compression realtime.  What happens then?  Is rsyslogd multithreaded
 enough (or can it be setup to be multithreaded enough) to spin up more
 threads to handle the compressed writes?  Will it ever drop messages?

 We'll I can't do the actual lab for you (well, under a support contract...).
 But what I can say is that I have the strong feeling this will work for you.
 I know at least of one datacenter which has a far higher data rate than you
 have and they work very successfully with that feature. But YMMV: you need to
 do some testing, which will identify potential bottlenecks, if there are any.

I am thrilled to do testing and can even do some in production, but
I'm not sure how to be sure no messages are dropped.  Any suggestions
for trackable high-load-generation?  I've been using logger and/or nc
in a loop from the command line to log # rather long message...
where # increases sequentially to be sure no messages were dropped,
but the production log stream is just a bunch of http requests and I
don't have any gauge of when one doesn't make it through, so
real-world testing might not be informative.


 - we're willing to change some configuration, but here's the only
 special config we have now:
    - $MainMsgQueueType Direct

 Outch - why that? This practically disables all multi-threading and thightly
 couples producer and consumer.

Hmm, it was there when I arrived and I never researched it.  I suspect
it was put in place for debugging purposes.  I'll remove it when I try
this out.


-- 
Chris McCraw | Operations
New Relic - http://blog.newrelic.com - @NewRelic on Twitter
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

[rsyslog] rsyslogd 5.8.5 + heavy message load + compression - failure mode?

[Chris McCraw]

Hi folks,

I probably missed it, but after awhile searching the docs fruitlessly,
I decided I'd ask the experts.

We have an rsyslog server that handles a couple million messages (from
a single remote server) per minute.  It logs these to several logfiles
with a cpu load of about 40% of a single CPU core.  logrotate
currently takes about 12 hours to single-threadedly, consecutively,
compress and rotate these logs.  It's been suggested that we take
logrotate out of the loop and just have rsyslog write compressed
files.  This seems like a great idea, but I'm curious about how it
scales, since I don't have a good test environment (just some
underpowered VM's which don't seem to generate comparable load no
matter how I try) to work with.

Suppose that rsyslog needs more than a core's worth of CPU to do the
compression realtime.  What happens then?  Is rsyslogd multithreaded
enough (or can it be setup to be multithreaded enough) to spin up more
threads to handle the compressed writes?  Will it ever drop messages?

some more info:
- The highest traffic logs are in 2 separate files, which have about
60% of the load together, the rest is going into a dozen other smaller
files.
- we'd be setting OMFileZipLevel to 1
- we're logging via tcp and splitting based on priority and sending IP
(though 99.9% of everything comes from one IP)
- we're willing to change some configuration, but here's the only
special config we have now:
   - $MainMsgQueueType Direct

Thanks for your insight!

-- 
Chris McCraw | Operations
New Relic - http://blog.newrelic.com - @NewRelic on Twitter
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

[rsyslog] DNS lookups in rsyslog v5

[Chris McCraw]

Hi list,

Longtime user, first time optimizer of rsyslog.  Here's my situation:

We just upgraded a machine that gets a ridiculous amount of log
traffic from one IP (our load balancer)--firehose levels, hundreds of
MB/minute.  This machine also takes logs of a few dozen low-traffic
servers on the same subnet.  With the upgrade from v4.6.2 to v5.8.5,
we gained UDP Multiruleset binding, yay!  We've moved all of our
logging via the firehose from TCP to UDP, because the TCP logging was
very fragile and would simply stop if the rsyslog restart for log
rotation took a microsecond too long.

Logging works great.  Our nameserver load shot way up, because it
seems our TCP-only 4.6.2 setup was not doing a DNS lookup for every
message...yet using the same file (with the addition of the UDP
ruleset binding) with v5.8.5 and -c5 instead of -c4 on the command
line for rsyslog has changed the lookup behavior of rsyslog, and named
is spinning constantly, presumably on the same host name.

Any pointers to the docs on how to mitigate this?  We're open to any
number of solutions (hopefully not including upgrading to v6)--put all
hostnames in /etc/hosts, for instance.  Since the firehose is all
bound to specific files anyway, those logs don't even need DNS
lookups--we know exactly where they come from.  We don't want to turn
off DNS entirely if we can avoid it, but we could partition into
normal port 514 tcp traffic gets lookups and other port UDP traffic
doesn't.  I'm guessing there is more than one way to do this =)

Thanks for your advice!
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

Re: [rsyslog] NFS log files not re-opened after rotation

[Chris Toomey]

I upgraded to rsyslog 4.6.4-2ubuntu4, which is what comes w/ Ubuntu 11.04,
and that fixes the problem.  I'm not sure how this relates to
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/407862 , maybe my
problem was fixed but other cases are still not handled.

Thanks David and Rainer for your help.

Chris
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

[rsyslog] [PATCH] avoid use of non-word size atomics

[Chris Metcalf]

The Tile platform doesn't natively support atomic operations other than
4-byte and 8-byte.  Although our runtime can handle subword atomics (by
doing a word-aligned read, inserting the sub-word properly, and trying
compare-and-exchange) it is more efficient to use word-size atomics where
possible.  I suspect this may also be true for other non-Intel platforms,
and certainly the top of rsyslog's runtime/atomic.h does say THESE MACROS
MUST ONLY BE USED WITH WORD-SIZED DATA TYPES!.

The attached patch against 6.1.0 converts msg_t's iRefCount from short to
int, and moves it in the structure so that the neighboring sbool and short
types can be packed more efficiently.

-- 
Chris Metcalf, Tilera Corp.
http://www.tilera.com



--- rsyslog-6.1.0/runtime/msg.h.orig2010-11-22 09:42:22.971057000 -0500
+++ rsyslog-6.1.0/runtime/msg.h 2010-11-22 09:43:19.10011 -0500
@@ -60,8 +60,8 @@
flowControl_t flowCtlType; /** type of flow control we can apply, for 
enqueueing, needs not to be persisted because
once data has entered the queue, this 
property is no longer needed. */
pthread_mutex_t mut;
+   int iRefCount;  /* reference counter (0 = unused) */
sbool   bDoLock; /* use the mutex? */
-   short   iRefCount;  /* reference counter (0 = unused) */
short   iSeverity;  /* the severity 0..7 */
short   iFacility;  /* Facility code 0 .. 23*/
short   offAfterPRI;/* offset, at which raw message WITHOUT PRI 
part starts in pszRawMsg */


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com