Re: [rsyslog] Please help with Snare Format

[David Lang]

On Thu, 29 Nov 2012, jdguingao wrote:


Will it still force escape even if I use this directive
$EscapeControlCharactersOnReceive off ?


I'm not sure, but if it doesn't, then it won't do anything (since the tests look 
for the escaped character sequences). It wouldn't be a lot of work to modify it 
to handle the non-escaped characters, but you would then need to compile your 
own copy (until the next release)


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[jdguingao]

David thank you for your help I already solve the problem.


This message is part of the syslog tag: MSWinEventLog0  
Security957 Fri 

So i just use this command to extract the security field. syslogtag:F:3.
Again thank your for all your help


Cheers
Jong



--
View this message in context: 
http://rsyslog-rsyslog-users.1305293.n2.nabble.com/Please-help-with-Snare-Format-tp7579234p7579247.html
Sent from the rsyslog -- rsyslog-users mailing list archive at Nabble.com.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[jdguingao]

Will it still force escape even if I use this directive
$EscapeControlCharactersOnReceive off ?



--
View this message in context: 
http://rsyslog-rsyslog-users.1305293.n2.nabble.com/Please-help-with-Snare-Format-tp7579234p7579245.html
Sent from the rsyslog -- rsyslog-users mailing list archive at Nabble.com.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[David Lang]

I've run into problems with the version not exactly matching everything else. In 
theory it will work, but I don't know where the landmines are.


the pmsnare module only works on the first couple of fields of the message 
(timestamp, hostname, and possibly the MSWinEventLog string), everything else it 
leaves alone, but it will force escaping, so all tabs will be replaced by #011, 
the escaping is fine if you are using external stuff (like perl) to parse the 
message, but rsyslog doesn't have multi-character split capability, so it will 
make it hard to extract the fields with rsyslog format tricks like the example 
you listed does.


David Lang

On Thu, 29 Nov 2012, jdguingao wrote:


Thanks for the help David and Dan. What I am thinking now is to use the
pmsnare module to test if I can extract that field but my installation of
rsyslog does not have it. I use the RPM that the rsyslog team provided in
their website. Is their anyway to upload a module to my existing rsyslog
installation or do I have to compile it from source?



--
View this message in context: 
http://rsyslog-rsyslog-users.1305293.n2.nabble.com/Please-help-with-Snare-Format-tp7579234p7579243.html
Sent from the rsyslog -- rsyslog-users mailing list archive at Nabble.com.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[jdguingao]

Thanks for the help David and Dan. What I am thinking now is to use the
pmsnare module to test if I can extract that field but my installation of
rsyslog does not have it. I use the RPM that the rsyslog team provided in
their website. Is their anyway to upload a module to my existing rsyslog
installation or do I have to compile it from source?



--
View this message in context: 
http://rsyslog-rsyslog-users.1305293.n2.nabble.com/Please-help-with-Snare-Format-tp7579234p7579243.html
Sent from the rsyslog -- rsyslog-users mailing list archive at Nabble.com.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[Woodruff, Dan]

I've been using this filter to parse security event log messages into their own 
file with great success, if this helps at all:

if $syslogtag contains 'Security' \
then ?WindowsSecFile;WindowsFileMsgFormat
& ~


-Original Message-
From: rsyslog-boun...@lists.adiscon.com 
[mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of David Lang
Sent: Thursday, November 29, 2012 4:11 PM
To: rsyslog-users
Subject: Re: [rsyslog] Please help with Snare Format

On Thu, 29 Nov 2012, jdguingao wrote:

> I will enclose in curly braces the message that I want to extract
>
> 2012-11-30T02:41:46+08:00 CX-CDOWKSMIS003.ph.gbsorg.net MSWinEventLog   0
> {Security}491 Fri Nov 30 02:41:44 20124689
> Microsoft-Windows-Security-Auditing PH\CX-CDOWKSMIS003$ N/A
> Success Audit   CX-CDOWKSMIS003.ph.gbsorg.net   Process Termination
> A process has exited.Subject:   Security ID:  S-1-5-18   Account Name:
> CX-CDOWKSMIS003$   Account Domain:  PH   Logon ID:  0x3e7Process
> Information:   Process ID: 0x1d50   Process Name:
> C:\Windows\System32\SearchFilterHost.exe   Exit Status: 0x0  265
>
> I think I understand the log format a little better. I thought that 
> when you put %msg:F:3% It will extract the Security message that I 
> want but it will have this message 
> Microsoft-Windows-Security-Auditing. So the Tab splitting starts with 
> this field  Fri Nov 30 02:41:44 2012 (when I use %msg:F:1%).

a good way to see this is by logging with the RSYSLOG_DebugFormat. It will tell 
you how the message got broken apart by the rsyslog parser.

if it's starting with the date, that means that, as configured, your version of 
snare is not sending a valid syslog message, it's using tabs between the date, 
hostname, and MSWinEventLog fields. My Snare installs sometimes do this, 
sometimes don't, and sometimes have a null character in there somewhere. This 
is the sort of stuff that pmsnare tries to fix up (so that MSWinEventLog ends 
up in the programname field and the message starts with the field after that, 
in this case '0')

Another thing to watch out for is if escaping of control characters gets 
triggered, all the tab characters will become '#011', which really messes up 
parsing as you can't split on multi-character fields with rsyslog.

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is 
a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our 
control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[David Lang]

On Thu, 29 Nov 2012, jdguingao wrote:


I will enclose in curly braces the message that I want to extract

2012-11-30T02:41:46+08:00 CX-CDOWKSMIS003.ph.gbsorg.net MSWinEventLog   0
{Security}491 Fri Nov 30 02:41:44 20124689
Microsoft-Windows-Security-Auditing PH\CX-CDOWKSMIS003$ N/A
Success Audit   CX-CDOWKSMIS003.ph.gbsorg.net   Process Termination
A process has exited.Subject:   Security ID:  S-1-5-18   Account Name:
CX-CDOWKSMIS003$   Account Domain:  PH   Logon ID:  0x3e7Process
Information:   Process ID: 0x1d50   Process Name:
C:\Windows\System32\SearchFilterHost.exe   Exit Status: 0x0  265

I think I understand the log format a little better. I thought that when you
put %msg:F:3% It will extract the
Security message that I want but it will have this message
Microsoft-Windows-Security-Auditing. So the Tab splitting starts with this
field  Fri Nov 30 02:41:44 2012 (when I use %msg:F:1%).


a good way to see this is by logging with the RSYSLOG_DebugFormat. It will tell 
you how the message got broken apart by the rsyslog parser.


if it's starting with the date, that means that, as configured, your version of 
snare is not sending a valid syslog message, it's using tabs between the date, 
hostname, and MSWinEventLog fields. My Snare installs sometimes do this, 
sometimes don't, and sometimes have a null character in there somewhere. This is 
the sort of stuff that pmsnare tries to fix up (so that MSWinEventLog ends up in 
the programname field and the message starts with the field after that, in this 
case '0')


Another thing to watch out for is if escaping of control characters gets 
triggered, all the tab characters will become '#011', which really messes up 
parsing as you can't split on multi-character fields with rsyslog.


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[jdguingao]

I will enclose in curly braces the message that I want to extract

2012-11-30T02:41:46+08:00 CX-CDOWKSMIS003.ph.gbsorg.net MSWinEventLog   0 
{Security}491 Fri Nov 30 02:41:44 20124689   
Microsoft-Windows-Security-Auditing PH\CX-CDOWKSMIS003$ N/A
Success Audit   CX-CDOWKSMIS003.ph.gbsorg.net   Process Termination
A process has exited.Subject:   Security ID:  S-1-5-18   Account Name: 
CX-CDOWKSMIS003$   Account Domain:  PH   Logon ID:  0x3e7Process
Information:   Process ID: 0x1d50   Process Name:
C:\Windows\System32\SearchFilterHost.exe   Exit Status: 0x0  265 

I think I understand the log format a little better. I thought that when you
put %msg:F:3% It will extract the 
Security message that I want but it will have this message
Microsoft-Windows-Security-Auditing. So the Tab splitting starts with this
field  Fri Nov 30 02:41:44 2012 (when I use %msg:F:1%). 



--
View this message in context: 
http://rsyslog-rsyslog-users.1305293.n2.nabble.com/Please-help-with-Snare-Format-tp7579234p7579238.html
Sent from the rsyslog -- rsyslog-users mailing list archive at Nabble.com.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[David Lang]

On Thu, 29 Nov 2012, jdguingao wrote:


I want to mimic the standard Event log data that I can see in PhpLogcon. I
have borrowed a template from a user in rsyslog forum. Here is the link
kb.monitorware.com/post20457.html#p20457
and I want to extract
this field

2012-11-30T02:41:46+08:00 CX-CDOWKSMIS003.ph.gbsorg.net MSWinEventLog   0
*Security *   491 Fri Nov 30 02:41:44 20124689
Microsoft-Windows-Security-Auditing PH\CX-CDOWKSMIS003$ N/A
Success Audit   CX-CDOWKSMIS003.ph.gbsorg.net   Process Termination
A process has exited.Subject:   Security ID:  S-1-5-18   Account Name:
CX-CDOWKSMIS003$   Account Domain:  PH   Logon ID:  0x3e7Process
Information:   Process ID: 0x1d50   Process Name:
C:\Windows\System32\SearchFilterHost.exe   Exit Status: 0x0  265

(See bold letters) to be my message in Eventlog Type.


the bold letters are not getting through to me (either in my text mail reader or 
my webmail reader)


Ok, looking at the post you are referring to, it is splitting the fields on tabs 
%msg:F:3% in a template says to put the third field from the message into this 
spot.


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[jdguingao]

I want to mimic the standard Event log data that I can see in PhpLogcon. I
have borrowed a template from a user in rsyslog forum. Here is the link 
kb.monitorware.com/post20457.html#p20457
and I want to extract
this field 

2012-11-30T02:41:46+08:00 CX-CDOWKSMIS003.ph.gbsorg.net MSWinEventLog   0  
*Security *   491 Fri Nov 30 02:41:44 20124689   
Microsoft-Windows-Security-Auditing PH\CX-CDOWKSMIS003$ N/A
Success Audit   CX-CDOWKSMIS003.ph.gbsorg.net   Process Termination
A process has exited.Subject:   Security ID:  S-1-5-18   Account Name: 
CX-CDOWKSMIS003$   Account Domain:  PH   Logon ID:  0x3e7Process
Information:   Process ID: 0x1d50   Process Name:
C:\Windows\System32\SearchFilterHost.exe   Exit Status: 0x0  265 

(See bold letters) to be my message in Eventlog Type.



--
View this message in context: 
http://rsyslog-rsyslog-users.1305293.n2.nabble.com/Please-help-with-Snare-Format-tp7579234p7579236.html
Sent from the rsyslog -- rsyslog-users mailing list archive at Nabble.com.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Please help with Snare Format

[David Lang]

which fields are you wanting extracted? lots of them could be considered 
'security fields'


David LAng

On Thu, 29 Nov 2012, jdguingao wrote:


Date: Thu, 29 Nov 2012 10:52:53 -0800 (PST)
From: jdguingao 
Reply-To: rsyslog-users 
To: rsyslog@lists.adiscon.com
Subject: [rsyslog] Please help with Snare Format

HI All, Please help me how to extract the security fields in this message
using regex or any other methods

Here is a Sample log from Snare


2012-11-30T02:41:46+08:00 CX-CDOWKSMIS003.ph.gbsorg.net MSWinEventLog   0
Security491 Fri Nov 30 02:41:44 20124689
Microsoft-Windows-Security-Auditing PH\CX-CDOWKSMIS003$ N/A
Success Audit   CX-CDOWKSMIS003.ph.gbsorg.net   Process Termination
A process has exited.Subject:   Security ID:  S-1-5-18   Account Name:
CX-CDOWKSMIS003$   Account Domain:  PH   Logon ID:  0x3e7Process
Information:   Process ID: 0x1d50   Process Name:
C:\Windows\System32\SearchFilterHost.exe   Exit Status: 0x0  265

I have tried some process but to no avail. I have use the Snare parser in
php logcon but it is not working. I dont have the pmsnare module as i did
not compile my rsyslog installation from source. Im still new to rsyslog and
regex. Thanks




--
View this message in context: 
http://rsyslog-rsyslog-users.1305293.n2.nabble.com/Please-help-with-Snare-Format-tp7579234.html
Sent from the rsyslog -- rsyslog-users mailing list archive at Nabble.com.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.