Ludovic,
I was about to answer your mail, but then saw John's response -- he has
nicely explained the idea how to attack the problem.
In addition, however, I'd recommend to simplify the regular expression
that parses the long list of comma separated values. Since you are only
looking for the 13th field, you don't need to set match variables for
fields 1,2,...12, since this consumes more resources. Also, the regular
expression might look more readable if it is written as follows:
pattern=ERROR,(?:[^,]*,){12}([^,]*)
Now the value of the 13th field is assigned to $1 match variable :)
Just try the following simple example:
type=Single
ptype=regexp
pattern=ERROR,(?:[^,]*,){12}([^,]*)
desc=13th field is $1
action=write - %s
hope this helps,
risto
On 06/17/2011 05:32 PM, Ludovic Hutin wrote:
> Hi,
>
> Here is the context, we would like to send alert when user do too
> many authentification
> We send mail at 3, 5 and 10 authentification failure in 30mn, in
> this email we send the history of failed authentification, and all the
> known IP for the user.
>
> We got two input file, the first with the authentification failure
> (FILE 1) and the second with the couple username / IP (FILE2)
>
> We have two ways of authentification, so we don't need to log IP
> which doesn't generate failure authentification after X minute.
>
> Because we are using some strange stuff, they are no order in the
> centralize log solution. We can have an entry in the FILE1 and after an
> entry in FILE2 or begin with an entry in FILE2 and after an entry come
> in FILE1 (I hope you understood me)
>
> I give you what i have done with (partial) sample.
>
> ## here i log ip for user when we found a failed authentification
> type=EventGroup
> continue=TakeNext
> ptype=regexp
> context=USERIP_$13
> pattern=ERROR,([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*)
> desc=User $13 appear
> action=write result/$13.ip %t USERIP_$13 ;
> window=1800
>
> ## eventgroup to send mail when we have 3 failure in 30mn
> type=EventGroup
> continue=TakeNext
> ptype=regexp
> pattern=ERROR,([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*)
> count=lcall %ret $13 -> ( sub { ++$ucounts3{$_[0]}; } ); \
> write /logs/rsa-authManager/result/$13.login %t $8 ; \
> add USER3_$13 $0
> desc=User $13 appear
> action=pipe 'envoiMail' /root/sendMail3.pl $13 ;
> multact=no
> end=lcall %ret $13 -> ( sub { return delete $ucounts3{$_[0]}; } ); \
> delete USER3_$13
> window=1800
> thresh=3
>
>
> ## eventgroup to send mail when we have 5 failure in 30mn
> type=EventGroup
> continue=TakeNext
> ptype=regexp
> pattern=ERROR,([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*)
> count=lcall %ret $13 -> ( sub { ++$ucounts5{$_[0]}; } ); \
> add USER5_$13 $0
> desc=User $13 appear
> action=pipe 'envoiMail' /root/sendMail5.pl $13 ;
> multact=no
> end=lcall %ret $13 -> ( sub { return delete $ucounts5{$_[0]}; } ); \
> delete USER5_$13
> window=1800
> thresh=5
>
>
> ## eventgroup to send mail when we have 10 failure in 30mn
> type=EventGroup
> continue=TakeNext
> ptype=regexp
> pattern=ERROR,([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*)
> count=lcall %ret $13 -> ( sub { ++$ucounts10{$_[0]}; } ); \
> add USER10_$13 $0
> desc=User $13 appear
> action=pipe 'envoiMail' /root/sendMail10.pl $13 ;
> multact=no
> end=lcall %ret $13 -> ( sub { return delete $ucounts10{$_[0]}; } ); \
> delete USER10_$13
> window=1800
> thresh=10
>
>
> ## Here we log IP address when we already have a failure authentification.
> type=SingleWithSuppress
> continue=takenext
> ptype=regexp
> context=USER3_$2
> pattern=.* logger: ([^\t]*)\t([^\t-]*)\t([^\t]*)\t([^\t]*)
> desc=User $2 logged in from $1
> action=write result/$2.ip %t $1 ;
> window=600
>
> ## Here we log IP address when we don't find failure authentification.
> type=SingleWithSuppress
> continue=takenext
> ptype=regexp
> context=!USER3_$2
> pattern=.* logger: ([^\t]*)\t([^\t-]*)\t([^\t]*)\t([^\t]*)
> desc=User $2 logged in from $1
> action=add USERIP_$2 $1;
> window=600
>
>
> Entry exemple :
>
>
> FILE2 exemple :
> Jun 17 16:03:58 XXXXXXXXXXXXXXX logger: IPOFTHEUSER UseRLoGiN
> [17/Jun/2011:16:03:58 +0200] PARAM1
>
> FILE1 exemple :
>
> Jun 17 16:06:35 XXXXXXXXXXXXXX 2011-06-17
> 16:06:35,957,,XXXX,ERROR,RRRR,AAAA,,ZZZZ,TTTT,13001,SUCCESS,ZZZZ,TTTTT,AAAA,BBBB,CCCC,userlogin,XXX,XXX,,,,,,,,,,,,,,,,,,,,
>
>
> There is no order in the event correlation. The two event are generated
> on differents computer in a very short period, they can be redirected to
> the centralized syslog server in any order.
>
> After that, we would like to sendMail when in a short time period (1
> hour ?), we saw a login with 3 differents ip address.
>
> I hope i am clear and i give enough information,
>
> Ludovic.
>
>
> Le 17/06/2011 15:48, Risto Vaarandi a écrit :
>> hi Ludovic,
>> it is impossible to set case insensitive processing for context names.
>> However, there are several workarounds. First, with a PerlFunc pattern
>> it is possible to convert the user name to lower (or upper) case
>> format, and return it for the match variable. Second, if you would
>> like to correlate two events, it might be possible to do this with
>> Pair or PairWithWindow rule. In this case, you can take advantage of
>> case insensitive regular expression matching in the 'pattern2' field,
>> with employing match variables inside the regexp.
>> In order to advise one or the other method, can you describe your
>> event correlation scenario more precisely? What are the input events
>> and what exactly you would like to achieve?
>> kind regards,
>> risto
>>
>> 2011/6/17 Ludovic Hutin<[email protected]>:
>>> Hi,
>>>
>>> I am back again, with another question, it's possible to set the
>>> context param to be case insensitive ?
>>> In my first log, username come in lowercase, and into other log
>>> they come with some uppercase
>>>
>>> context=USER_$13 with $13 in the first logs is "toto" and in the
>>> second log is "TOTO"
>>>
>>> The result is simple, impossible to correlate the 2 entry :-(
>>>
>>> Any idea ?
>>>
>>> Best regards,
>>> Ludovic.
>>>
>>> ------------------------------------------------------------------------------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>> http://p.sf.net/sfu/ephox-dev2dev
>>> _______________________________________________
>>> Simple-evcorr-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>>>
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Simple-evcorr-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>
>
------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
