hi Jaren,
I would recommend to divide the task into two parts:
1) normalization of log messages and the creation of one synthetic
event from three raw log events
2) writing a thresholding rule for synthetic events generated during step 1
As I understand from examples, each incoming e-mail generates three
messages with the same numerical message ID. This message ID can be
utilized during normalization, and here is a simple example that
involves three Single rules:
type=single
ptype=RegExp
pattern=Mail\s(\d+):from=(.+@(.+))
desc=store sender address
action=create SENDER_$1 5; add SENDER_$1 $2; add SENDER_$1 $3
type=single
ptype=RegExp
pattern=Mail\s(\d+):subject=(.*)
context=SENDER_$1
desc=store subject
action=create SUBJECT_$1 5; add SUBJECT_$1 $2
type=single
ptype=RegExp
pattern=Mail\s(\d+):status=(.+)
context=SENDER_$1
desc=generate synthetic event for mail message
action=shift SENDER_$1 %email; shift SENDER_$1 %domain; \
shift SUBJECT_$1 %subject; delete SENDER_$1; delete SUBJECT_$1; \
event
MAIL_$1_STATUS_$2_EMAIL_%{email}_DOMAIN_%{domain}_SUBJECT_%{subject}
The first and second rule match the e-mail and subject messages,
respectively, and use contexts with a short lifetime to store relevant
information extracted from these events. It is assumed that the
subject message always comes after the e-mail message (as you can see,
the second rule checks if SENDER_$1 context exists that has been
created by first rule). The third rule reacts to the status message
and creates a synthetic event from information extracted from all
three events for the given message ID. For example, for the example
events from your previous post, the following synthetic events are
created:
MAIL_1_STATUS_delivered_EMAIL_sec@sec.com_DOMAIN_sec.com_SUBJECT_Sec
Configuration Rule
MAIL_2_STATUS_delivered_EMAIL_pinker@sec.com_DOMAIN_sec.com_SUBJECT_Sec
Configuration Rule
MAIL_3_STATUS_delivered_EMAIL_plaster@sec.com_DOMAIN_sec.com_SUBJECT_Sec
Configuration Rule
MAIL_4_STATUS_delivered_EMAIL_plaster@panik.com_DOMAIN_panik.com_SUBJECT_Sec
Configuration Rule
MAIL_4_STATUS_Notdelivered_EMAIL_plaster@paniki.com_DOMAIN_paniki.com_SUBJECT_Sec
Configuration Rule
Once the normalization is done, you can use simple SingleWithThreshold
rules for thresholding, without having a need to consider the
"multiline event issue" each time you have to write a rule. For
example, the following rule fires if 3 e-mails from the same domain
with the same subject line have been delivered within 60 seconds:
type=SingleWithThreshold
ptype=RegExp
pattern=MAIL_(\d+)_STATUS_delivered_EMAIL_(\S+?@\S+?)_DOMAIN_(\S+?)_SUBJECT_(.*)
desc=3 e-mails from domain $3 with the same subject $4 have been
delivered within 1 minute
action=write - %s
thresh=3
window=60
Hopefully these examples are helpful,
risto
2017-06-06 13:45 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
> Hi,
>
> I want to get all email from the same domain with the same subjects and
> check if they have been delivered(status=delivered) with a thresh. The
> problem is that the email log is split in different lines. The email comes
> with more lines but i´m only interested to get this(Log file lines).
>
> I have made 2 approches but i can´t validate domain variable in varmap with
> their aliaes to attach different context to a general context and validate
> all rule.
> desc field must be "subject" variable.
>
> I´m using SEC 2.6.2 with strawberry perl.
>
> Log file(Sometimes could come disordered):
>
> Mail 1:from=s...@sec.com
> Mail 1:subject=Sec Configuration Rule
> Mail 1:status=delivered
> Mail 2:from=pin...@sec.com
> Mail 2:subject=Sec Configuration Rule
> Mail 2:status=delivered
> Mail 3:from=plas...@sec.com
> Mail 3:subject=Sec Configuration Rule
> Mail 3:status=delivered
> Mail 4:from=plas...@panik.com
> Mail 4:subject=Sec Configuration Rule
> Mail 4:status=delivered
> Mail 4:from=plas...@paniki.com
> Mail 4:subject=Sec Configuration Rule
> Mail 4:status=Notdelivered
>
> ________________________________________________________________________________
>
>
> First idea:
>
> type = Single
> ptype = RegExp
> continue = Takenext
> context= Domain_$+{domain}
> desc = $0
> pattern = Mail\s(\d+)\:from\=(.*@(.*))
> varmap= mid=1; email=2;domain=3;
> action = alias Domain_$+{domain} HIT_Domain_$+{mid};fill Domain_$+{domain}
> $+{domain}
>
>
> type = Single
> ptype = RegExp
> continue = Takenext
> context= !Domain_$+{domain}
> desc = $0
> pattern = Mail\s(\d+)\:from\=(.*@(.*))
> varmap= mid=1; email=2;domain=3;
> action = create Domain_$+{domain} 86400; alias Domain_$+{domain}
> HIT_Domain_$+{mid};
>
>
>
>
> type=EventGroup2
> ptype=regexp
> pattern=Mail\s(\d+)\:subject\=(.*)
> varmap= mid=1; subject=2
> context = HIT_Domain_$+{mid}
> thresh=2
> count=alias Domain_$+{domain} Domain_Subject_$+{mid};
> ptype2=regexp
> pattern2=Mail\s(\d+)\:status\=delivered
> varmap2= mid=1;
> context2 = Domain_Subject_$+{mid} && HIT_Domain_$+{mid}
> thresh=2
> desc=Domain_$+{subject}
> action= write - "Test String"
> window=86400
>
> _______________________________________________________________________________
>
> Second idea:
>
> type=EventGroup4
> ptype = RegExp
> continue = Takenext
> context= Domain_$+{domain}
> desc = $0
> pattern = Mail\s(\d+)\:from\=(.*@(.*))
> varmap= mid=1; email=2;domain=3;
> count = alias Domain_$+{domain} HIT_Domain_$+{mid};
> ptype2 = RegExp
> continue2 = Takenext
> context2= !Domain_$+{domain}
> pattern2 = Mail\s(\d+)\:from\=(.*@(.*))
> varmap2= mid=1; email=2;domain=3;
> count2 = create Domain_$+{domain} 86400; alias Domain_$+{domain}
> HIT_Domain_$+{mid};
> ptype3=regexp
> pattern3=Mail\s(\d+)\:subject\=(.*)
> varmap3= mid=1; subject=2
> context3 = HIT_Domain_$+{mid}
> thresh3=2
> count3=alias DOMAIN_$+{domain} Domain_Subject_$+{mid};
> ptype4=regexp
> pattern4=Mail\s(\d+)\:status\=delivered
> varmap4= mid=1;
> context4 = Domain_Subject_$+{mid} && HIT_Domain_$+{mid}
> thresh4=2
> desc=Domain_$+{subject}
> action= write - "Test String"
> window=86400
>
>
> Any ideas?
>
> Thank you. Regards.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
