Hi,
Thank you Risto. It works like a charm.
I was on holidays and i could not test it.
Regards.
2017-06-13 14:40 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com>:
> hi Jaren,
> if you are using 'copy' instead of 'shift', I would modify the ruleset
> a bit, and use additional context DOMAIN_$1 for storing the domain. In
> the previous example, SENDER_$1 was always holding two lines, and
> 'shift' actions were required for extracting these lines into two
> different variables. Since 'copy' always assings *both* lines (entire
> event store) to a variable, the lines need to be stored into two
> distinct contexts. Here is the updated ruleset (I haven't had time to
> test it, and hopefully all modifications are correct):
>
> type=single
> ptype=RegExp
> pattern=Mail\s(\d+):from=(.+@(.+))
> desc=store sender address
> action=create SENDER_$1 5; create DOMAIN_$1 5; add SENDER_$1 $2; add
> DOMAIN_$1 $3
>
> type=single
> ptype=RegExp
> pattern=Mail\s(\d+):subject=(.*)
> context=SENDER_$1
> desc=store subject
> action=create SUBJECT_$1 5; add SUBJECT_$1 $2
>
> type=single
> ptype=RegExp
> pattern=Mail\s(\d+):status=(.+)
> context=SENDER_$1
> desc=generate synthetic event for mail message
> action=copy SENDER_$1 %email; copy DOMAIN_$1 %domain; \
> copy SUBJECT_$1 %subject; delete SENDER_$1; delete DOMAIN_$1;
> delete SUBJECT_$1; \
> event MAIL_$1_STATUS_$2_EMAIL_%{email}_DOMAIN_%{domain}_
> SUBJECT_%{subject}
>
>
>
> Hope this helps,
> risto
>
> 2017-06-13 14:33 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
> > Hi,
> >
> > I´ve tested the rule and the problem that i see, it´s that "shift"
> command
> > is not included in Sec 2.6. 2. I have changed to "copy" and it is working
> > properly, i didnt see any incovenience.
> >
> > Thank you so much Risto!.Regards!.
> >
> > 2017-06-08 17:18 GMT+02:00 Jaren Peich <burkol...@gmail.com>:
> >>
> >> Hi,
> >>
> >> Thank you Risto! I was still blocked. I test it tomorrow and i´ll tell.
> >>
> >> Regards.
> >>
> >> 2017-06-07 13:26 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com>:
> >>>
> >>> hi Jaren,
> >>>
> >>> I would recommend to divide the task into two parts:
> >>> 1) normalization of log messages and the creation of one synthetic
> >>> event from three raw log events
> >>> 2) writing a thresholding rule for synthetic events generated during
> step
> >>> 1
> >>>
> >>> As I understand from examples, each incoming e-mail generates three
> >>> messages with the same numerical message ID. This message ID can be
> >>> utilized during normalization, and here is a simple example that
> >>> involves three Single rules:
> >>>
> >>> type=single
> >>> ptype=RegExp
> >>> pattern=Mail\s(\d+):from=(.+@(.+))
> >>> desc=store sender address
> >>> action=create SENDER_$1 5; add SENDER_$1 $2; add SENDER_$1 $3
> >>>
> >>> type=single
> >>> ptype=RegExp
> >>> pattern=Mail\s(\d+):subject=(.*)
> >>> context=SENDER_$1
> >>> desc=store subject
> >>> action=create SUBJECT_$1 5; add SUBJECT_$1 $2
> >>>
> >>> type=single
> >>> ptype=RegExp
> >>> pattern=Mail\s(\d+):status=(.+)
> >>> context=SENDER_$1
> >>> desc=generate synthetic event for mail message
> >>> action=shift SENDER_$1 %email; shift SENDER_$1 %domain; \
> >>> shift SUBJECT_$1 %subject; delete SENDER_$1; delete SUBJECT_$1;
> \
> >>> event
> >>> MAIL_$1_STATUS_$2_EMAIL_%{email}_DOMAIN_%{domain}_SUBJECT_%{subject}
> >>>
> >>> The first and second rule match the e-mail and subject messages,
> >>> respectively, and use contexts with a short lifetime to store relevant
> >>> information extracted from these events. It is assumed that the
> >>> subject message always comes after the e-mail message (as you can see,
> >>> the second rule checks if SENDER_$1 context exists that has been
> >>> created by first rule). The third rule reacts to the status message
> >>> and creates a synthetic event from information extracted from all
> >>> three events for the given message ID. For example, for the example
> >>> events from your previous post, the following synthetic events are
> >>> created:
> >>>
> >>> MAIL_1_STATUS_delivered_EMAIL_sec@sec.com_DOMAIN_sec.com_SUBJECT_Sec
> >>> Configuration Rule
> >>> MAIL_2_STATUS_delivered_EMAIL_pinker@sec.com_DOMAIN_sec.com_
> SUBJECT_Sec
> >>> Configuration Rule
> >>> MAIL_3_STATUS_delivered_EMAIL_plaster@sec.com_DOMAIN_sec.
> com_SUBJECT_Sec
> >>> Configuration Rule
> >>>
> >>> MAIL_4_STATUS_delivered_EMAIL_plaster@panik.com_DOMAIN_
> panik.com_SUBJECT_Sec
> >>> Configuration Rule
> >>>
> >>> MAIL_4_STATUS_Notdelivered_EMAIL_plaster@paniki.com_
> DOMAIN_paniki.com_SUBJECT_Sec
> >>> Configuration Rule
> >>>
> >>> Once the normalization is done, you can use simple SingleWithThreshold
> >>> rules for thresholding, without having a need to consider the
> >>> "multiline event issue" each time you have to write a rule. For
> >>> example, the following rule fires if 3 e-mails from the same domain
> >>> with the same subject line have been delivered within 60 seconds:
> >>>
> >>> type=SingleWithThreshold
> >>> ptype=RegExp
> >>>
> >>> pattern=MAIL_(\d+)_STATUS_delivered_EMAIL_(\S+?@\S+?)_
> DOMAIN_(\S+?)_SUBJECT_(.*)
> >>> desc=3 e-mails from domain $3 with the same subject $4 have been
> >>> delivered within 1 minute
> >>> action=write - %s
> >>> thresh=3
> >>> window=60
> >>>
> >>>
> >>> Hopefully these examples are helpful,
> >>> risto
> >>>
> >>>
> >>>
> >>> 2017-06-06 13:45 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
> >>> > Hi,
> >>> >
> >>> > I want to get all email from the same domain with the same subjects
> and
> >>> > check if they have been delivered(status=delivered) with a thresh.
> The
> >>> > problem is that the email log is split in different lines. The email
> >>> > comes
> >>> > with more lines but i´m only interested to get this(Log file lines).
> >>> >
> >>> > I have made 2 approches but i can´t validate domain variable in
> varmap
> >>> > with
> >>> > their aliaes to attach different context to a general context and
> >>> > validate
> >>> > all rule.
> >>> > desc field must be "subject" variable.
> >>> >
> >>> > I´m using SEC 2.6.2 with strawberry perl.
> >>> >
> >>> > Log file(Sometimes could come disordered):
> >>> >
> >>> > Mail 1:from=s...@sec.com
> >>> > Mail 1:subject=Sec Configuration Rule
> >>> > Mail 1:status=delivered
> >>> > Mail 2:from=pin...@sec.com
> >>> > Mail 2:subject=Sec Configuration Rule
> >>> > Mail 2:status=delivered
> >>> > Mail 3:from=plas...@sec.com
> >>> > Mail 3:subject=Sec Configuration Rule
> >>> > Mail 3:status=delivered
> >>> > Mail 4:from=plas...@panik.com
> >>> > Mail 4:subject=Sec Configuration Rule
> >>> > Mail 4:status=delivered
> >>> > Mail 4:from=plas...@paniki.com
> >>> > Mail 4:subject=Sec Configuration Rule
> >>> > Mail 4:status=Notdelivered
> >>> >
> >>> >
> >>> > ____________________________________________________________
> ____________________
> >>> >
> >>> >
> >>> > First idea:
> >>> >
> >>> > type = Single
> >>> > ptype = RegExp
> >>> > continue = Takenext
> >>> > context= Domain_$+{domain}
> >>> > desc = $0
> >>> > pattern = Mail\s(\d+)\:from\=(.*@(.*))
> >>> > varmap= mid=1; email=2;domain=3;
> >>> > action = alias Domain_$+{domain} HIT_Domain_$+{mid};fill
> >>> > Domain_$+{domain}
> >>> > $+{domain}
> >>> >
> >>> >
> >>> > type = Single
> >>> > ptype = RegExp
> >>> > continue = Takenext
> >>> > context= !Domain_$+{domain}
> >>> > desc = $0
> >>> > pattern = Mail\s(\d+)\:from\=(.*@(.*))
> >>> > varmap= mid=1; email=2;domain=3;
> >>> > action = create Domain_$+{domain} 86400; alias Domain_$+{domain}
> >>> > HIT_Domain_$+{mid};
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > type=EventGroup2
> >>> > ptype=regexp
> >>> > pattern=Mail\s(\d+)\:subject\=(.*)
> >>> > varmap= mid=1; subject=2
> >>> > context = HIT_Domain_$+{mid}
> >>> > thresh=2
> >>> > count=alias Domain_$+{domain} Domain_Subject_$+{mid};
> >>> > ptype2=regexp
> >>> > pattern2=Mail\s(\d+)\:status\=delivered
> >>> > varmap2= mid=1;
> >>> > context2 = Domain_Subject_$+{mid} && HIT_Domain_$+{mid}
> >>> > thresh=2
> >>> > desc=Domain_$+{subject}
> >>> > action= write - "Test String"
> >>> > window=86400
> >>> >
> >>> >
> >>> > ____________________________________________________________
> ___________________
> >>> >
> >>> > Second idea:
> >>> >
> >>> > type=EventGroup4
> >>> > ptype = RegExp
> >>> > continue = Takenext
> >>> > context= Domain_$+{domain}
> >>> > desc = $0
> >>> > pattern = Mail\s(\d+)\:from\=(.*@(.*))
> >>> > varmap= mid=1; email=2;domain=3;
> >>> > count = alias Domain_$+{domain} HIT_Domain_$+{mid};
> >>> > ptype2 = RegExp
> >>> > continue2 = Takenext
> >>> > context2= !Domain_$+{domain}
> >>> > pattern2 = Mail\s(\d+)\:from\=(.*@(.*))
> >>> > varmap2= mid=1; email=2;domain=3;
> >>> > count2 = create Domain_$+{domain} 86400; alias Domain_$+{domain}
> >>> > HIT_Domain_$+{mid};
> >>> > ptype3=regexp
> >>> > pattern3=Mail\s(\d+)\:subject\=(.*)
> >>> > varmap3= mid=1; subject=2
> >>> > context3 = HIT_Domain_$+{mid}
> >>> > thresh3=2
> >>> > count3=alias DOMAIN_$+{domain} Domain_Subject_$+{mid};
> >>> > ptype4=regexp
> >>> > pattern4=Mail\s(\d+)\:status\=delivered
> >>> > varmap4= mid=1;
> >>> > context4 = Domain_Subject_$+{mid} && HIT_Domain_$+{mid}
> >>> > thresh4=2
> >>> > desc=Domain_$+{subject}
> >>> > action= write - "Test String"
> >>> > window=86400
> >>> >
> >>> >
> >>> > Any ideas?
> >>> >
> >>> > Thank you. Regards.
> >>
> >>
> >
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
