...just one additional suggestion (I know, it is a very different
approach from yours) -- in the case *all* relevant messages appear in
sec input at some point of time, you could also address the problem
with following sec rules:
# This rule associates the Subject-line and the sender-line with the
same MID value,
# and generates a synthetic event that represents both events, for example:
#
# MID:52366 Domain:sec.com Subject:Hello World
#
# The rule assumes that Subject-line and sender-line for the same MID value can
# appear in any order and are separated by at most 60 seconds
type=EventGroup2
ptype=RegExp
pattern=email:Mid (\d+) sender:'\S+?@(\S+)'
count=fill MID_$1_DOMAIN $2
ptype2=RegExp
pattern2=email:Mid (\d+) Subject:'(.+)'
count2=fill MID_$1_SUBJECT $2
desc=Associate the subject with the sender domain based on mid $1
action=copy MID_$1_SUBJECT %subject; copy MID_$1_DOMAIN %domain; \
event MID:$1 Domain:%domain Subject:%subject
init=create MID_$1_DOMAIN; create MID_$1_SUBJECT
end=delete MID_$1_DOMAIN; delete MID_$1_SUBJECT
window=60
# This rule counts synthetic events generated by counting operations
of the previous rule,
# and generates an alert if three messages are seen with the same
subject lines and
# the same sender domain within 1 hour (if the MID values have to be
unique for counted
# synthetic events, this rule can be easily elaborated into EventGroup
rule with this functionality)
type=SingleWithThreshold
ptype=RegExp
pattern=MID:\d+ Domain:(\S+) Subject:(.+)
desc=Three messages from the same domain $1 with the same subject $2
action=write - %s
window=3600
thresh=3
I am not sure if the above suggestion fits into your environment, and
if not, maybe it is useful for tackling some future event correlation
task :)
kind regards,
risto
2016-10-04 15:11 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
> Sorry i missed to add. The idea is to find the first 3 mids code with the
> same subject and then compare domain email senders of 3 codes searched.
> If every domain from the sender is equal, print a message on the screen. In
> the case that 1 of them is different just not to show the message.
>
> I find mid numbers with the rules and keep in a context and in the perl
> code. Then extract mid numbers and find domain sender sender. Keep domain
> sender in an array to compare after the search.
>
> I erased this part just to test:
> if($arraymids[0] eq $arraymids[1] && $arraymids[0] eq $arraymids[2]){\
> print "\n=======>SAME DOMAIN\n";\
> }else{\
> print "\n=======>Different DOMAIN\n";\
> }\
>
> Regards Risto.
>
> 2016-10-04 13:55 GMT+02:00 Jaren Peich <burkol...@gmail.com>:
>>
>> Hi,
>>
>> In this case the rules are ordered but there are some cases, i´ve seen
>> that there are disordered data and in other files. I´ve seen today. I have
>> to do perl search.
>>
>> It can be as input also like this, its quite a pity:
>>
>> email:Mid 52365 sender:'ja...@sec.com'
>> email:Mid 52366 sender:'s...@sec.com'
>> email:Mid 52364 Subject:'Hello World'
>> email:Mid 52366 Subject:'Hello World'
>> email:Mid 52364 sender:'pe...@sec.com'
>> email:Mid 52365 Subject:'Hello World'
>>
>>
>> I´m trying to access the context data, i just only copy the action:
>>
>> action = copy SM_$+{subject} %hashmids; lcall %o %hashmids -> ( sub{\
>> $i=0;\
>> my(@arraymids)=();\
>> %pmid=%{ $_[0] };\
>> @midcont = values %pmid;\
>> foreach my $lmid (@midcont) {\
>> print "\nlmid===>$lmid\n" if defined($mid);\
>> }\
>> @files = <C:\\files\\m*.log>;\
>> print "@files";\
>> if(@files){\
>> foreach $file (@files) {\
>> open (FILE, "$file");\
>> while($line= <FILE> ){\
>> my ($mid, $email)= $line=~ m/email:Mid\s(\d+)\ssender\:\'([^\']*)\'/gi;\
>> print "\n====>Vuelta\n";\
>> print "$midcont[0]";\
>> foreach my $lmid (@midcont) {\
>> print "\nMid fichero===>$mid\tlmid===>$lmid\n" if defined($mid);\
>> if ($mid == $lmid){\
>> my ($username, $domain) = $email =~ /(.*)@(.*)/;\
>> print "\nDOMAIN FINDED===>$domain\n";\
>> unshift(@arraymids, $domain);\
>> }\
>> }\
>> }\
>> }\
>> }else{\
>> print "========>NO FILES TO SEARCH";\
>> }\
>> }\
>> );reset 0;
>>
>> Output is not as expected:
>>
>> Stdin connected to terminal, SIGINT can't be used for changing the logging
>> level
>> Creating SEC internal context 'SEC_INTERNAL_EVENT'
>> Creating SEC internal event 'SEC_STARTUP'
>> Deleting SEC internal context 'SEC_INTERNAL_EVENT'
>> Creating context 'SM_Hello World'
>> Adding event(s) '52364' to context 'SM_Hello World'
>> Adding event(s) '52365' to context 'SM_Hello World'
>> Adding event(s) '52366' to context 'SM_Hello World'
>> Copying context 'SM_Hello World' to variable '%hashmids'
>> Variable '%hashmids' set to '52364
>> 52365
>> 52366'
>> Calling code 'CODE(0x2927f34)' and setting variable '%o'
>> C:\log.log
>> ====>Vuelta
>> Use of uninitialized value $main::SEC::midcont[0] in string at (eval 4)
>> line 1, <FILE> line 1.
>>
>> ====>Vuelta
>> Use of uninitialized value $main::SEC::midcont[0] in string at (eval 4)
>> line 1, <FILE> line 2.
>>
>> ====>Vuelta
>> Use of uninitialized value $main::SEC::midcont[0] in string at (eval 4)
>> line 1, <FILE> line 3.
>> Variable '%o' set to ''
>> Terminating event correlation operation 'C:\prueba.conf | 1 | create
>> context by subject_Hello World'
>> Creating SEC internal context 'SEC_INTERNAL_EVENT'
>> Creating SEC internal event 'SEC_SHUTDOWN'
>> Deleting SEC internal context 'SEC_INTERNAL_EVENT'
>>
>> I don´t know why i can´t access to the %hashmids that I copied before and
>> added as a parameter. How can i reference it?
>>
>>
>> Thank you for your help Risto. Regards.
>>
>>
>> 2016-10-03 15:33 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com>:
>>>
>>> hi Jaren,
>>> let me add few suggestions below:
>>>
>>> >
>>> > rem=create context and store first mid-count 2 and 3
>>> > type=EventGroup1
>>> > ptype=Cached
>>> > pattern=SubjectMID
>>> > context=SM_$+{subject}
>>> > thresh=2
>>> > window=60
>>> > count=add SM_$+{subject} $+{mid}
>>> > desc=create context by subject_$+{subject}
>>> > end=delete SM_$+{subject}
>>> > action = copy SM_$+{subject} %hashmids; lcall %o %hashmids -> ( \
>>> > print "\nArrive!!!!!";\
>>>
>>> The Perl code that has been provided with 'lcall' action is not a
>>> valid function definition, and you need to enclose the code in sub { }
>>> construct. For example:
>>>
>>> lcall %o %hashmids -> ( sub { print "\nArrive!!!!!"; } )
>>>
>>> Also, the value of the %hashmids variable is not used in the function
>>> (in the perl function, $_[0] variable refers to the first input
>>> parameters.
>>>
>>> > Output:
>>> >
>>> > SEC (Simple Event Correlator) 2.6.2
>>> > Reading configuration from C:\prueba.conf
>>> >
>>> > Rule in C:\prueba.conf at line 10: Eval 'print "\nArrive!!!!!";' didn't
>>> > return a code reference: 1
>>> > Rule in C:\prueba.conf at line 10: Invalid action list ' copy
>>> > SM_$+{subject}
>>> > %hashmids; lcall %o %hashmids -> ( print "\nArrive!!!!!";);reset 0; '
>>>
>>> you are seeing this error message since 'lcall' does not have a
>>> correct function definition (see my previous remark).
>>>
>>> >
>>> > Is it possible to pass to the perl function a changing context name
>>> > depending on the $+{subject} variable and process this data as i told
>>> > you
>>> > before?
>>>
>>> you can do that, but then you would have to access SEC's internal
>>> context data structure, in order to process the context event store
>>> (the context name serves as a key into %main::context_list hash table,
>>> and in this hash table). In my opinion, that is not a very clean way,
>>> and passing a newline separated string of numerals into the function
>>> is both simpler and more readable. Also, you wouldn't have any
>>> dependencies on sec internals.
>>>
>>> Just out of curiosity -- is my understanding correct that the lines
>>> you want to process are all appearing in the same input file:
>>>
>>> email:Mid 52364 Subject:'Hello World'
>>> email:Mid 52365 Subject:'Hello World'
>>> email:Mid 52366 Subject:'Hello World'
>>> email:Mid 52366 sender:'s...@sec.com'
>>> email:Mid 52365 sender:'ja...@sec.com'
>>> email:Mid 52364 sender:'pe...@sec.com'
>>>
>>> If that is the case, it might be cheaper not to repeatedly search the
>>> input file with Perl code which is expensive, but rather implement the
>>> same logic with sec rules. What is your actual goal for extracting the
>>> e-mail addresses based on subject lines? There might be a much shorter
>>> and more elegant way for addressing this problem.
>>>
>>> kind regards,
>>> risto
>>>
>>>
>>> > One create a 'SM_Hello World', other email context can be 'SM_Hello
>>> > from
>>> > SEC', other one 'SM_Hallo Frank', 'SM_Julius notes', etc...
>>> >
>>> > Finally in the first email i just added the perl function that i
>>> > designed to
>>> > search into the files this data.
>>> >
>>> > This one(Not working, because i add notes text between ##########):
>>> >
>>> > eval %o ( \
>>> > $i=0;\
>>> > my(@arraymids)=();\
>>> > ####################################\
>>> > #Here i want to read again the context and extract the values. Add to
>>> > the
>>> > array for processing.\
>>> > @midcont=("52366","52365","52364");\
>>> > ###################################
>>> > @files = <C:\\files\\m*.log>;\
>>> > print "@files";\
>>> > if(@files){\
>>> > foreach $file (@files) {\
>>> > open (FILE, "$file");\
>>> > while($line= <FILE> ){\
>>> > my ($mid, $email)= $line=~
>>> > m/email:Mid\s(\d+)\ssender\:\'([^\']*)\'/gi;\
>>> > print "\n====>Vuelta\n";\
>>> > foreach my $lmid (@midcont) {\
>>> > print "\nMid fichero===>$mid\tlmid===>$lmid\n" if defined($mid);\
>>> > if ($mid == $lmid){\
>>> > my ($username, $domain) = $email =~ /(.*)@(.*)/;\
>>> > print "\nDOMAIN FINDED===>$domain\n";\
>>> > unshift(@arraymids, $domain);\
>>> > }\
>>> > }\
>>> > }\
>>> > }\
>>> > if($arraymids[0] eq $arraymids[1] && $arraymids[0] eq $arraymids[2]){\
>>> > print "\n=======>SAME DOMAIN\n";\
>>> > }else{\
>>> > print "\n=======>Different DOMAIN\n";\
>>> > }\
>>> > }else{\
>>> > print "========>NO FILES TO SEARCH";\
>>> > }\
>>> > );reset 0;
>>> >
>>> >
>>> >
>>> > Thank you Risto again. Regards.
>>> >
>>> > 2016-10-01 12:50 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com>:
>>> >>
>>> >> 2016-09-30 12:20 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
>>> >> > Hi Risto,
>>> >> >
>>> >> > I have a little doubt that i haven´t seen in the documentation.
>>> >> > I want to detect and keep some data from the logs and then read the
>>> >> > context
>>> >> > again and extract this data to process this info throught a perl
>>> >> > function.
>>> >> >
>>> >>
>>> >> ...
>>> >>
>>> >> > action = eval %o ( \
>>> >> > $i=0;\
>>> >> > my(@arraymids)=();\
>>> >> > ####################################\
>>> >> > #Here i want to read again the context and extract the values. Add
>>> >> > to
>>> >> > the
>>> >> > array for processing.\
>>> >> > @midcont=("52366","52365","52364");\
>>> >> > ###################################
>>> >>
>>> >> I am still not sure whether I have understood the problem correctly,
>>> >> but if you would like to access the event store of the SEC context
>>> >> from a Perl function/expression, I would recommend to use the 'copy'
>>> >> action before the Perl code is invoked. For example, if the event
>>> >> store of context TEST contains events "test1", "test2" and "test3",
>>> >> then the following action
>>> >>
>>> >> copy TEST %events
>>> >>
>>> >> will set the action list variable %events to the following string
>>> >> "test1<NEWLINE>test2<NEWLINE>test3".
>>> >> After the %events variable has been set, you can pass it as an input
>>> >> parameter to the Perl function invoked with 'lcall' action. If you are
>>> >> using 'eval', the %events variable can be directly accessed from the
>>> >> Perl code, since it is substituted before each compilation and
>>> >> execution. (Again, I would take the opportunity and recommend the use
>>> >> of 'lcall' instead of 'eval', since 'lcall' is much more efficient.)
>>> >>
>>> >> kind regards,
>>> >> risto
>>> >
>>> >
>>
>>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
