Hi Risto,
This was more than help, thank you very much.
I made some searches on internet, and it seems difficult to find a smart
algorithm to match strings similarity.
To fill this need, I implemented a keyword/expression dictionnary using
perl arrays. In my context, I added a condition to check whether both var4
fields match at least one keyword/expression.
Jon.
2015-04-15 15:14 GMT+02:00 Risto Vaarandi <[email protected]>:
>
>
> 2015-04-15 10:07 GMT+03:00 Jonathan Snowe <[email protected]>:
>
>> Well,
>>
>> It is exaclty the rule I was building, based on your previous answer :-)
>>
>> Now, what I'd like to know is, if I have a 4th field containing some text
>> in my events, would it be possible to add a condition of pattern matching ?
>> This would result as a condition like this: S1.var1 != S2.var1 && S1.var2
>> == S2.var2 && S1.var3 == S2.var3 && S1.var4 ~~ S2.var4
>>
>> This should match this for example: "Brute force on SSH observed" and
>> "SSH brute force"
>>
>> I don't know how to handle this condition, and I am not sure that SEC is
>> able to do this (even with a script call for example).
>>
>
> ...this particular scenario is somewhat tricky to implement with two
> Single rules, since the notion of ~~ is blurry, and sec does not support
> fuzzy context name matching. Therefore, the fuzzy matching has to be
> implemented for match variables, and in order to retain them between two
> matches, Pair rule is the best option:
>
> type=Pair
> ptype=RegExp
> pattern=var1=(\S+) var2=(\S+) var3=(\S+) var4=(\S+)
> context=!EVENT_WITH_VARS_OBSERVED_$2_$3
> desc=$2 $3
> action=create EVENT_WITH_VARS_OBSERVED_$2_$3
> ptype2=RegExp
> pattern2=var1=(\S+) var2=($2) var3=($3) var4=(\S+)
> context2=$1 %1 -> ( sub { $_[0] ne $_[1] } ) && \
> $4 %4 -> ( sub { $_[0] eq $_[1] } )
> desc2=second event
> action2=delete EVENT_WITH_VARS_OBSERVED_$2_$3; \
> write - There was a match for $2 and $3: 1st var - $1 %1, 4th var
> $4 %4
> window=0
>
> In this rule we have again used the EVENT_WITH_VARS_OBSERVED_$2_$3 context
> for letting 'pattern' match only once. All the comparisons between match
> variables from two different matches ($1 and %1 , but also $4, %4) are done
> in context2 field. The context expression first verifies that $1 and %1 are
> not equal (done with a Perl function
> sub { $_[0] ne $_[1] } )
>
> Also, instead of fuzzy matching I've configured an exact matching between
> $4 and %4:
> $4 %4 -> ( sub { $_[0] eq $_[1] } )
>
> In order to implement fuzzy matching, you first have to define it, and
> then the 'eq' operator simply has to be replaced with the newly defined
> fuzzy matching operator.
>
> Since there are many Perl modules around which implement huge number of
> string matching functions, you will probably end up using external Perl
> module. In order to load one at startup, have a look into "Perl Integration
> "section in sec official documentation, or the FAQ, for example:
> http://simple-evcorr.sourceforge.net/FAQ.html#14
> http://simple-evcorr.sourceforge.net/man.html#lbBB
>
> hope this helps,
> risto
>
>
>
>
>>
>> Thank you,
>>
>>
>> 2015-04-14 16:41 GMT+02:00 Risto Vaarandi <[email protected]>:
>>
>>> You can address the problem by straightforward modification of the
>>> previous ruleset -- just include $2 and $3 variables in context names:
>>>
>>> type=Single
>>> ptype=regexp
>>> pattern=var1=(\S+) var2=(\S+) var3=(\S+)
>>> context=!EVENT_WITH_VARS_OBSERVED_$2_$3
>>> desc=var1 $1 has been observed
>>> action=create EVENT_WITH_VARS_OBSERVED_$2_$3; \
>>> alias EVENT_WITH_VARS_OBSERVED_$2_$3 VAR1_$1_OBSERVED_$2_$3; \
>>> fill EVENT_WITH_VARS_OBSERVED_$2_$3 $1
>>>
>>> type=Single
>>> ptype=regexp
>>> pattern=var1=(\S+) var2=(\S+) var3=(\S+)
>>> context=EVENT_WITH_VARS_OBSERVED_$2_$3 && !VAR1_$1_OBSERVED_$2_$3
>>> desc=var1 $1 has been observed
>>> action=copy EVENT_WITH_VARS_OBSERVED_$2_$3 %previous; \
>>> write - mytest: var1=$1 var1_previous=%previous var2=$2 var3=$3; \
>>> delete EVENT_WITH_VARS_OBSERVED_$2_$3
>>>
>>>
>>> Also, the first rule writes the initial value of var1 into the
>>> EVENT_WITH_VARS_OBSERVED_$2_$3 context, and retrieves it in the second rule
>>> after a different value for var1 has been observed. Both values are then
>>> included in the string which gets written to standard output.
>>>
>>> hope this helps,
>>> risto
>>>
>>> 2015-04-14 15:00 GMT+03:00 Jonathan Snowe <[email protected]>:
>>>
>>>> Actually, the problem is a bit more complex, I might have mis-explained
>>>> it, let me try again.
>>>>
>>>> What I want to catch is events with the following behaviour:
>>>> Source1: var1="var1_val1" var2="var2_val1" var3="var3_val1"
>>>> Source2: var1="var1_val2" var2="var2_val1" var3="var3_val1"
>>>>
>>>> AKA, same values on var2 and var3 for both events, and a different
>>>> value on var1.
>>>> The thing I can't know is what will be the value of these fields at
>>>> first.
>>>>
>>>> When the matching is done, I'd like to write somewhere a line with
>>>> "var1_val1 var1_val2 var2_val1 var3_val1" for example.
>>>>
>>>> I don't know if it's clearer.
>>>>
>>>> Thank you for your anwser,
>>>>
>>>> Jon.
>>>>
>>>> 2015-04-14 11:16 GMT+02:00 Risto Vaarandi <[email protected]>:
>>>>
>>>>> Jonathan,
>>>>> you can accomplish the task with the following two single rules:
>>>>>
>>>>> type=Single
>>>>> ptype=regexp
>>>>> pattern=var1=(\S+) var2=\S+ var3=\S+
>>>>> context=!EVENT_WITH_VARS_OBSERVED
>>>>> desc=var1 $1 has been observed
>>>>> action=create EVENT_WITH_VARS_OBSERVED; \
>>>>> alias EVENT_WITH_VARS_OBSERVED VAR1_$1_OBSERVED
>>>>>
>>>>> type=Single
>>>>> ptype=regexp
>>>>> pattern=var1=(\S+) var2=\S+ var3=\S+
>>>>> context=!VAR1_$1_OBSERVED
>>>>> desc=var1 $1 has been observed
>>>>> action=write - an event with a different var1 value has been observed;
>>>>> \
>>>>> delete EVENT_WITH_VARS_OBSERVED
>>>>>
>>>>> The first event with var1, var2 and var3 will create a context
>>>>> EVENT_WITH_VARS_OBSERVED with an alias name which holds the value of var1.
>>>>> After the context EVENT_WITH_VARS_OBSERVED has been created, further
>>>>> events
>>>>> are passed to the second rule which checks for the presence of the alias
>>>>> name. If the alias name for var1 value is missing, the second rule matches
>>>>> and writes a message to standard output.
>>>>>
>>>>> Note that the order of the rules is important, and if you want to
>>>>> change this, the 'context' field of the second rule should be written as
>>>>> context=EVENT_WITH_VARS_OBSERVED && !VAR1_$1_OBSERVED
>>>>>
>>>>>
>>>>> You can rewrite the above two rules into a one Pair rule, but I would
>>>>> personally prefer to have two Single rules, since it's a bit clearer.
>>>>> Also,
>>>>> event correlation logic is implemented with contexts, so the rule having
>>>>> Pair instead of Single rules doesn't really matter here.
>>>>>
>>>>> kind regards,
>>>>> risto
>>>>>
>>>>> 2015-04-14 11:05 GMT+03:00 Jonathan Snowe <[email protected]>:
>>>>>
>>>>>> Hello guys,
>>>>>>
>>>>>> I'd like some help on a little problem have to solve.
>>>>>> Here's my situation:
>>>>>> I have multiple sources, receiving similar events. SEC is configured
>>>>>> to listen to these inputs.
>>>>>>
>>>>>> Events are of this type:
>>>>>> var1="" var2="" var3=""
>>>>>>
>>>>>> What I want to do is, trigger an action if a first event appears with
>>>>>> "whatever the value of these fields" (large regex), and a second event
>>>>>> appears with "var1 different from the FIRST var1".
>>>>>>
>>>>>> I tried to configure a Pair rule, but the problem is that my two
>>>>>> events trigger the large pattern1 regex and never trigger the pattern2.
>>>>>>
>>>>>> The objective is then to write somewhere vars from the event 1 and
>>>>>> event 2.
>>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>> --
>>>>>> *Jon.*
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>>>>>> Develop your own process in accordance with the BPMN 2 standard
>>>>>> Learn Process modeling best practices with Bonita BPM through live
>>>>>> exercises
>>>>>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>>>>>> event?utm_
>>>>>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>>>>>> _______________________________________________
>>>>>> Simple-evcorr-users mailing list
>>>>>> [email protected]
>>>>>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Jon.*
>>>>
>>>
>>>
>>
>>
>> --
>> *Jon.*
>>
>
>
--
*Jon.*
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
