On 28 Jul 2008, at 16:27, John P. Rouillard wrote:
> In message <[EMAIL PROTECTED]>,
> Ton Voon writes:
>> With the patch below, the processing of this logfile came down from
>> 18
>> seconds to 5.5 seconds.
>>
>> The difference is the ContPreEval part is not run. If I move the
>> suppress return below this section, then the time goes back up to 18
>> seconds. However, I don't understand what this context part does.
>>
>> Is it safe to say that a suppress rule does not require this
>> ContPreEval?
>
> When a rule is evaluated three things happen (well simplified):
>
> 1 check the context to see if the rule should be evaluated (this
> pre execution context is indicated with a context expression in
> square brackets 'context = [ context ]'.
> 2 check the pattern according to ptype to see if the rule matches
> 3 check the context (if it's not just a pre-context) to see if the
> rule's actions should trigger
>
> IIRC ContPreEval is the Cont(ext)Pre(pattern)Eval(uation). That was
> added to permit the use of:
>
> context = [ test ]
>
> so that a rule can be skipped entirely based on the state of the SEC
> engine's contexts (since pattern application is the expensive part of
> every rule regardless of whether it matches).
>
> However, you are claiming that
>
> if ($ref->{"ContPreEval"}) {
>
> # if the value of the context expression is FALSE and the rule is
> # of type Pair*, look also for all active correlation operations
> # associated with the current rule and check if 2nd pattern
> matches
>
> if (!valid_formula($ref->{"Context"})) {
> if ( ($ref->{"Type"} == PAIR || $ref->{"Type"} ==
> PAIR_W_WINDOW)
> && scalar(%{$ref->{"Operations"}}) ) {
> if (process_rules2($ref) &&
> $ref->{"WhatNext2"} == DONTCONT) { return 1; }
> }
> next;
> }
>
> $context = $ref->{"Context"};
>
> }
>
> (from my SEC 2.4.1 release, not sure which version you are using), is
> causing a 3x slowdown. If you aren't using a context for the suppress
> rule, the true/false test of ($ref->{"ContPreEval"}) should be quite
> fast as it should be null. Hmm, maybe it needs a:
>
> defined($ref->{"ContPreEval"}) && $ref->{"ContPreEval"}
>
> or some such?
>
> Risto any ideas?
>
> In any case, the patch you supplied
>
> + if ($ref->{"Type"} == SUPPRESS) { ++$ref->{"MatchCount"};
> return 1; }
>
> if I understand it, doesn't evaluate the context expression at all, so
> if I had a rule:
>
> type=suppress
> context = from9to5
> ptype=substr
> pattern= charlie
>
> it would fire at all times not just when the from9to5 context was
> true.
> This is certainly incorrect.
Yes, I've realised this too.
It turns out that I've been using contexts when I should be using
separate rule files. So I had a rule file like:
type=suppress
context=logtype1
...
type=single
context=logtype1
type=suppress
context=logtype2
type=single
context=logtype2
With a startup of -input file=logtype2
Now this meant that all the rules for logtype1 had to be checked
before logtype2 was being run.
Moving the first suppress for logtype2 rules to the top of my testing
decreased the time to 6 seconds. So the slowdown is due to evaluating
lots of rules that are not applicable, rather than the context
checking. In summary, this looks like user error at the moment.
Ton
http://www.altinity.com
UK: +44 (0)870 787 9243
US: +1 866 879 9184
Fax: +44 (0)845 280 1725
Skype: tonvoon
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
