Hello,
back with some questions:
1. As long as we exclude the longer window expiration case (so we just want
to suppress events for 10 secs), why you suggest to use two Single rules
instead of a SingleWithSuppress ?
I’m referring to these rules:
type=Single
ptype=RegExp
pattern=event_(\w+)
context=!SUPP_$1
desc=react to event $1 and set up suppression
action=write - We have observed event $1; create SUPP_$1 10
type=Single
ptype=RegExp
pattern=event_(\w+)
context=SUPP_$1
desc=do not react to event $1 and only update its suppression context
action=set SUPP_$1 10
1. Does SingleWithSuppress use a sliding window ? According to my latest
tests it does not, so it should be equivalent to your two Single
rules..shouldn’t it ?
1. For the more complex scenario, when you introduced also a MAXLIFE
context of 60 secs, so i’m referring to:
type=Single
ptype=RegExp
pattern=event_(\w+)
context=!SUPP_$1
desc=react to event $1 and set up suppression
action=write - We have observed event $1; \
create SUPP_$1_MAXLIFE 60 ( delete SUPP_$1 ); \
create SUPP_$1 10 ( delete SUPP_$1_MAXLIFE )
type=Single
ptype=RegExp
pattern=event_(\w+)
context=SUPP_$1
desc=do not react to event $1 and only update its suppression context
action=set SUPP_$1 10
..as far as i can see, the shortest 10secs context will always expire first,
thus deleting the MAXLIFE as well…so in the end this behaves just like the
longer context does not exist at all. It seems equivalent to your rules in
bullet 1), am i wrong ?
I’ve been generating identical events every two seconds, and with those two
rules i get 1 line every 10 seconds…instead i would expect to get one every 60
seconds…because i expected the 10sec window to be constantly sliding thus
suppressing everything until the 60 seconds windows expires and basically reset
everything.
What am i missing ?
M.
Da: Spelta Edoardo
Inviato: martedì 14 marzo 2023 17:21
A: Eric V. Smith <[email protected]>; Risto Vaarandi <[email protected]>
Cc: [email protected]
Oggetto: R: [Simple-evcorr-users] Duplicate suppression and rearming
Absolutely amazing!
@Risto Vaarandi<mailto:[email protected]> I need some time to understand
deeply what you propose and if it’s applicable to hundreds of conditions, but
it sounds really promising !!
I’ll post some feedback as soon as i can
Thanks a lot !
M.
Da: Eric V. Smith <[email protected]<mailto:[email protected]>>
Inviato: martedì 14 marzo 2023 17:06
A: Risto Vaarandi <[email protected]<mailto:[email protected]>>;
Spelta Edoardo
<[email protected]<mailto:[email protected]>>
Cc:
[email protected]<mailto:[email protected]>
Oggetto: Re: [Simple-evcorr-users] Duplicate suppression and rearming
Risto:
You (and sec) really are a treasure. Thanks for all you do. I don't currently
have the issue that Mugugno mentions, but I might in the future. Your
explanation below is excellent.
Eric
On 3/14/2023 11:52 AM, Risto Vaarandi wrote:
hi Mugugno,
thanks for clarifying your scenario! Perhaps I can first explain a simpler
ruleset that is addressing a part of your scenario, and then provide a slightly
more complex solution that addresses your scenario fully.
Let's assume that you want to react to the first instance of some event, and
then keeping repeated instances of the same event suppressed, provided that the
time frame between consecutive suppressed events is at most 10 seconds. In
order to do that, you could utilize this ruleset of 2 Single rules:
type=Single
ptype=RegExp
pattern=event_(\w+)
context=!SUPP_$1
desc=react to event $1 and set up suppression
action=write - We have observed event $1; create SUPP_$1 10
type=Single
ptype=RegExp
pattern=event_(\w+)
context=SUPP_$1
desc=do not react to event $1 and only update its suppression context
action=set SUPP_$1 10
These two rules process events in the following format: event_<sometext> (for
example, event_X). The rules utilize the contexts SUPP_* for suppressing events
of a given type. For example, for suppressing repeated instances of event_X,
the context SUPP_X is used.
The first rule matches the event and checks if the suppression context is
present. If the suppression context is not found, we are dealing with the first
event we have seen, or the first event after previous suppression has ended.
Therefore, the rule prints a notice "We have observed event *" to standard
output. Also, after executing this action, the rule activates the suppression
for the given event type. For example, if the event was event_X, the context
SUPP_X is created. Note that the lifetime of the SUPP_X context is 10 seconds.
If no subsequent events event_X will be observed during these 10 seconds,
SUPP_X context will simply expire, and any further event_X will again trigger a
notice written to standard output.
However, if some event_X will be seen within 10 seconds, it will match the
second rule instead of the first one. The second rule does not produce any
notice (in other words, it is suppressing event_X without action), but it is
rather extending the lifetime of the suppression context SUPP_X for additional
10 seconds starting from the *current moment*. This means that as long as
events event_X will keep arriving with the time internal not larger than 10
seconds between them, the lifetime of the SUPP_X context will get extended for
another 10 seconds on the arrival of each event_X.
Obviously, the above ruleset has the following problem -- if events event_X
keep on arriving at least once per 10 seconds indefinitely, the suppression of
event_X will never end. For addressing this problem, you can use a further
development of the previous ruleset that sets an upper limit of 60 seconds for
the suppression:
type=Single
ptype=RegExp
pattern=event_(\w+)
context=!SUPP_$1
desc=react to event $1 and set up suppression
action=write - We have observed event $1; \
create SUPP_$1_MAXLIFE 60 ( delete SUPP_$1 ); \
create SUPP_$1 10 ( delete SUPP_$1_MAXLIFE )
type=Single
ptype=RegExp
pattern=event_(\w+)
context=SUPP_$1
desc=do not react to event $1 and only update its suppression context
action=set SUPP_$1 10
When event_X is observed and the suppression is currently not active, the first
rule matches and writes a notice to standard output. In addition to context
SUPP_X, the context SUPP_X_MAXLIFE gets created with the lifetime of 60 seconds
(that is the maximum lifetime of the event suppression process). Note that the
lifetime of this context is *not* extended by the ruleset, and also, when this
context expires, it will delete the SUPP_X context that implements the
suppression. This means that the suppression can never be active for more than
60 seconds after we saw the first event that triggered a notice message to
standard output.
Also, note that when SUPP_X context expires, it will delete the SUPP_X_MAXLIFE
context, since this context is no longer necessary. In other words, whatever
context (SUPP_X or SUPP_X_MAXLIFE) expires first, it will always delete the
other context, in order to continue with the clean state for the given event
type.
By using the above technique with two contexts, you can implement the scenario
you want. Also, I hope that the examples I have provided are clear enough and
helpful for illustrating the main point.
kind regards,
risto
Kontakt Spelta Edoardo
(<[email protected]<mailto:[email protected]>>)
kirjutas kuupäeval T, 14. märts 2023 kell 16:58:
Hello,
you definitely described my scenario better than me.
This is the one i’m after:
0, 2, 23, 54 -- events at minutes 2 and 23 should be suppressed, but the event
at minute 54 should trigger an action, since it happens 31 minutes after the
event 23
0, 11, 32, 44, 61 -- events at minutes 11, 32 and 44 should be suppressed,
since they are separated by less than 30 minutes, but the event at minute 61
should trigger an action, since the suppression can't last for longer than 1
hour
Any ideas ?
M.
Da: Risto Vaarandi <[email protected]<mailto:[email protected]>>
Inviato: martedì 14 marzo 2023 15:52
A: Spelta Edoardo
<[email protected]<mailto:[email protected]>>
Cc:
[email protected]<mailto:[email protected]>
Oggetto: Re: [Simple-evcorr-users] Duplicate suppression and rearming
hi Mugugno,
let me clarify your scenario a bit, considering the diagram from your post:
T1---------------------------T27-----------T30-------------------T57-------------T60----------T61
Event suppr suppr suppr suppr suppr
Event
Do you want the suppression to start after the time moment T1 when the first
event was observed and run for 1 hour, so that this window would not be sliding?
Or do you rather want the time window between two *consecutive* events to be
less than N minutes in order to suppress them, so that suppression would work
during max 1 hour? For example, suppose N=30minutes and consider the following
events happening at these minutes:
0, 2, 23, 54 -- events at minutes 2 and 23 should be suppressed, but the event
at minute 54 should trigger an action, since it happens 31 minutes after the
event 23
0, 11, 32, 44, 61 -- events at minutes 11, 32 and 44 should be suppressed,
since they are separated by less than 30 minutes, but the event at minute 61
should trigger an action, since the suppression can't last for longer than 1
hour
I would appreciate it if you could provide some additional comments on your
scenario.
kind regards,
risto
Hello,
i’ve recently been struggling with SEC to implement something for this specific
use case: suppressing specific entries read from a syslog for a certain amount
of time (say 30min) but make sure that after a longer time (say 1h), if they
are still being received, i’m getting a new one.
If these events are being received continously they will always be suppressed
because the windows will be sliding accordingly but i’m trying to find a way to
make it stop after 1h
The sequence should look like this:
T1---------------------------T27-----------T30-------------------T57-------------T60----------T61
Event suppr suppr suppr suppr suppr
Event
I tried combining a SingleWithSuppress (which is ok for the suppression part)
and context expiration but i cannot find a working solution.
Anybody already faced this use case ?
Any help appreciated!
Thanks and regards,
Mugugno
_______________________________________________
Simple-evcorr-users mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
_______________________________________________
Simple-evcorr-users mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
