This looks like a special case of the (slightly) more general rule that I
fondly remember implementing in various ways for several in-house application
monitoring systems during the 90s:
When n or more events of type X occur within a time window m seconds wide,
and no message of type Y has been generated in the last t seconds, then
generate a new message of type Y.
The implementation is simple: you maintain a queue of time-stamped events of
type X and a time-stamp (initially 0) for the "last message of type Y". Each
time a new event of type X is added to "fresh" end of the queue you first
discard all events more than m seconds old from the "stale" end of the queue
and check to see if n or more events remain. If so, then check to see if the
"last message of type Y" timestamp is more than t seconds old - if so then
generate a new message of type Y and update the timestamp.
This seems like such a fundamental event-correlation pattern than IMHO SEC
should support it "natively", rather than relying on an external script (or set
of scripts).
Fun factoid: this a pattern that I hear was originally implemented by TelCos in
*analog* form 25+ years earlier - the event queue took the form of a
"capacitor" (actually a spring loaded wheel) which "bled" at a fixed rate
(unwinding the wheel) and the incoming "events" would load the capacitor
(applying torque to the wheel), and when the capacitor hit a certain limit it
would ring a bell.
Cheers,
Peter Wolfenden
-----Original Message-----
From: Pierre Vigneras [mailto:[email protected]]
Sent: Tuesday, February 02, 2010 2:26 AM
To: Risto Vaarandi
Cc: [email protected]
Subject: Re: [Simple-evcorr-users] Fw: Complex rule ? Any idea ?
Thanks for your reply,
I will sketch out some use cases taken from your example:
|----'----'----'----|----'----'----'----|
A B C B
In that case, first three distincts events are ABC, therefore the action is
launched. Then, B is also in the window, the action is launched again
(threshold is for 3 distinct events, other events within the time interval
always raise an action, whether they are distinct or not from previous
events). Therefore, result is ABC,B.
|----'----'----'----|----'----'----'----|
A B C B
In that case, result is BCB, obviously: the time window has been shifted
towards first B.
|----'----'----'----|----'----'----'----|
A B C B
In that case, nothing happens since there are no 3 distinct events within a 4
seconds time interval.
We are still unable to find a satisfactory solution.
Thanks again for your reply.
Le vendredi 29 janvier 2010 14:50:55, Risto Vaarandi a écrit :
> hi Pierre,
>
> first of all, sorry for the late reply :(
> I had a look at the problem description and it seems to me that it would
> be quite hard to implement the solution with SingleWithThreshold rule (a
> SEC standard rule that would be an easier solution than having custom
> Perl code). Unfortunately, in your case there is a requirement to
> implement window shifting not when the first event appears to be out of
> window, but rather on the occurrence of certain event.
>
> Also, given the problem description, it is hard to envision how the
> window shifting should be implemented for certain cases. For example,
> what if you see events in the following order: A, B, C, B. Should the
> window be moved to C with dropping event A, or should no shifting done
> at all since we would lose A otherwise?
>
> regards,
> risto
>
> On 01/20/2010 05:53 PM, Pierre Vigneras wrote:
> > Hum, sorry, but it seems that this ascii-art representation is really a
> > problem !! ;-)
> >
> > Actually, the quotation makes the events on 2 lines where they should
> > only be on one line. Therefore, the result is probably not
> > understandable.
> >
> > Here is what it should look like. I also attach a screenshot in case it
> > does not work.
> >
> > |----'----'----'----|----'----'----'----|
> >
> > A B A B A C D E
> >
> > Best Regards.
> >
> > PS: Hope it will work!
> >
> > Le mercredi 20 janvier 2010 15:58:33, Risto Vaarandi a écrit :
> >> I am forwarding this mail on behalf of a list member.
> >> risto
> >>
> >> --- On Wed, 1/20/10, Pierre Vigneras<[email protected]> wrote:
> >>> From: Pierre Vigneras<[email protected]>
> >>> Subject: Complex rule ? Any idea ?
> >>> To: [email protected]
> >>> Cc: [email protected]
> >>> Date: Wednesday, January 20, 2010, 3:25 PM
> >>> Dear all,
> >>>
> >>> Here is a problem we can't find a "good" solution to. Any
> >>> idea in that regard
> >>> would be appreciated! ;-)
> >>>
> >>> We would like to implement the following functionnal use
> >>> case: "when at least
> >>> n warning alerts from distinct hosts are detected within a
> >>> given time window,
> >>> then an action should be triggered on those hosts during
> >>> that time window".
> >>>
> >>> Warning alerts have a form that allows the identification
> >>> of sending host such
> >>> as (simplified):
> >>>
> >>> timestamp host WARNING
> >>> 1253021598 phebus WARNING
> >>>
> >>> Let's take a sample data (please use a fixed font size such
> >>> as monospace for
> >>>
> >>> the following):
> >>> |----'----'----'----|----'----'----'----|
> >>>
> >>> A B A B
> >>> A C D E
> >>>
> >>>
> >>> Where |----' represents one unit of time (second), |...| a
> >>> window time,
> >>> A, B, C, D warning alerts sent by host A, B, C and D
> >>> respectively (actually,
> >>> received by SEC).
> >>>
> >>> In our use case, with a window time of 4 seconds and n=3,
> >>> we would like an
> >>> action to be triggered for hosts B,A,C and D (a script is
> >>> first called with
> >>> parameter ABC, and another time with parameter D). The
> >>> reasoning is the
> >>> following:
> >>>
> >>> On first A, a time window is "opened". On first B, we
> >>> "increment the alert
> >>> counter" (2: AB). On second A, since we already have an A,
> >>> the beginning of
> >>> the window is shifted toward the first B (counter=2: BA).
> >>> When the second B is
> >>> encoutered, there is already a B, therefore, the window is
> >>> shifted toward the
> >>> second A (counter=2, AB). On third A, we already have an A,
> >>> therefore, the
> >>> window is also shifted toward B (counter=2, BA). On event
> >>> C, we now have 3
> >>> different hosts, therefore, the action is triggered (script
> >>> is called with
> >>> parameter BAC). Finally, when D is received, we are still
> >>> in a time window of
> >>> 4 seconds (between last B and D, we only have 3 seconds).
> >>> Therefore, the
> >>> action is also triggered (script is called with parameter
> >>> D). Finally, one
> >>> second after D, the time window is closed. The receipt of E
> >>> starts a new one
> >>> and the process starts again.
> >>>
> >>> At first, we implemented the use case using a
> >>> SingleWithThreshold rule, but it
> >>> does not work because it forgets following events after the
> >>> threshold has been
> >>> reached. Then we used a combination of SingleWithThreshold
> >>> rule followed by a
> >>> Single rule using a takenext=continue and a context, but it
> >>> does not work
> >>> either because we are unable to make SEC understand what we
> >>> want (counting
> >>> only "distincts alerts").
> >>>
> >>> We are thinking of a solution based on a lot of Perl code
> >>> inside a Single
> >>> rule, but this solution looks pretty ugly to us (since most
> >>> of the rule
> >>> smartness would be in the custom Perl code, not in SEC). We
> >>> wonder if there is
> >>> a simple way to achieve this in SEC.
> >>>
> >>> Thanks again for any help.
> >>> Best Regards.
> >>
> >> ------------------------------------------------------------------------
> >>--- --- Throughout its 18-year history, RSA Conference consistently
> >> attracts the world's best and brightest in the field, creating
> >> opportunities for Conference attendees to learn about information
> >> security's most important issues through interactions with peers,
> >> luminaries and emerging and established companies.
> >> http://p.sf.net/sfu/rsaconf-dev2dev
> >> _______________________________________________
> >> Simple-evcorr-users mailing list
> >> [email protected]
> >> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
> >
> > -------------------------------------------------------------------------
> >----- Throughout its 18-year history, RSA Conference consistently attracts
> > the world's best and brightest in the field, creating opportunities for
> > Conference attendees to learn about information security's most important
> > issues through interactions with peers, luminaries and emerging and
> > established companies. http://p.sf.net/sfu/rsaconf-dev2dev
> >
> >
> >
> > _______________________________________________
> > Simple-evcorr-users mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>
--
HPC R&D
BARD / Bruyères-Le-Châtel
Bull, Architect of an Open World TM (www.bull.com)
Direct Line: +33 (0)1 69 26 56 95
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
