Hi!
I've been a little bit busy, so I could'nt answer before.
Below you can find my comments after yours.
---- On Sat, 19 Nov 2011 11:52:17 -0800 Risto
Vaarandi<risto.vaara...@gmail.com> wrote ----
> 2011/11/17 mindman101 <mindman...@zoho.com>:
> > Hi!
> >
> > Don't worry for the delay, your help is valuable.
> >
> > Now I have some thoughs clear but others not.
> >
> > Well, I've modified the EvenGroup example trying to get the idea of your
> > answers. So, if I'm right, the rule below should work for the idea of P1
> > followed by S1|S3|S2:
> >
> > type=EventGroup3
> > ptype=regexp
> > pattern=kernel: iptables:.* SRC=([d.]+)
> > varmap= ip=1
> > count=create OPER_$$+{ip}
> > ptype2=regexp
> > pattern2=sshd[d+]: Accepted .+ for (S+) from ([d.]+) port d+
> > ssh2
> > varmap2= user=1; ip=2
> > context2=OPER_$$+{ip}
> > ptype3=regexp
> > pattern3= sshd[d+]: Failed .+ for (S+) from ([d.]+) port d+
> > ssh2
> > varmap3= user=1; ip=2
> > context3=OPER_$$+{ip}
> > desc=Client $$+{ip} accessed a firewalled port and had difficulties
> > with logging in
> > action=logonly
> > window=30
> >
> > However, it seems that something it's not working properly. If an iptables
> > event arrives, it creates the context from the count field but doesn't
> > match
> > the next two events, so the rule never run the action.
> >
> > So, what is the mistake in this rule? Is the EvenGroup type the right rule
> > to match a first event no matter the following events? I mean, I'll always
> > have the same event as the first one who will be followed by N events that
> > could happen or no. In your last answer I understood it's mandatory that
> > all
> > events arrive at least once.
>
> hi Gaoke,
> I tried the rule quickly and it seems to work for me. I was typing in
> all input events from the keyboard, and I have labeled them with a
> suffix "<-- my input":
>
> [risto@localhost ~]$$ sec -conf=gaoke.conf -input=-
> SEC (Simple Event Correlator) 2.6.1
> Reading configuration from gaoke.conf
> 1 rules loaded from gaoke.conf
> Opening input file -
> Stdin connected to terminal, SIGINT can't be used for changing the logging
> level
> sshd[1]: Accepted password for risto from 10.1.1.1 port 12345 ssh2
> (<-- my input)
> sshd[2]: Failed password for risto from 10.1.1.1 port 12345 ssh2
> (<-- my input)
> kernel: iptables: SRC=10.1.1.1 (<-- my input)
> Creating context 'OPER_10.1.1.1'
> sshd[2]: Failed password for risto from 10.1.1.1 port 12345 ssh2
> (<-- my input)
> sshd[1]: Accepted password for risto from 10.1.1.1 port 12345 ssh2
> (<-- my input)
> Client 10.1.1.1 accessed a firewalled port and had difficulties with logging
> in
>
> As you can see, first the sshd login and login failure events were not
> matched by the rule, since the context for the 10.1.1.1 was not there
> yet. However, after iptables event it was created, and therefore the
> same input lines were now matched with an output event "Client
> 10.1.1.1 accessed a firewalled port and had difficulties with logging
> in" correctly produced. Do your input events look similar? There could
> be some subtle differences in log messages in your system which might
> require some tuning of the regular expressions.
>
:$, you right, I had a little difference in the input, so the pattern didn't
match. It worked ok!
>
> >
> > Besides, I want to report the matched events within the rule, I mean, P1
> > and
> > S3|S1|S2 or none if just P1 arrives, does it possible with EventGroup
> > type?
>
> if you want to report all the events that have matched the rule, this
> can be done with a context -- it is possible to store events into a
> context, and report the event store of a context later.
>
> >
> > Finally, for example, an event creates a context with a time window of 60
> > seconds, what happen if the same event arrives N times until the window
> > expires? will them create more contexts? Could I report those events at
> > the
> > end of the window?
>
> Basically, each matching iptables event will create a context in the
> rule above. If the context for the given IP address exists, the
> context will be recreated for the next 60 seconds, dropping all data
> related to the previous instance.
>
> From some of the questions above, I have got an understanding that you
> would actually like to report both lone instances of event P1, and
> optionally S1, S2, or S3 if they follow. If that's the case, the
> EventGroup might not be the best solution, since it expects *all*
> events to occur. If you want to act on lone instances of events,
> several Single rules that use the same context might be a much better
> idea.
> Can you actually provide us with a somewhat more detailed insight into
> the events you would like to process?
Off course, I can.
I've found a temporal solution but it's not the desired. The rule looks like
below:
###################### Start #####################
#Root cause creates a context
type=Single
continue=TakeNext
ptype=RegExp
pattern=\|(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\|\S+(linkDown)\sifIndex\s(\d+).*ifDescr\s+(?!(aps-|lag-|bundle-ima|\S+\sE1|\S+\sVT2|\d\/\d\/\d\.\d,\sDS0GRP)).*\|([A-Z]{3}\-\S+\-\S{1,4}|[A-Z]{3}\-\S+)\|(.*)\|Metro
varmap= ip=1
desc=$0
action=create linkDown_$+{ip} 5
#Rule to match events after the root cause
type=PairWithWindow
ptype=RegExp
pattern=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\S+\slinkDown\sifIndex\s(\d+).*ifDescr\s(\w+)\|(\S+)\|
varmap= ip=1; ifIndex=2; Descr=4; devname=7;
desc=$+{ip}, $+{devname} has a linkDown
action=none
continue2=TakeNext
ptype2=regexp
pattern2=$+{ip}.+$+{devname}.+vRtrRsvpIfStateChange|$+{ip}.+$+{devname}.+vRtrRsvpIfNbrStateDown|$+{ip}.+$+{devname}.+vRtrLdpInstanceStateChange|$+{ip}.+\|$+{devname}.+vRtrMplsIfStateChange
context2=linkDown_%2
desc2= $+{ip}, $+{devname} has a linkDown followed by one or more events
action2=logonly
window=8
###################### End #####################
Also, I have a modified version of this with the Eventgroup type as follow:
############### Start ################
type=EventGroup3
ptype=regexp
pattern=
(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\S+\slinkDown\sifIndex\s(\d+).*ifDescr\s(\w+)\|(\S+)\|
varmap= ip=1
count= create linkDown_$+{ip} 15
ptype2= regexp
pattern2= (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).+\|(\S+)\|.+vRtrRsvpIfStateChange
varmap2= ip=1; devname=2
context2= linkDown_$+{ip}
ptype3= regexp
pattern3=
(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).+\|(\S+)\|.+vRtrRsvpIfNbrStateDown
varmap3= ip=1; devname=2
context3= linkDown_$+{ip}
desc= linkDown on $+{devname} and Ip address $+{ip} followed by two events
action= logonly
window= 15
#################### End ###################
The idea is to catch an interface linkDown and its associated protocols that
comes down after.
The first one works great but the pattern2 matches only the first it match. I
have used several Or on it. I want to match one, two or more events that
arrived and report them after rule match.
The second one, as you says, needs at least one event for each pattern. However
that's not always true for me, so the rule never match.
Can you suggest me a possible solution to catch the son events following the
root cause whatever the number that arrive after? So, in this way, I'll be able
to match the root cause and all its symptons events.
>
> kind regards,
> risto
Thanks for you help!
Regards,
Gaoke.
>
> >
> > Again, thanks for your help.
> >
> > You have made a great work and it is helping us at work.
> >
> > Gaoke.
> >
> > ---- On Fri, 11 Nov 2011 17:46:32 -0800 Risto Vaarandi
> > <risto.vaara...@gmail.com> wrote ----
> >
> > hi,
> > sorry for the somewhat late answer, I am traveling at the moment and
> > without permanent internet connectivity. I'll try to answer your
> > questions below:
> >
> > 2011/11/9 mindman101 <mindman...@zoho.com>:
> >> Hi Risto,
> >> Thanks for your answer. It helped.
> >> I was checking the sec manual for the EventGroup example and I tried it
> >> several times to understand its working.
> >> So, now I have some questions because I get a little confused. This is
> >> the
> >> example:
> >> type=EventGroup3
> >> ptype=regexp
> >> pattern=sshd[d+]: Failed .+ for (S+) from ([d.]+) port d+
> >> ssh2
> >> varmap= user=1; ip=2
> >> count=alias OPER_$$+{ip} LOGIN_FAILED_$$+{user}_$$+{ip}
> >> ptype2=regexp
> >> pattern2=sshd[d+]: Accepted .+ for (S+) from ([d.]+) port d+
> >> ssh2
> >> varmap2= user=1; ip=2
> >> context2=LOGIN_FAILED_$$+{user}_$$+{ip}
> >> ptype3=regexp
> >> pattern3=kernel: iptables:.* SRC=([d.]+)
> >> varmap3= ip=1
> >> desc=Client $$+{ip} accessed a firewalled port and had
> >> difficulties
> >> with logging in
> >> action=pipe %t: %s /bin/mail root@localhost
> >> init=create OPER_$$+{ip}
> >> slide=delete OPER_$$+{ip}; reset 0
> >> end=delete OPER_$$+{ip}
> >> window=120
> >> Questions:
> >> 1. According to the example, you say that "at least an iptables event
> >> was observed for the IP *and* and least one login failure for a user
> >> which was followed by a successful login for the same user" however I
> >> tested
> >> it but it didn't trigger the action unless iptables and login failure
> >> occur
> >> twice followed by one succesful login. I tried it cutting and pasting in
> >> the
> >> standard input, could I make a mistake?
> >
> > there are no thresholds specified in this rule, which means that it is
> > sufficient if each event occurs once. However, the login success must
> > come after the login failure for the action to be triggered. If you
> > have two iptables events and two login failures without login success,
> > the action will not be triggered, since one condition is not fulfilled
> > (there must be at least one login success in the time window).
> >
> >> 2. I see varmap lets get readable names for variables matched in the
> >> pattern
> >> field. Can I use the same variable name throughout the next rules? I need
> >> to
> >> match the same IP on all events.
> >
> > No, that's not possible. Match variables are created by the regular
> > expression from the current rule, but since the following rules can
> > create their own variables, this will lead to clashes, ambiguities,
> > and referencing issues. Therefore, match variables can be only used
> > within the scope of the current rule.
> >
> >> 3. I've asked you about an event P1 as the first event and S1|S3|S2|Sn as
> >> its symptoms. You suggested create a context for the P1 event from the
> >> count
> >> field, will have the same effect if I create the context with the action
> >> field? what's the difference?
> >
> > the 'count' field will trigger an action each time the corresponding
> > regular expression matches an event. This allows for reacting to one
> > event type in a certain way. On the other hand, the content of the
> > 'action' field gets executed when all match conditions for all event
> > types are fulfilled.
> >
> >> 4. why use context alias?
> >
> > This is done, in order to delete all contexts at the end of the event
> > correlation operation (ie, garbage collection). The deletion is done
> > from the 'end' field.
> >
> > regards,
> > risto
> >
> >> Thanks again for your help, meanwhile I will keep playing with sec.
> >> Gaoke.
> >>
> >> ---- On Fri, 04 Nov 2011 15:37:34 -0700 Risto Vaarandi
> >> <risto.vaara...@gmail.com> wrote ----
> >>
> >> hi,
> >> yes, the EventGroup rule is probably the best solution for this case,
> >> since it does matching for unordered event groups. However, in order
> >> to verify that P1 is the first event to match the rule and trigger the
> >> event correlation operation, you could create a context from the
> >> 'count' field for P1. Then, you can check for the presence of this
> >> context when each S event arrives, which verifies that P1 has been
> >> observed in the past.
> >> hope this helps,
> >> risto
> >>
> >> 2011/11/4 mindman101 <mindman...@zoho.com>:
> >>>
> >>> Hello list!
> >>> I've been using SEC for a while just configuring single rules. However,
> >>> now
> >>> I'm figuring out how to configure a rule that match a problem and then,
> >>> match its symptoms. I mean, for example, match the event P1 and next
> >>> wait
> >>> for time window where S2|S5|S1|S3|S4 will arrive. As you can see, no
> >>> matter
> >>> the arrival order.
> >>> I think the new event type EventGroup could help but how?
> >>> Thanks for your help.
> >>> Gaoke.
> >>>
> >>>
> >>> ------------------------------------------------------------------------------
> >>>
> >>> RSA(R) Conference 2012
> >>> Save $$700 by Nov 18
> >>> Register now
> >>> http://p.sf.net/sfu/rsa-sfdev2dev1
> >>> _______________________________________________
> >>> Simple-evcorr-users mailing list
> >>> Simple-evcorr-users@lists.sourceforge.net
> >>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
> >>>
> >>>
> >>
> >>
> >>
> >> ------------------------------------------------------------------------------
> >>
> >> RSA(R) Conference 2012
> >> Save $$700 by Nov 18
> >> Register now
> >> http://p.sf.net/sfu/rsa-sfdev2dev1
> >> _______________________________________________
> >> Simple-evcorr-users mailing list
> >> Simple-evcorr-users@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
> >>
> >>
> >
> >
> > ------------------------------------------------------------------------------
> >
> > All the data continuously generated in your IT infrastructure
> > contains a definitive record of customers, application performance,
> > security threats, fraudulent activity, and more. Splunk takes this
> > data and makes sense of it. IT sense. And common sense.
> > http://p.sf.net/sfu/splunk-novd2d
> > _______________________________________________
> > Simple-evcorr-users mailing list
> > Simple-evcorr-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
> >
> >
>
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
