hi Thin, the ruleset you have could be improved a bit. Firstly, the 'desc' field of the PairWithWindow rule is set to $0 which holds the entire matching line. However, since 'desc' field sets the ID of event correlation operation(s) started by this rule, each line with a distinct timestamp would create a separate operation. For instance, the two lines you have provided as example input lines
2015-09-10T13:02:59+01:00 <local4.warning> gbsole001a event: SYSTEM: SYSTEM_CFGSYNC_DOWN: - - The config-sync feature is operationally down. 2015-09-10T13:03:02+01:00 <local4.warning> gbsole001a event: SYSTEM: SYSTEM_CFGSYNC_DOWN: - - The config-sync feature is operationally down. would start two event correlation operations, since timestamps 2015-09-10T13:02:59+01:00 and 2015-09-10T13:03:02+01:00 are different. In order to get a deeper insight how the 'desc' field can be used for changing the behavior of the rule, I would advise to look into the "Event Correlation Operations" section in the sec man page: http://simple-evcorr.github.io/man.html#lbAX For setting the 'desc' field correctly, you would need to answer the following question -- should SYSTEM_CFGSYNC_DOWN and SYSTEM_CFGSYNC_UP events be associated by hostname (or some other event property)? Since your example writes output events to files which are arranged into separate directories by hostname, I assume that the hostname property is important for you. In other words, the following events should not be handled by the same PairWithWindow operation, since they are not originating from the same host: 2015-09-10T13:02:59+01:00 <local4.warning> gbsole001a event: SYSTEM: SYSTEM_CFGSYNC_DOWN: - - The config-sync feature is operationally down. 2015-09-10T13:03:02+01:00 <local4.info> myhost985a event: SYSTEM: SYSTEM_CFGSYNC_UP: - - The config-sync feature is operationally up. On the other hand, the following events should be processed by the same PairWithWindow operation, since the hostname is 'test001a' for both events: 2015-09-10T13:02:59+01:00 <local4.warning> test001a event: SYSTEM: SYSTEM_CFGSYNC_DOWN: - - The config-sync feature is operationally down. 2015-09-10T13:03:02+01:00 <local4.info> test001a event: SYSTEM: SYSTEM_CFGSYNC_UP: - - The config-sync feature is operationally up. In order to take hostnames into account, your ruleset could be modified as follows: #1 #Correlate the pattern1(SYSTEM_CFGSYNC_DOWN) and pattern2(SYSTEM_CFGSYNC_UP). If pattern2 doesn't occur within 10secs, write pattern1 to test1.log type=PairWithWindow ptype=RegExp pattern=(<\S+>)(\s?)(\S+)(\s?)((\S+:\s){2})SYSTEM_CFGSYNC_DOWN:((\s-){2}).*? desc=SYSTEM_CFGSYNC_DOWN event for $3 action=write /root/secTEMP/poctest/$3/test1.log $0; create CT_DOWN_$3; ptype2=RegExp pattern2=(<\S+>)(\s?)$3(\s?)((\S+:\s){2})SYSTEM_CFGSYNC_UP:((\s-){2}).*? desc2=$0 action2=none; window=10 #2 #Check if SYSTEM_CFGSYNC_UP should write to the log type=Single ptype=RegExp pattern=(<\S+>)(\s?)(\S+)(\s?)((\S+:\s){2})SYSTEM_CFGSYNC_UP:((\s-){2}).*? context=CT_DOWN_$3 desc=SYSTEM_CFGSYNC_UP event for $3 action=write /root/secTEMP/poctest/$3/test1.log $0; Note that apart from changes in the 'desc' field, there are two other modifications which are important. The original context CT_DOWN has been replaced with CT_DOWN_$3 which contains the hostname and thus creates a proper connection between PairWithWindow and Single rules. Secondly, the 'pattern2' field of the PairWithWindow rule has been changed to (<\S+>)(\s?)$3(\s?)((\S+:\s){2})SYSTEM_CFGSYNC_UP:((\s-){2}).*? which does not match *any* SYSTEM_CFGSYNC_UP event as before, but rather a SYSTEM_CFGSYNC_UP event that originates from the relevant hostname. Logically, the $3 variable in 'pattern2' plays the same role as in the CT_DOWN_* context, and restricts the matches to specific hostnames only. Also, the previously mentioned "Event Correlation Operations" section of the sec official documentation contains a similar example about Pair* rules, and I'd again recommend to have a look into this section. kind regards, risto 2015-09-21 20:01 GMT+03:00 thin aung <thinza...@gmail.com>: > Hello, > > I would like to seek help with the following Rules I created. > Basically, I am trying to correlate pattern1 and pattern2 and write both > of them to the new log file when the event2 occurred after 10 secs of > event1. > > > When i tested the rule by feeding the sample log files thru keyboard input > and it is working as expected. > > But, the issue happened when i actually monitor log files (I've more than > 1 log to monitor and i used one SEC process to monitor them all - > --input=/logs/g*/system.log) and the rule doesn't work as tested. > > > The sec version i used is 2.7.7 > > > Any suggestion? > > > Thanks > > Thin > > > > #1 > #Correlate the pattern1(SYSTEM_CFGSYNC_DOWN) and pattern2( > SYSTEM_CFGSYNC_UP). If pattern2 doesn't occur within 10secs, write > pattern1 to test1.log > > type=PairWithWindow > > ptype=RegExp > > > pattern=(<\S+>)(\s?)(\S+)(\s?)((\S+:\s){2})SYSTEM_CFGSYNC_DOWN:((\s-){2}).*? > > desc=$0 > > action=write /root/secTEMP/poctest/$3/test1.log $0; create CT_DOWN; > > ptype2=RegExp > > pattern2=(<\S+>)(\s?)(\S+)(\s?)((\S+:\s){2})SYSTEM_CFGSYNC_UP:((\s-){2}).*? > > desc2=$0 > > action2=none; > > window=10 > > > > #2 > > #Check if SYSTEM_CFGSYNC_UP should write to the log > > type=Single > > ptype=RegExp > > pattern=(<\S+>)(\s?)(\S+)(\s?)((\S+:\s){2})SYSTEM_CFGSYNC_UP:((\s-){2}).*? > > context=CT_DOWN > > desc=$0 > > action=write /root/secTEMP/poctest/$3/test1.log $0; > > > > Sample Log: > > ============ > > 2015-09-10T13:02:59+01:00 <local4.warning> gbsole001a event: SYSTEM: > SYSTEM_CFGSYNC_DOWN: - - The config-sync feature is operationally down. > > 2015-09-10T13:03:02+01:00 <local4.info> gbsole001a event: SYSTEM: > SYSTEM_CFGSYNC_UP: - - The config-sync feature is operationally up. > > > > 2015-09-10T13:03:02+01:00 <local4.warning> gbsole001a event: SYSTEM: > SYSTEM_CFGSYNC_DOWN: - - The config-sync feature is operationally down. > > 2015-09-10T13:03:03+01:00 <local4.info> gbsole001a event: SYSTEM: > SYSTEM_CFGSYNC_UP: - - The config-sync feature is operationally up. > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Simple-evcorr-users mailing list > Simple-evcorr-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users > >
------------------------------------------------------------------------------
_______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
