Georgi, it is good to hear that you got things working! Just a curious question -- why was the env-based approach not working, did you get a syntax error from shell? If so, enclosing match variables between apostrophes might help. Also, thanks for offering the beer, I'll keep that in mind for the case we run into each other at some conference :) kind regards, risto
2014-01-31 George Lakovski <[email protected]>: > Hi Risto, > You have a beer from me. Just say where to send it :) > > I followed your hints and managed to achieve my ideas. There are slight > difference from your proposal. > 1) I sent you the wrong log. Sorry for that. The messages were from the > device, not from the syslog itself. There, the device name is also present. > That why I still need the (\w+-\w+-\w+\d) pattern. > 2) the "env" commands weren't accepted and I used my option with "eval" > 3) I had to add one more single type rule to much cases of changes to the > devices, which have no modification of the descriptions of the interfaces. > You can see it below. > 4) The changes to the RANCID config file were as described in my first post. > > I hope that this can help other people monitor their devices. > > type=Pair > ptype=RegExp > pattern=\S (\w+-\w+-\w+\d) .*%PARSER-5-CFGLOG_LOGGEDCMD.* User:(\w+) logged > command:description.*By VPNSC: Job Id# = (\d+) .* > > desc=Waiting for possible change in the config of $1 by $2 > action=none > ptype2=RegExp > pattern2=\S $1 .*SYS-5-CONFIG_I.* by $2 on vty\d+ \((\d+.\d+.\d+.\d+)\) > desc2=Please verify. There is a change in the config of %1 by %2 > action2=eval %TEST1 ( $ENV {'EXECUTOR'}="%2" );\ > eval %TEST2 ( $ENV {'JOB_ID'}="%3" );\ > eval %TEST3 ( $ENV {'SOURCE_IP'}="$1" );\ > shellcmd /usr/local/rancid/bin/rancid-run -r %1 > window=300 > > > type=Single > ptype=RegExp > pattern=\S (\w+-\w+-\w+\d) .*SYS-5-CONFIG_I.* by (\w+) on vty\d+ > \((\d+.\d+.\d+.\d+)\) > desc=Please verify. There is a change in the config of $1 by $2 > action=eval %TEST2 ( $ENV {'EXECUTOR'}="$2" );\ > eval %TEST2 ( $ENV {'JOB_ID'}="XXXXX" );\ > > eval %TEST3 ( $ENV {'SOURCE_IP'}="$3" );\ > shellcmd /usr/local/rancid/bin/rancid-run -r $1 > window=300 > > > BR, > Georgi > > > > > > > 2014-01-30 Risto Vaarandi <[email protected]>: > >> Georgi, >> I had a look into the sample log messages and the previous post. Is my >> understanding correct that if the following message appears >> >> 002375: Dec 9 15:13:54.645 BG: %PARSER-5-CFGLOG_LOGGEDCMD: >> User:iscuser logged command:description GigabitEthernet9/1.2013 dot1q >> vlan id=2013. By VPNSC: Job Id# = 20321 >> (3032121_Customer1_Ref_Vienna_Data) >> >> you would like to extract the router name (BG), the user name >> (iscuser) and job id (20321). Then, an event correlation operation for >> this router name and user name should be started which would wait >> during 5 minutes for the %SYS-5-CONFIG_I message where the router name >> and the user name are the same. For the above LOGGEDCMD message, the >> message which the operation is waiting could be the following: >> >> 002397: Dec 9 15:13:55.377 BG: %SYS-5-CONFIG_I: Configured from >> console by iscuser on vty2 (10.192.21.13) >> >> If such message is observed during 5 minutes, rancid-run should be >> executed. For the above two messages, the rancid-run command should be >> the following: >> >> env SOURCE_IP=10.192.21.13 EXECUTOR=iscuser JOB_ID=20321 >> /usr/local/bin/rancid-run -r BG >> >> If my assumptions are correct, your rule could look like this: >> >> type=Pair >> ptype=RegExp >> pattern=(\w+): %PARSER-5-CFGLOG_LOGGEDCMD.* User:(\w+) logged >> command:description.*Job Id# = (\d+) >> desc=Waiting for possible change in the config of $1 by $2 >> action=none >> ptype2=RegExp >> pattern2=$1: %SYS-5-CONFIG_I.* by $2 on vty\d+ \((\d+.\d+.\d+.\d+)\) >> desc2=Please verify. There is a change in the config of %1 by %2 >> action2=shellcmd env SOURCE_IP=$1 EXECUTOR=%2 JOB_ID=%3 >> /usr/local/bin/rancid-run -r %1 >> window=300 >> >> Note that I have removed >> (\w+-\w+-\w+\d) >> from your original regular expression, since this construct does not >> match anything in the given test messages. Since it seems to be >> designed for matching the router name (I assume that's BG in your test >> messages), I have simply used (\w+): for setting $1 to the router >> name. If the host name is encoded in some other way in your actual >> messages, you need to tweak the regular expression accordingly. >> >> You should also note that since you are running rancid-run after *two* >> regular expression matches, the more recent match will shadow the >> match variables from the previous match. Therefore, you need %1, %2 >> and %3 match variables for fetching the variable values set by >> (\w+): %PARSER-5-CFGLOG_LOGGEDCMD.* User:(\w+) logged >> command:description.*Job Id# = (\d+) >> >> For additional information, I would recommend to have a look into this >> section of the documentation: >> http://simple-evcorr.sourceforge.net/man.html#lbAP >> Although it provides an example of PairWithWindow rule, it explains in >> a detailed way how match variables are set if the rule has two regular >> expressions, and the discussion is equally valid for Pair rule. >> >> Finally, instead of setting environment variables in Perl, I have set >> the environment with /bin/env for executed command line only (which >> IMHO is both clearer and cleaner). >> >> I hope my assumptions about the relevant message fields were correct >> ones. If not, the example rule should still be easy to modify. >> >> kind regards, >> risto >> >> >> 2014-01-30 George Lakovski <[email protected]>: >> > Hi, >> > The sample of the log message is below: >> > >> > 02374: Dec 9 15:13:54.569 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:interface GigabitEthernet9/1.2013 >> > 002375: Dec 9 15:13:54.645 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:description GigabitEthernet9/1.2013 dot1q vlan id=2013. >> > By >> > VPNSC: Job Id# = 20321 (3032121_Customer1_Ref_Vienna_Data) >> > 002376: Dec 9 15:13:54.681 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:encapsulation dot1Q 2013 >> > 002377: Dec 9 15:13:54.717 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:ip vrf forwarding Cust-VPN-sfs2 >> > 002378: Dec 9 15:13:54.765 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:ip address 10.10.10.1 255.255.255.252 >> > 002379: Dec 9 15:13:54.801 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:no shutdown >> > 002392: Dec 9 15:13:55.233 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:interface GigabitEthernet9/1.2013 >> > 002393: Dec 9 15:13:55.253 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:bandwidth 2048 >> > 002394: Dec 9 15:13:55.297 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:service-policy input 2M_Voice_in >> > 002395: Dec 9 15:13:55.329 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:service-policy output 2M_Voice_out >> > 002396: Dec 9 15:13:55.361 BG: %PARSER-5-CFGLOG_LOGGEDCMD: User:iscuser >> > logged command:ip access-group ITS_Voice in >> > 002397: Dec 9 15:13:55.377 BG: %SYS-5-CONFIG_I: Configured from console >> > by >> > adminuser on vty2 (10.192.21.13) >> > >> > Some details for the RANCID itself: >> > 1) the poll of the devices is made by the command: >> > >> > /usr/local/rancid/bin/rancid-run -r $device_name >> > >> > 2) Every version of configuration in RANCID is made with a the relevant >> > comment. >> > >> > I modified this comment and added another variable "EXECUTOR". This part >> > is >> > working. Now I need to add additionally the variables >> > >> > "SOURCE_IP" and "JOB_ID". I am using SEC to parse the log and to >> > generate >> > this global variables, which are used by RANCID. The whole point is to >> > >> > be able to parse them with the specific requirement in the initial post >> > and >> > this should be done in the described sequence with the described >> > interval. >> > >> > I was wondering whether the goal can be achieved via CONTEXTs or another >> > SEC >> > technics. >> > >> > BR, >> > Georgi >> > >> > >> > >> > 2014-01-29 Risto Vaarandi <[email protected]> >> > >> >> hi Georgi, >> >> can you post the actual log messages and describe how exactly they >> >> should be handled? Since most people in the list are probably not >> >> familiar with RANCID, it is quite hard to grasp what events should be >> >> processed and how the processing should be done. >> >> regards, >> >> risto >> >> >> >> 2014-01-29 George Lakovski <[email protected]>: >> >> > >> >> > >> >> > Hi, >> >> > I need some help for the work of RANCID with SEC.pl. >> >> > >> >> > I need to achieve the following functionality observing the >> >> > syslog >> >> > of >> >> > Cisco devices. >> >> > >> >> > 1) look for a description of an interface that matches specific >> >> > pattern. >> >> > If >> >> > there is a match to check whether there is a "SYS-5-CONFIG_I" message >> >> > in >> >> > the >> >> > log and initiate a new poll of the specific device reported the >> >> > messages. >> >> > The log message for the description of an interface and write config >> >> > should >> >> > be for the same device and user and in 5 min interval. Otherwise >> >> > nothing >> >> > should be done. >> >> > >> >> > 2) if the above conditions are not met as described to poll the >> >> > device >> >> > only >> >> > by a "SYS-5-CONFIG_I" message >> >> > >> >> > All this is with the idea to match a specific "change ID", coded in >> >> > the >> >> > description and to pass it together with the user, who made the >> >> > change >> >> > to >> >> > RANCID. This data will appear in the logs of RANCID as global >> >> > varaibles >> >> > passed to RANCID in the "bin/control_rancid" file to the >> >> > >> >> > "if [ "X$device" != "X" ] ; then >> >> > $RCSSYS commit -m "$EXECUTOR via $SOURCE_IP due to JOB_ID updated >> >> > $mailrcpt" >> >> > subject="$GROUP/$device $subject" >> >> > else >> >> > $RCSSYS commit -m "$EXECUTOR via $SOURCE_IP due to JOB_ID update" >> >> > subject="$GROUP $subject" >> >> > fi" >> >> > >> >> > section. >> >> > >> >> > I imagine it to something like the following, but I can't make it >> >> > work. >> >> > >> >> > # in case of change to an interface description combined with write >> >> > config >> >> > >> >> > type=Pair >> >> > ptype=RegExp >> >> > pattern=\S (\w+-\w+-\w+\d) .*%PARSER-5-CFGLOG_LOGGEDCMD.* User:(\w+) >> >> > logged >> >> > command:description.*Job Id# = (\d+) .* >> >> > desc=This is the description of the changed interface >> >> > action=eval %TEST1 ( $ENV {'JOB_ID'}="$3" ) >> >> > ptype2=RegExp >> >> > pattern2=\S $1 .*SYS-5-CONFIG_I.* by $2 on vty\d+ >> >> > \((\d+.\d+.\d+.\d+)\) >> >> > desc2=Please verify. There is a change in the config of $1 by $2 >> >> > action2=eval %TEST2 ( $ENV {'EXECUTOR'}="$2" );\ >> >> > eval %TEST3 ( $ENV {'SOURCE_IP'}="$3" );\ >> >> > shellcmd /usr/local/rancid/bin/rancid-run -r $1 >> >> > window=1800 >> >> > >> >> > >> >> > # in case there is no change of description, only changes not related >> >> > to >> >> > interface description >> >> > >> >> > type=Single >> >> > ptype=RegExp >> >> > pattern=\S (\w+-\w+-\w+\d) .*SYS-5-CONFIG_I.* by (\w+) on vty\d+ >> >> > \((\d+.\d+.\d+.\d+)\) >> >> > desc=Please verify. There is a change in the config of $1 by $2 >> >> > action=eval %TEST2 ( $ENV {'EXECUTOR'}="$2" );\ >> >> > eval %TEST3 ( $ENV {'SOURCE_IP'}="$3" );\ >> >> > shellcmd /usr/local/rancid/bin/rancid-run -r $1 >> >> > window=1800 >> >> > >> >> > Probably the first part may be done with CONTEXTs.... >> >> > Any ideas? I will appreciate any help how to accomplish my ideas. >> >> > >> >> > BR, >> >> > Georgi >> >> > >> >> > >> >> > >> >> > >> >> > ------------------------------------------------------------------------------ >> >> > WatchGuard Dimension instantly turns raw network data into actionable >> >> > security intelligence. It gives you real-time visual feedback on key >> >> > security issues and trends. Skip the complicated setup - simply >> >> > import >> >> > a virtual appliance and go from zero to informed in seconds. >> >> > >> >> > >> >> > http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk >> >> > _______________________________________________ >> >> > Simple-evcorr-users mailing list >> >> > [email protected] >> >> > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users >> >> > >> > >> > > > ------------------------------------------------------------------------------ WatchGuard Dimension instantly turns raw network data into actionable security intelligence. It gives you real-time visual feedback on key security issues and trends. Skip the complicated setup - simply import a virtual appliance and go from zero to informed in seconds. http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
