On Tue, 25 Aug 2015, Ganji, Shashirekha Yadav wrote: > David, > Excuse my ignorance.I just checked and it appears that there was delay of > 4hrs for few other events yesterday evening. > But currently events are coming out well. > > SO what do u suggest here?
Ok, this makes more sense :-) https://www.usenix.org/publications/login/december-2013-volume-38-number-6/using-sec it talks a bit about performace tuning but the bottom line is that you will want to split your rules up into ones that look at different types of logs, then configure syslog to split up your logs to match and run multiple instances of sec. and keep an eye on the cpu utilization to catch if it's maxing out. David Lang > Thanks, > shashi > > -----Original Message----- > From: Ganji, Shashirekha Yadav > Sent: Monday, August 24, 2015 5:11 PM > To: 'David Lang' > Cc: [email protected] > Subject: RE: [Simple-evcorr-users] SEC multiple events match same time > > David, > > SEC is perfectly fine is processing other alerts with out any delay.The BGP > alert is an exceptional case we have seen so far from past 2 years which was > alerted with some delay. > > Just wondering is anything to do with multiple events matching the same at > eactly same time? > > If it was ready behind so far,then the issue should have been seen in next > events right.But rest all other events before 8 :00pm or after 8:00pm were > perfectly fine without any delays. > > Thanks, > shashi > > -----Original Message----- > From: David Lang [mailto:[email protected]] > Sent: Monday, August 24, 2015 5:07 PM > To: Ganji, Shashirekha Yadav > Cc: [email protected] > Subject: Re: [Simple-evcorr-users] SEC multiple events match same time > > how are the logs getting from syslog to SEC? Is SEC just reading the files > that syslog writes? is syslog writing to stdin on SEC? other? > > I would guess that you have syslog writing to file(s) and sec reading those > files. In this case, it was probably that sec was just that far behind in > processing logs. > > David Lang > > On Tue, 25 Aug 2015, Ganji, Shashirekha Yadav wrote: > >> David, >> >> We are forwarding all devices logs to syslog server and using >> different facilities based on the technologies. >> >> I see actual device logs coming around 8:00pm on our syslog local >> files but SEC alerted them @00:00hrs with a delay of 4hrs.I see few >> other events alerted by SEC in this window(8:00pm-00:00hrs) with no isues. >> >> The 2 alerts that were delayed were from 2 connected devices whose >> both sides BGP neighbors were down and hence both the syslog messages >> were matching the same rule same at exactly same time. >> Thanks, shashi >> >> >> >> -----Original Message----- >> From: David Lang [mailto:[email protected]] >> Sent: Monday, August 24, 2015 4:34 PM >> To: Ganji, Shashirekha Yadav >> Cc: [email protected] >> Subject: Re: [Simple-evcorr-users] SEC multiple events match same time >> >> On Mon, 24 Aug 2015, Ganji, Shashirekha Yadav wrote: >> >>> Hi, >>> >>> I'am using SEC in our infrastructure past 2 years and our customers are >>> extremely happy with the tool.It was all good so far but yesterday >>> experienced a peculiar issue. >>> >>> We have SEC rule setup as below: >>> >>> ## Rule:2 >>> ## Last Updated At: 2015-03-19T17:39:21.297Z ## Rule:1 Vendor:Cisco >>> BGP neighbor down alarm, alarm will be suppressed if neighbor recovers >>> within 60 seconds. In case of 5 such events witin 5 min a %BGP-5-FLAP: >>> notification will be generated. >>> type=pairWithWindow >>> ptype=regexp >>> continue=dontcont >>> pattern=Date=.* ,Device=(\S+) ,Msg=.*((%BGP-5-ADJCHANGE:).* >>> (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) Down.*) >>> desc=$1 $3 $4 >>> action=shellcmd perl /etc/syslog-config/send2mom/sec_s2m_v2.pl >>> --targetparent $1 --target $4 --notifying_group NETRS --severity >>> MAJOR --kpi Network --pattern "$3" --log "$2" --source SEC >>> --sendevent on ptype2=regexp >>> pattern2=Date=.* ,Device=($1) ,Msg=.*(($3).* ($4) Up.*) >>> desc2=$1 BGP Neighbor $4 flap detected action2=event %s; shellcmd >>> echo `date` "Source=SEC, KpiName=Network, Severity=-, >>> Action=Suppress, Device=$1, Pattern=$3, Notify Group=-, Log $0" >> >>> /local/mnt/workspace/logs/sec-logs/sec-messages.log >>> window=60 >>> >>> >>> I noticed there were 2 events matching the above pattern.Device A and >>> Device B connected to each other and both the devices BGP nei >>> connecting to each other was down. >>> >>> Problem was SEC alerted the above alerts with a delay of 4 hrs,Can >>> you explain why is this delay and how can i fix the issue. >> >> SEC doesn't delay sending any alerts, so the question is did it take that >> log to get the log to SEC, or was SEC that far behind in processing messages? >> >> if you enable a dumpfile, you can send SEC a signal and then look in >> the resulting file to see the most recent logs it's processed. That >> will tell you if it's way behind (although sec using 100% cpu for any >> significant amountof time will tell you is is not keeping up) >> >> how are you reading the logs? >> >> David Lang >> > > ------------------------------------------------------------------------------ > _______________________________________________ > Simple-evcorr-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users > ------------------------------------------------------------------------------ _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
