nterface Vlan217 description CUSTOMER A ip address x.x.x.x.x ip access-group 178 in no ip redirects no ip unreachables no ip proxy-arp ip multicast ttl-threshold 3
shcpu CPU utilization for five seconds: 92%/51%; one minute: 92%; five minutes: 92% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 9 14412 39169 367 0.95% 0.19% 0.08% 0 ARP Input 51 155152 901076 172 2.55% 0.92% 0.93% 0 Fifo Error Detec 67 12541 522329 24 0.15% 0.07% 0.05% 0 HLFM address ret 115 622003 413812 1503 7.34% 7.52% 7.49% 0 Hulc LED Process 136 166229 17815 9330 0.63% 0.60% 0.60% 0 PI MATM Aging Pr 168 5892258 12519191 470 25.23% 23.54% 24.45% 0 IP Input 171 32572 45322 718 0.15% 0.13% 0.12% 0 Spanning Tree thanks for input 2009/4/24 Lee <ler...@gmail.com> > > These TTL=1 are causing the high CPU. > > Just out of curiousity, would adding "ip multicast ttl-threshold 3" > and/or "no ip unreachable" on the interface reduce cpu usage? > > Lee > > > On 4/24/09, Richard Gallagher <rgall...@cisco.com> wrote: > > Input queue was full of packets like this: > > > > Buffer information for RxQ3 buffer at 0x2E792F0 > > data_area 0x7BB2AB0, refcount 1, next 0x2E7E210, flags 0x200 > > linktype 7 (IP), enctype 1 (ARPA), encsize 14, rxtype 1 > > if_input 0x3ABBAE0 (Vlan217), if_output 0x0 (None) > > inputtime 00:00:00.000 (elapsed never) > > outputtime 00:00:00.000 (elapsed never), oqnumber 65535 > > datagramstart 0x7BB2AF6, datagramsize 82, maximum size 2196 > > mac_start 0x7BB2AF6, addr_start 0x7BB2AF6, info_start 0x0 > > network_start 0x7BB2B04, transport_start 0x7BB2B18, caller_pc > > 0x6D1024 > > > > source: 74.212.165.187, destination: 224.0.0.252, id: 0x3CDA, ttl: 1, > > TOS: 0 prot: 17, source port 58064, destination port 5355 > > > > Buffer information for RxQFB buffer at 0x2672BB0 > > data_area 0x758C35C, refcount 1, next 0x263960C, flags 0x200 > > linktype 7 (IP), enctype 1 (ARPA), encsize 14, rxtype 1 > > if_input 0x3ABBAE0 (Vlan217), if_output 0x0 (None) > > inputtime 00:00:00.000 (elapsed never) > > outputtime 00:00:00.000 (elapsed never), oqnumber 65535 > > datagramstart 0x758C3A2, datagramsize 64, maximum size 2196 > > mac_start 0x758C3A2, addr_start 0x758C3A2, info_start 0x0 > > network_start 0x758C3B0, transport_start 0x0, caller_pc 0x6D1024 > > > > source: 74.212.165.187, destination: 224.0.0.252, id: 0x3CDA, ttl: 1, > > TOS: 0 prot: 17, source port 58064, destination port 5355 > > > > These TTL=1 are causing the high CPU. > > > > > > On 24 Apr 2009, at 14:26, Chris Lane wrote: > > > >> Richard Gallagher found that it was one of my customers sending mcast > >> packets with a TTL 1. Tried adding ACL's to lower CPU but this > >> didn't fix. > >> We shutdown Vlan to verify and CPU came down 40% to adequate levels. > >> > >> I have a call into out customer notifying them to fix. > >> > >> Thanks to all for your input > >> > >> Regards > >> Chris > >> > >> 2009/4/24 Chris Lane <clane1...@gmail.com> > >> > >>> Yes with a high preference. > >>> > >>> 2009/4/24 junior <drr...@ya.ru> > >>> > >>> Hello. > >>>> > >>>> Does this switch have default route? > >>>> > >>>> Chris Lane wrote: > >>>> > >>>>> sh ip traffic IP statistics: > >>>>> Rcvd: 37788273 total, 24253 local destination > >>>>> 0 format errors, 0 checksum errors, 9771492 bad hop count > >>>>> 0 unknown protocol, 27979860 not a gateway > >>>>> 0 security failures, 0 bad options, 7762670 with options > >>>>> Opts: 0 end, 0 nop, 0 basic security, 0 loose source route > >>>>> 0 timestamp, 0 extended security, 0 record route > >>>>> 0 stream ID, 0 strict source route, 7762670 alert, 0 > >>>>> cipso, 0 ump > >>>>> 0 other > >>>>> Frags: 0 reassembled, 0 timeouts, 0 couldn't reassemble > >>>>> 0 fragmented, 0 couldn't fragment > >>>>> Bcast: 2884 received, 87 sent > >>>>> Mcast: 2334 received, 2209 sent > >>>>> Sent: 24621 generated, 8328118 forwarded > >>>>> Drop: 4258 encapsulation failed, 0 unresolved, 83 no adjacency > >>>>> 69 no route, 0 unicast RPF, 0 forced drop > >>>>> 0 options denied, 0 source IP address zero > >>>>> > >>>>> ICMP statistics: > >>>>> Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 0 > >>>>> unreachable > >>>>> 9560 echo, 0 echo reply, 0 mask requests, 0 mask replies, 0 > >>>>> quench > >>>>> 0 parameter, 0 timestamp, 0 info request, 0 other > >>>>> 0 irdp solicitations, 0 irdp advertisements > >>>>> Sent: 0 redirects, 3129 unreachable, 0 echo, 9560 echo reply > >>>>> 0 mask requests, 0 mask replies, 0 quench, 0 timestamp > >>>>> 0 info reply, 47 time exceeded, 0 parameter problem > >>>>> 0 irdp solicitations, 0 irdp advertisements > >>>>> > >>>>> TCP statistics: > >>>>> Rcvd: 7710 total, 8 checksum errors, 1 no port > >>>>> Sent: 6762 total > >>>>> > >>>>> UDP statistics: > >>>>> Rcvd: 4615 total, 0 checksum errors, 1430 no port > >>>>> Sent: 2909 total, 0 forwarded broadcasts > >>>>> > >>>>> IP-EIGRP statistics: > >>>>> Rcvd: 0 total > >>>>> Sent: 0 total > >>>>> > >>>>> BGP statistics: > >>>>> Rcvd: 162 total, 1 opens, 0 notifications, 1 updates > >>>>> 160 keepalives, 0 route-refresh, 0 unrecognized > >>>>> Sent: 159 total, 1 opens, 0 notifications, 0 updates > >>>>> 158 keepalives, 0 route-refresh > >>>>> > >>>>> PIMv2 statistics: Sent/Received > >>>>> Total: 0/0, 0 checksum errors, 0 format errors > >>>>> Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0, > >>>>> Hellos: > >>>>> 0/0 > >>>>> Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0 > >>>>> Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0 > >>>>> State-Refresh: 0/0 > >>>>> > >>>>> IGMP statistics: Sent/Received > >>>>> Total: 0/0, Format errors: 0/0, Checksum errors: 0/0 > >>>>> Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0 DVMRP: > >>>>> 0/0, PIM: > >>>>> 0/0 > >>>>> > >>>>> OSPF statistics: > >>>>> Rcvd: 2363 total, 0 checksum errors > >>>>> 1900 hello, 12 database desc, 2 link state req > >>>>> 345 link state updates, 104 link state acks > >>>>> > >>>>> Sent: 2231 total > >>>>> 1904 hello, 11 database desc, 4 link state req > >>>>> 223 link state updates, 89 link state acks > >>>>> > >>>>> ARP statistics: > >>>>> Rcvd: 2254 requests, 82 replies, 0 reverse, 0 other > >>>>> Sent: 4178 requests, 2447 replies (2 proxy), 0 reverse > >>>>> Drop due to input queue full: 0 > >>>>> > >>>>> Thanks for looking. > >>>>> > >>>>> On Fri, Apr 24, 2009 at 7:45 AM, junior <drr...@ya.ru <mailto: > >>>>> drr...@ya.ru>> wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> Did You check TAC cases? > >>>>> Can You post this switch current configuration with sh ip traffic > >>>>> command output? > >>>>> > >>>>> WBR > >>>>> Roman A. Nozdrin > >>>>> > >>>>> Chris Lane wrote: > >>>>> > >>>>> 1 routed interface.sh platform ip unicast failed route > >>>>> Total of 0 covering fib entries > >>>>> > >>>>> Thanks for reply.. I checked earlier regarding sdm. > >>>>> Its the same on all of my 3750's i have about 20 of them > >>>>> throughout the > >>>>> states, this is probably the quietest one in regards to > >>>>> bandwidth and > >>>>> services. > >>>>> > >>>>> > >>>>> > >>>>> On Fri, Apr 24, 2009 at 7:21 AM, Brian Turnbow < > b.turn...@twt.it > >>>>> <mailto:b.turn...@twt.it>> wrote: > >>>>> > >>>>> how many routed interfaces do you have ( sh ip int brief > >>>>> with ip > >>>>> addresses ) ? > >>>>> if more than 8 change the sdm template to routing > >>>>> > >>>>> you can use sh platform ip unicast failed route to see > >>>>> if > >>>>> routes are > >>>>> failing to be programmed into tcam > >>>>> > >>>>> Brian > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> ------------------------------ > >>>>> *From:* Chris Lane [mailto:clane1...@gmail.com > >>>>> <mailto:clane1...@gmail.com>] > >>>>> *Sent:* venerdě 24 aprile 2009 11.17 > >>>>> > >>>>> *To:* Brian Turnbow > >>>>> *Cc:* Peter Rathlev; cisco-nsp@puck.nether.net > >>>>> <mailto:cisco-nsp@puck.nether.net> > >>>>> > >>>>> > >>>>> *Subject:* Re: [c-nsp] 3750 High Cpu IP Input > >>>>> > >>>>> sh controllers cpu-interface > >>>>> ASIC Rxbiterr Rxunder Fwdctfix Txbuflos > >>>>> Rxbufloc > >>>>> Rxbufdrain > >>>>> > >>>>> > ------------------------------------------------------------------------- > >>>>> ASIC0 0 0 0 0 0 > >>>>> 0 > >>>>> ASIC1 0 0 0 0 0 > >>>>> 0 > >>>>> > >>>>> > >>>>> cpu-queue-frames retrieved dropped invalid hol- > >>>>> block > >>>>> stray > >>>>> ----------------- ---------- ---------- ---------- > >>>>> ---------- ---------- > >>>>> rpc 0 0 0 0 > >>>>> 0 > >>>>> stp 1807 0 0 0 > >>>>> 0 > >>>>> ipc 0 0 0 0 > >>>>> 0 > >>>>> routing protocol 1516326 0 0 0 > >>>>> 0 > >>>>> L2 protocol 27 0 0 0 > >>>>> 0 > >>>>> remote console 0 0 0 0 > >>>>> 0 > >>>>> sw forwarding 915 0 0 0 > >>>>> 0 > >>>>> host 2014 0 0 0 > >>>>> 0 > >>>>> broadcast 1766 0 0 0 > >>>>> 0 > >>>>> cbt-to-spt 0 0 0 0 > >>>>> 0 > >>>>> igmp snooping 1518651 0 0 0 > >>>>> 0 > >>>>> icmp 45 0 0 0 > >>>>> 0 > >>>>> logging 0 0 0 0 > >>>>> 0 > >>>>> rpf-fail 0 0 0 0 > >>>>> 0 > >>>>> queue14 0 0 0 0 > >>>>> 0 > >>>>> cpu heartbeat 14116 0 0 0 > >>>>> 0 > >>>>> > >>>>> ODD i have disabled IGMP SNOOPING... > >>>>> > >>>>> On Fri, Apr 24, 2009 at 4:19 AM, Brian Turnbow > >>>>> <b.turn...@twt.it <mailto:b.turn...@twt.it>> wrote: > >>>>> > >>>>> You can use show controller cpu to help see whats > >>>>> going to the cpu > >>>>> Make sure you have no ip redirects and no proxy arp > >>>>> on > >>>>> all the interfaces. > >>>>> How many routed interfaces do you have ? > >>>>> The output below for "max" is for 8 routed > >>>>> interfaces if > >>>>> you have more you > >>>>> should change to the desktop switching template. > >>>>> With your roughly your values for indirectly > >>>>> connected > >>>>> routes and 13 ip > >>>>> interfaces on a box I needed to switch the template > >>>>> "sdm > >>>>> prefer routing" > >>>>> requies reload. > >>>>> > >>>>> Regards > >>>>> > >>>>> Brian > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -----Original Message----- > >>>>> From: cisco-nsp-boun...@puck.nether.net > >>>>> <mailto:cisco-nsp-boun...@puck.nether.net> [mailto: > >>>>> cisco-nsp-boun...@puck.nether.net > >>>>> <mailto:cisco-nsp-boun...@puck.nether.net>] On > >>>>> Behalf Of > >>>>> Chris Lane > >>>>> Sent: venerdě 24 aprile 2009 1.09 > >>>>> To: Peter Rathlev > >>>>> Cc: cisco-nsp@puck.nether.net > >>>>> <mailto:cisco-nsp@puck.nether.net> > >>>>> Subject: Re: [c-nsp] 3750 High Cpu IP Input > >>>>> > >>>>> sh platform tcam utilization > >>>>> > >>>>> CAM Utilization for ASIC# 0 Max > >>>>> Used > >>>>> Masks/ > >>>>> Values > >>>>> Masks/values > >>>>> > >>>>> Unicast mac addresses: > >>>>> 784/6272 > >>>>> 37/235 > >>>>> IPv4 IGMP groups + multicast routes: > >>>>> 144/1152 > >>>>> 6/26 > >>>>> IPv4 unicast directly-connected routes: > >>>>> 784/6272 > >>>>> 37/235 > >>>>> IPv4 unicast indirectly-connected routes: > >>>>> 272/2176 > >>>>> 52/326 > >>>>> IPv4 policy based routing aces: 0/0 > >>>>> 0/0 > >>>>> IPv4 qos aces: > >>>>> 528/528 > >>>>> 18/18 > >>>>> IPv4 security aces: > >>>>> 1024/1024 > >>>>> 57/57 > >>>>> > >>>>> Note: Allocation of TCAM entries per feature uses > >>>>> a complex algorithm. The above information is meant > >>>>> to provide an abstract view of the current TCAM > >>>>> utilization > >>>>> > >>>>> Hope this helps. > >>>>> > >>>>> On Thu, Apr 23, 2009 at 4:41 PM, Peter Rathlev > >>>>> <pe...@rathlev.dk <mailto:pe...@rathlev.dk>> wrote: > >>>>> > >>>>> On Thu, 2009-04-23 at 16:15 -0400, Chris Lane > >>>>> wrote: > >>>>> > >>>>> This box has been in production for over a > >>>>> year > >>>>> and doesn't really do > >>>>> to much as you can see from my orig thread it > >>>>> moves about 11MB. > >>>>> > >>>>> This just started late last night yet we > >>>>> didn't > >>>>> add any new customer > >>>>> nor did anybody even touch switch as the > >>>>> device > >>>>> is remote. > >>>>> > >>>>> I read in an older thread regarding same > >>>>> thing > >>>>> that the person > >>>>> rebooted and of course it resolved issue. I > >>>>> am > >>>>> planning to do that > >>>>> Early tomorrow am, but > >>>>> i really want to know what the heck is > >>>>> causing > >>>>> this. > >>>>> > >>>>> Yes CEF is running. > >>>>> > >>>>> What about TCAM utilisation ("show platform tcam > >>>>> utilization")? > >>>>> > >>>>> Regards, > >>>>> Peter > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> //CL > >>>>> _______________________________________________ > >>>>> cisco-nsp mailing list cisco-nsp@puck.nether.net > >>>>> <mailto:cisco-nsp@puck.nether.net> > >>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp > >>>>> archive at http://puck.nether.net/pipermail/cisco- > >>>>> nsp/ > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> //CL > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> //CL > >>>>> > >>>> > >>>> > >>> > >>> > >>> -- > >>> //CL > >>> > >> > >> > >> > >> -- > >> //CL > >> _______________________________________________ > >> cisco-nsp mailing list cisco-nsp@puck.nether.net > >> https://puck.nether.net/mailman/listinfo/cisco-nsp > >> archive at http://puck.nether.net/pipermail/cisco-nsp/ > > > > _______________________________________________ > > cisco-nsp mailing list cisco-nsp@puck.nether.net > > https://puck.nether.net/mailman/listinfo/cisco-nsp > > archive at http://puck.nether.net/pipermail/cisco-nsp/ > > > -- //CL _______________________________________________ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/