Re: [c-nsp] 4x10G Etherchannel overruns

[Jean-Francois . Dube]

Hi Peter,

Do the overrun match with input drops on the interfaces input queue? Maybe 
this traffic being punted to CPU?

I had this issue on a Cisco 7600 when using "mpls ldp explicit-null" and 
many Gbps of traffic needed to be sent to CPU.

Hope this helps.

Cheers,

JF

Jean-François Dubé
Technicien, Opérations Réseau IP
Ingénierie Exploitation des Réseaux
Vidéotron 

"cisco-nsp"  a écrit sur 2017-03-03 
12:04:28 :

> De : "Peter Kranz" 
> A : , 
> Date : 2017-03-03 12:09
> Objet : [c-nsp] 4x10G Etherchannel overruns
> Envoyé par : "cisco-nsp" 
> 
> On a WS-X6908-10G DCEF2T line card with SUP2T's, I ran into overruns
> yesterday on a 4x10G etherchannel that I am at a loss to resolve:
> 
> 
> 
> Constantly increasing overrun counter:
> 
>6418130558941 packets input, 9277559958229871 bytes, 0 no buffer
> 
>  Received 668274 broadcasts (0 IP multicasts)
> 
>  0 runts, 190 giants, 0 throttles
> 
>  192 input errors, 1 CRC, 0 frame, 51591389 overrun, 0 ignored
> 
> 
> 
> Latency into the router rose by 40ms when these overrun's started to 
appear
> 
> 
> 
> This happened at a BW of ~28 Gbps 
> 
> 
> 
> I've built the etherchannel in this manner:
> 
> 
> 
> Index   Load  Port  EC state   No of bits
> 
> --+--++--+---
> 
> 0  0ATe1/1 Active   2
> 
> 3  81Te1/2 Active   2
> 
> 1  60Te1/3 Active   2
> 
> 2  14Te1/4 Active   2
> 
> 
> 
> Is it necessary to instead stagger 1/1, 1/3, 1/5, 1/7 to spread the load
> across the card ASICs? I didn't think the WS-X6908 was an oversubscribed
> card so didn't bother initially.
> 
> 
> 
> Peter Kranz
> www.UnwiredLtd.com  
> Desk: 510-868-1614 x100
> Mobile: 510-207-
> pkr...@unwiredltd.com  
> 
> 
> 
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Cisco CMTS and ISC DHCP Strange Issue

[Jean-Francois . Dube]

Hi Rupesh,

I suggest you first make sure DHCP works properly for the affected CPE.

debug cable mac H.H.H
debug cable dhcp

These should allow you to see DHCP D.O.R.A. for a CPE with MAC H.H.H

The CMTS needs to use leasequery (RFC 4388) if you experienced an outage of
some sort to rebuild its internal database.

If leasequery option 13 is not working properly with ISC servers then you
may have this problem.

You could try removing cable source verify as a test.


Hope this helps.

Cheers,

JF

Jean-François Dubé
Technicien, Opérations Réseau IP
Ingénierie Exploitation des Réseaux
Vidéotron
cisco-nsp cisco-nsp-boun...@puck.nether.net a écrit sur 2014-05-24
09:17:37 :

 De : Rupesh Basnet brup...@subisu.net.np
 A : cisco-nsp@puck.nether.net,
 Date : 2014-05-24 09:24
 Objet : [c-nsp] Cisco CMTS and ISC DHCP Strange Issue
 Envoyé par : cisco-nsp cisco-nsp-boun...@puck.nether.net

 Hi everyone,

 We have been using Cisco CMTS 7246 for about a decades. These days we
 are facing some strange issue with CPE IP assigned by DHCP. We are using
 ISC DHCP for both the modems and CPE. While I check modem searching for
 the particular modem only then CPE taken is seen 0 but when I check cpe
 with modem IP then I can see that customer CPE is taking IP address.
 While checking on CMTS, the arp taken by CPE arp is also seen but no any
 traffic are passing. If I bind the CPE mac and assign a IP then
 everything gets to normal.

 CMTS is configure to relay on two DHCP services running on fail over
 mode and soure verification is done for all the addresses.

 cmts2#show cable modem | in 814c
 00e0.6f70.814c 10.28.114.242   C3/1/U0   online   157   0.00 1942 *0
 *  N

 cmts2#show cable modem 10.28.114.242 cpe
 IP addressMAC address
 *10.34.16.12  9061.0c15.2535 *


 --
 Regards,
 
 Rupesh Basnet

 ___
 cisco-nsp mailing list  cisco-nsp@puck.nether.net
 https://puck.nether.net/mailman/listinfo/cisco-nsp
 archive at http://puck.nether.net/pipermail/cisco-nsp/


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] PIM and network redundancy

[Jean-Francois . Dube]

Hi Rob,

Did you mean to say you have IGMP static-groups on the 7600_3 to attract
multicast traffic toward your distribution switches?

If you meant 7600_1 then I'm not sure what your topology is but the
redundancy issue made me think of a similar issue I faced in the past.


Are you using any static routes (or mroutes) on 7600_3 to the multicast
source?

I think your multicast trafic must have failed RPF check when the link
between 7600_1 and 7600_2 failed.

Using show ip mroute count you can maybe check your Other counts
counters to confirm this. The RPF failed counter should be high.


WS-C6506#show ip mroute 232.32.1.1 10.154.128.58 count
IP Multicast Statistics
785 routes using 396312 bytes of memory
362 groups, 1.16 average sources per group
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kilobits per
second
Other counts: Total/RPF failed/Other drops(OIF-null, rate-limit etc)

Group: 232.32.1.1, Source count: 1, Packets forwarded: 630315119, Packets
received: 630659870
  Source: 10.154.128.58/32, Forwarding: 630315119/393/1344/4230, Other:
630659870/0/344751


PIM is protocol independent and depends on CEF/RIB so if OSPF converges
then PIM should converge as well.

Hope this helps.

Cheers,

JF

Jean-François Dubé
Technicien, Opérations Réseau IP
Ingénierie Exploitation des Réseaux
Vidéotron

cisco-nsp cisco-nsp-boun...@puck.nether.net a écrit sur 2014-02-03
17:39:44 :

 De : Robert Hass robh...@gmail.com
 A : cisco-nsp@puck.nether.net cisco-nsp@puck.nether.net,
 Date : 2014-02-03 17:47
 Objet : [c-nsp] PIM and network redundancy
 Envoyé par : cisco-nsp cisco-nsp-boun...@puck.nether.net

 Hi
 I have project where network looks like this:

 IPTV source
 |
 7600_1
 |  |
 |  |
 7600_2|
 |  |
 |  |
 7600_3-
 |
 IPTV distribution switches (~20 VLANs)

 I'm currently using PIM static joins on first 7600 next to the source and
 classic PIM spare-mode (ip pim spare-mode) sessions between rest 7600 and
 'pim passive' from 7600 to IPTV distribution switches. First 7600 is my
RP.

 My question is how I can provide redundancy in this network. I had issue,
 when link between 7600_1 and 7600_2 was down, and traffic switched to
link
 7600_1--7600_3. IPv4 works without problem as OSPF use backup path.

 But how deal with PIM and multicasts ? Intergrate MSDP or other
protocol ?
 Any recommendations ? I think about creating xconnect (L2VPN over MPLS)
 from first 7600_1 to 7600_3 and put additional Cat6500 on the end as PIM
 box.

 Rob
 ___
 cisco-nsp mailing list  cisco-nsp@puck.nether.net
 https://puck.nether.net/mailman/listinfo/cisco-nsp
 archive at http://puck.nether.net/pipermail/cisco-nsp/


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Quick question on HSRP...

[Jean-Francois . Dube]

Hi Jeff,

My understanding is that you are basically going to replace the default
gateway for in a couple of vlans. (Same IP but different MAC.)

Active HSRP router will issue gratuitous ARP (gARP) when it becomes Active
so there should be little disruption for the hosts inside the vlan trying
to reach the default gateway.

If hosts ignore gARP then they will try to forward frames to the wrong
destination MAC for as long as their ARP entry is valid.

Servers usually have ARP timeout of a few seconds to a few minutes where
routers can have timeout of 4 hours (like Cisco) or more.

HSRP is one thing. You still need to make sure the new routers know how to
route packets the same way the old ones did.

Good luck,

JF

Jean-François Dubé
Technicien, Opérations Réseau IP
Ingénierie Exploitation des Réseaux
Vidéotron

cisco-nsp cisco-nsp-boun...@puck.nether.net a écrit sur 2013-12-30
18:27:12 :

 De : Jeff Kell jeff-k...@utc.edu
 A : cisco-nsp@puck.nether.net cisco-nsp@puck.nether.net,
 Date : 2013-12-30 18:30
 Objet : [c-nsp] Quick question on HSRP...
 Envoyé par : cisco-nsp cisco-nsp-boun...@puck.nether.net

 Quick question for someone that's been there, done that, as I'm a bit
 rushed to try to lab test this...

 We're adding some new routers (4500Xs) for an upgraded server farm
 arrangement with a number of server-side vlans / VRFs.  The plan was to
 trunk it with the existing L3 router, and fire up HSRP (v2) across them
 to transition the L3 routing to the new router without being too
 terribly disruptive.  Not sure if we want to leave the HSRP in place
 (thinking yes) or remove it (and the old router) after the migration,
 but will cross that bridge when we get there.

 HSRP would place the current default gateway as the virtual IP, and I
 presume it will pick up a new MAC address.  I'm concerned this will
 affect the active hosts with the ARP cached for their gateway.  The MAC
 address would still be valid (should match the original gateway) but the
 traffic would be directed to the original (now virtual) IP, as opposed
 to the new physical gateway on the router.

 So just how disruptive will introducing HSRP really be?

 Jeff

 ___
 cisco-nsp mailing list  cisco-nsp@puck.nether.net
 https://puck.nether.net/mailman/listinfo/cisco-nsp
 archive at http://puck.nether.net/pipermail/cisco-nsp/


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] %PLATFORM-CIH-5-ASIC_ERROR_SCRUB_THRESH

[Jean-Francois . Dube]

Hi Antonio,

I had a similar issue and decided to reload the linecard.

pse_pogo_driver[281]: %PLATFORM-CIH-5-ASIC_ERROR_SPECIAL_HANDLE : pse[1]: A
sbe error has occurred causing  data corrected. 0x12470007


I don't like to see any messages regarding single bit error (SBE) and even
less when it's in packet switching engine (PSE) ASIC so that's why I
reloaded the linecard. The messages went away.

I used show asic-errors all detail location 0/x/CPU0 to see the errors on
the linecard.


Cheers,

JF

Jean-François Dubé
Technicien, Opérations Réseau IP
Ingénierie Exploitation des Réseaux
Vidéotron

cisco-nsp cisco-nsp-boun...@puck.nether.net a écrit sur 2013-10-28
11:52:02 :

 De : Antonio Soares amsoa...@netcabo.pt
 A : cisco-nsp@puck.nether.net,
 Date : 2013-10-28 11:54
 Objet : [c-nsp] %PLATFORM-CIH-5-ASIC_ERROR_SCRUB_THRESH
 Envoyé par : cisco-nsp cisco-nsp-boun...@puck.nether.net

 Hello Team,

 I'm getting this message:

 
 pse_pogo_driver[244]: %PLATFORM-CIH-5-ASIC_ERROR_SCRUB_THRESH : pse[1]: A
 sbe error has occurred causing data corrected. 0x12470009 Threshold has
been
 exceeded
 

 Exactly every 14 minutes and 10 seconds. I found bug CSCts11174 and they
 say:

 
 Workaround:
 Sometimes an LC reload can fix the issue but it is not guaranteed.
 This does not harm any user or control traffic and should not trigger RMA
or
 EFA in particular.
 

 Can someone confirm that this is really cosmetic ?

 I'm getting it on a 1X100GBE/CRS-FP140.


 Thanks.

 Regards,

 Antonio Soares, CCIE #18473 (RS/SP)
 amsoa...@netcabo.pt
 http://www.ccie18473.net


 ___
 cisco-nsp mailing list  cisco-nsp@puck.nether.net
 https://puck.nether.net/mailman/listinfo/cisco-nsp
 archive at http://puck.nether.net/pipermail/cisco-nsp/


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] multicast issue

[Jean-Francois . Dube]

We use IneoQuest gear to monitor multicast data after it leaves the source
and before it enters the destination.

The boxes give nice reports of both IP and MPEG, and are MDI compliant
(RFC4445)


Cheers,

JF

cisco-nsp cisco-nsp-boun...@puck.nether.net a écrit sur 2013-07-16
12:53:23 :

 De : R S dim0...@hotmail.com
 A : cisco-nsp@puck.nether.net cisco-nsp@puck.nether.net,
 Date : 2013-07-16 12:59
 Objet : [c-nsp] multicast issue
 Envoyé par : cisco-nsp cisco-nsp-boun...@puck.nether.net

 Hi all

 Just a brainstorming and your possible help.


 I manage a network where multicast is the most important traffic and
 sometimes I get issue by customer where they state that some packetsare
lost…


 Does anybody have an idea or can help me in understanding a possible
 solution in monitoring traffic in real time manner, maybe with the use of
some
 software or appliance or whatelse.

 In my idea I could monitor traffic on the source and on the destination,
 then with  a sort of parsing understand
 if it’s my network loosing the packets or not…



 Any idea ? suggestion ?


 tks
 ___
 cisco-nsp mailing list  cisco-nsp@puck.nether.net
 https://puck.nether.net/mailman/listinfo/cisco-nsp
 archive at http://puck.nether.net/pipermail/cisco-nsp/

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] converting cmts from pure ip routing to mpls pe - uBR7246VXR

[Jean-Francois . Dube]

Hi Aaron,

If you were already using MTU above 1508 for your CMTS to ME3600 links than
you would not need to change anything.

The issue with CMTS to ASR9K only exist if you have configured the very
same MTU on both sides.

You need to check that your IOS-XR MTU is equal to your IOS MTU + 14
bytes.

(You need two 4-bytes labels for MPLS VPN so if you are using Ethernet your
IOS MTU should be 1508 at least)


Cheers,

JF



De :aar...@gvtc.com
A : Jean-Francois Dube jean-francois.d...@videotron.com,
Cc :cisco-nsp@puck.nether.net
Date :  2013-03-29 15:21
Objet : Re: [c-nsp] converting cmts from pure ip routing to mpls pe -
uBR7246VXR



Thanks JF,  is there a reason why this would be required for CMTS to asr9k,
but not required for CMTS to me3600x ?

My CMTS PE to me3600 p is running fine, I didn't make any Mtu changes
there.

Aaron
- Original Message -
From: Jean-Francois Dube jean-francois.d...@videotron.com
To: cisco-nsp@puck.nether.net
Sent: Fri, 29 Mar 2013 09:18:33 -0400 (EDT)
Subject: Re: [c-nsp] converting cmts from pure ip routing to mpls pe -
uBR7246VXR
Hi Aaron,
It sounds like you may be having MTU issue.
At least that is my experience when you can ping and only browse some
websites.
Your CMTS is running IOS and your ASR9K is running IOS-XR.
In IOS-XR you need to account for the L2 header of 14 bytes so the
default MTU is 1514.
If you are running MPLS you'll need to increase the MTU even higher to
account for the extra headers/labels.
That means your CMTS interfaces should be using something like 1516 and
your ASR9K would be 1530.
Cheers,
JF
Jean-François Dubé
Technicien, Opérations Réseau IP
Ingénierie Exploitation des Réseaux
Vidéotron
cisco-nsp-boun...@puck.nether.net a écrit sur 2013-03-28 15:24:42 :
 De : Aaron aar...@gvtc.com
 A : cisco-nsp@puck.nether.net,
 Date : 2013-03-28 15:31
 Objet : [c-nsp] converting cmts from pure ip routing to mpls pe -
uBR7246VXR
 Envoyé par : cisco-nsp-boun...@puck.nether.net

 I have (5) cmts's (uBR7246VXR) ..4 operational and 1 in lab for testing.



 We have a new mpls network comprised of asr901's, me3600's and asr9k's
 functioning as p's and pe's.



 I wanted to move my cmts's off my traditional routed/switched network to
my
 new mpls network. I wanted to have cmts's function as pe's so as to
 potentially take advantage of the mpls LxVPN's



 I successfully converted one of my cmts's to pe and it's running nicely,
 uplinked into p box (me3600). What I did was basically convert wan
uplink
 to mpls, remove igp and replace with core mpls network igp process, and
then
 bring up the expected mp-ibgp and vrf stuff, and then convert all those
 traditional routing interfaces and services (ntp, logging, aaa and
tacacs)
 to be vrf based..works.



 Now for the second cmts that I wanted to convert to pe, I've tried twice
now
 and have seen similar strange behavior. wan uplink utilization drops to
 about 50% of what was previously seen before change..cpu utilization
drops
 from 30-40% utilization to about 0-10%given those observations on the
 first attempt last week, I left it that way, thinking not too much of it
as
 it was 2:30 a.m. in the morning and was thinking that low utilization at
 that hour is conceivable. later I got woken up with a phone call from one
of
 my front-line noc network analysts at 7:15 a.m. saying that we had
several
 subs calling in saying that they could not get to most internet web pages
 but only some were reachable.. (I think the web pages they could get to
were
 our local company web site hosted on-net, and some of our local Akamai
and
 other cached pages)..strangely I could ping and trace to and from those
 subnets on that cmts to and from internet route server (looking glass)
test
 locations.. I didn't know what to make of this..i couldn't find a
problem,
 so was forced to hurry up and throw the cmts back to old switched/routed
 network.



 ..i tried again a few nights ago and saw similar drop in wan utilization
and
 cpu load..not knowing what to make of it, and concerned that subs would
be
 unable to get to web sites that following morning, I moved it back. I
don't
 have a test modem off of this cmts to test with but will need to get one
out
 there if I try again.



 .I have a tac case open, and I am going to try to reproduce this in the
test
 cmts. (but all previous tests on the lab cmts show good results.and as I
 mentioned, the other cmts is running fine in mpls net)



 Difference between the one that worked and the one that doesn't is one is
 uplinked into me3600 (working one) and the one that didn't work is
uplinked
 into asr9k



 Interestingly, the module in the asr9k that I uplink that second cmts
into,
 crashed a couple weeks ago..it took a double ecc error and ios xr showed
a
 forced reset on that module..strange.. tac ios xr team said that it's
 probably an isolated (transient) error and shouldn't happen again, but if
it
 does, they will RMA that 2/20 module in that asr9k. ..several

Re: [c-nsp] converting cmts from pure ip routing to mpls pe - uBR7246VXR

[Jean-Francois . Dube]

Hi Aaron,

It sounds like you may be having MTU issue.

At least that is my experience when you can ping and only  browse some
websites.

Your CMTS is running IOS and your ASR9K is running IOS-XR.

In IOS-XR you need to account for the L2 header of 14 bytes so the
default MTU is 1514.

If you are running MPLS you'll need to increase the MTU even higher to
account for the extra headers/labels.

That means your CMTS interfaces should be using something like 1516 and
your ASR9K would be 1530.

Cheers,

JF

Jean-François Dubé
Technicien, Opérations Réseau IP
Ingénierie Exploitation des Réseaux
Vidéotron

cisco-nsp-boun...@puck.nether.net a écrit sur 2013-03-28 15:24:42 :

 De : Aaron aar...@gvtc.com
 A : cisco-nsp@puck.nether.net,
 Date : 2013-03-28 15:31
 Objet : [c-nsp] converting cmts from pure ip routing to mpls pe -
uBR7246VXR
 Envoyé par : cisco-nsp-boun...@puck.nether.net

 I have (5) cmts's (uBR7246VXR) ..4 operational and 1 in lab for testing.



 We have a new mpls network comprised of asr901's, me3600's and asr9k's
 functioning as p's and pe's.



 I wanted to move my cmts's off my traditional routed/switched network to
my
 new mpls network.  I wanted to have cmts's function as pe's so as to
 potentially take advantage of the mpls LxVPN's



 I successfully converted one of my cmts's to pe and it's running nicely,
 uplinked into p box (me3600).  What I did was basically convert wan
uplink
 to mpls, remove igp and replace with core mpls network igp process, and
then
 bring up the expected mp-ibgp and vrf stuff, and then convert all those
 traditional routing interfaces and services (ntp, logging, aaa and
tacacs)
 to be vrf based..works.



 Now for the second cmts that I wanted to convert to pe, I've tried twice
now
 and have seen similar strange behavior. wan uplink utilization drops to
 about 50% of what was previously seen before change..cpu utilization
drops
 from 30-40% utilization to about 0-10%given those observations on the
 first attempt last week, I left it that way, thinking not too much of it
as
 it was 2:30 a.m. in the morning and was thinking that low utilization at
 that hour is conceivable. later I got woken up with a phone call from one
of
 my front-line noc network analysts at 7:15 a.m. saying that we had
several
 subs calling in saying that they could not get to most internet web pages
 but only some were reachable.. (I think the web pages they could get to
were
 our local company web site hosted on-net, and some of our local Akamai
and
 other cached pages)..strangely I could ping and trace to and from those
 subnets on that cmts to and from internet route server (looking glass)
test
 locations.. I didn't know what to make of this..i couldn't find a
problem,
 so was forced to hurry up and throw the cmts back to old switched/routed
 network.



 ..i tried again a few nights ago and saw similar drop in wan utilization
and
 cpu load..not knowing what to make of it, and concerned that subs would
be
 unable to get to web sites that following morning, I moved it back.  I
don't
 have a test modem off of this cmts to test with but will need to get one
out
 there if I try again.



 .I have a tac case open, and I am going to try to reproduce this in the
test
 cmts. (but all previous tests on the lab cmts show good results.and as I
 mentioned, the other cmts is running fine in mpls net)



 Difference between the one that worked and the one that doesn't is one is
 uplinked into me3600 (working one) and the one that didn't work is
uplinked
 into asr9k



 Interestingly, the module in the asr9k that I uplink that second cmts
into,
 crashed a couple weeks ago..it took a double ecc error and ios xr showed
a
 forced reset on that module..strange.. tac ios xr team said that it's
 probably an isolated (transient) error and shouldn't happen again, but if
it
 does, they will RMA that 2/20 module in that asr9k.   ..several
connections
 are still working on that asr9k linecard and so I didn't think that this
 second cmts being mpls uplinked through there would be an issue..but I
had
 to mention it since I'm seeing weirdness..



 Any thoughts or input would be appreciated.



 Aaron





 ___
 cisco-nsp mailing list  cisco-nsp@puck.nether.net
 https://puck.nether.net/mailman/listinfo/cisco-nsp
 archive at http://puck.nether.net/pipermail/cisco-nsp/


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/