Re: [c-nsp] Nexus Architecture question
--- Begin Message --- I don't know that we can/should draw that conclusion - as you mentioned, you opened a TAC case but from my understanding it was never driven to a terminal resolution - either "known limitation, live with it" or "bug, we will/won't fix it" or "you're doing it wrong". I tested this on 2nd gen n9k and it works fine, I don't have 1st gen gear in my lab so can't empirically confirm that the broadcom-based line cards do or don't behave properly and why, or whether there's an alternative config that does work. One wrinkle here is that these linecards are now well past end of sale, and in fact are past End of Vulnerability/Security Support as well, so maybe it's a moot point anyway. For 10GBASE-T in the 2nd gen I'd suggest the X9788TC-FX linecard and FM-E2 fabric module, these should behave properly with the config I posted earlier. Tim From: Drew Weaver Sent: Friday, June 11, 2021 8:20 AM To: 'Jeffrey G. Fitzwater' ; Tim Stevenson (tstevens) ; 'cisco-nsp@puck.nether.net' Subject: RE: Nexus Architecture question Yes, the problem is that the OS doesn't compensate for the flaws in the Broadcom based line cards. Thanks, -Drew From: Jeffrey G. Fitzwater mailto:jf...@princeton.edu>> Sent: Friday, June 11, 2021 11:19 AM To: Drew Weaver mailto:drew.wea...@thenap.com>>; 'Tim Stevenson (tstevens)' mailto:tstev...@cisco.com>>; 'cisco-nsp@puck.nether.net' mailto:cisco-nsp@puck.nether.net>> Subject: Re: Nexus Architecture question I am not sure this question was asked in this thread, but are you using a custom COPP and not the default? If you have a custom COPP you must apply the new policy with that name prefix i.e. router-core-copp-acl-hsrp Vs copp-acl-hsrp. We do this on our 7 and 9ks so that any new code does not override the custom COPP policies. Jeff Fitzwater Princeton University From: cisco-nsp mailto:cisco-nsp-boun...@puck.nether.net>> on behalf of Drew Weaver mailto:drew.wea...@thenap.com>> Date: Friday, June 11, 2021 at 08:43 To: Drew Weaver mailto:drew.wea...@thenap.com>>, 'Tim Stevenson (tstevens)' mailto:tstev...@cisco.com>>, 'cisco-nsp@puck.nether.net' mailto:cisco-nsp@puck.nether.net>> Subject: Re: [c-nsp] Nexus Architecture question Just for the list this ended up being a Nexus/NXOS limitation. -Original Message- From: cisco-nsp mailto:cisco-nsp-boun...@puck.nether.net>> On Behalf Of Drew Weaver Sent: Tuesday, June 8, 2021 11:07 AM To: 'Tim Stevenson (tstevens)' mailto:tstev...@cisco.com>>; 'cisco-nsp@puck.nether.net' mailto:cisco-nsp@puck.nether.net>> Subject: Re: [c-nsp] Nexus Architecture question Sure, N9K-SUP-B version 9.3(6) Also tried: 7.0(3)I7(8) Thanks, -Drew -Original Message- From: Tim Stevenson (tstevens) mailto:tstev...@cisco.com>> Sent: Friday, June 4, 2021 4:40 PM To: Drew Weaver mailto:drew.wea...@thenap.com>>; 'cisco-nsp@puck.nether.net' mailto:cisco-nsp@puck.nether.net>> Subject: RE: Nexus Architecture question Hi Drew, Can you specify hardware platform and software version here? I am not seeing what you're seeing, the config I sent blocks a BGP port scan in nmap, and prevents BGP peering to anything other than the specified IP. I am testing on Nexus 9500 with 10.1 NXOS; I suspect you are using some other platform eg n5k or n7k? I may be able to try it out here depending on what you're using. Thanks, Tim -----Original Message- From: Drew Weaver mailto:drew.wea...@thenap.com>> Sent: Thursday, June 3, 2021 12:37 PM To: Tim Stevenson (tstevens) mailto:tstev...@cisco.com>>; 'cisco-nsp@puck.nether.net' mailto:cisco-nsp@puck.nether.net>> Subject: RE: Nexus Architecture question Hi Tim, I replied off-list with additional details but I wanted to reply on list to answer your specific questions: I copied the strict policy and then edited it with these classes at the top: policy-map type control-plane Test6 class custom-copp-system-p-bgp-allow police cir 19000 pps bc 128 packets conform transmit violate drop class custom-copp-system-p-bgp-deny police cir 0 pps bc 1 packets conform transmit violate drop These are the configured class-maps: Class-map type control-plane match-any custom-copp-system-p-bgp-allow match access-group name custom-copp-system-p-acl-bgp-allow Class-map type control-plane match-any custom-copp-system-p-bgp-deny match access-group name custom-copp-system-p-acl-bgp-deny These are the configured ACLs: IP access list custom-copp-system-p-acl-bgp-allow 3 permit tcp 192.168.1.2/32 gt 1023 any eq bgp 4 permit tcp 192.168.1.2/32 eq bgp any gt 1023 IP access list custom-copp-system-p-acl-bgp-deny 1 permit tcp any any eq bgp 10 permit tcp any gt 1023 any eq bgp 20 permit tcp any eq bgp any gt 1023 control-plane service-policy input Test6 The primary difference that
Re: [c-nsp] Nexus Architecture question
--- Begin Message --- Hi Drew, Can you specify hardware platform and software version here? I am not seeing what you're seeing, the config I sent blocks a BGP port scan in nmap, and prevents BGP peering to anything other than the specified IP. I am testing on Nexus 9500 with 10.1 NXOS; I suspect you are using some other platform eg n5k or n7k? I may be able to try it out here depending on what you're using. Thanks, Tim -Original Message- From: Drew Weaver Sent: Thursday, June 3, 2021 12:37 PM To: Tim Stevenson (tstevens) ; 'cisco-nsp@puck.nether.net' Subject: RE: Nexus Architecture question Hi Tim, I replied off-list with additional details but I wanted to reply on list to answer your specific questions: I copied the strict policy and then edited it with these classes at the top: policy-map type control-plane Test6 class custom-copp-system-p-bgp-allow police cir 19000 pps bc 128 packets conform transmit violate drop class custom-copp-system-p-bgp-deny police cir 0 pps bc 1 packets conform transmit violate drop These are the configured class-maps: Class-map type control-plane match-any custom-copp-system-p-bgp-allow match access-group name custom-copp-system-p-acl-bgp-allow Class-map type control-plane match-any custom-copp-system-p-bgp-deny match access-group name custom-copp-system-p-acl-bgp-deny These are the configured ACLs: IP access list custom-copp-system-p-acl-bgp-allow 3 permit tcp 192.168.1.2/32 gt 1023 any eq bgp 4 permit tcp 192.168.1.2/32 eq bgp any gt 1023 IP access list custom-copp-system-p-acl-bgp-deny 1 permit tcp any any eq bgp 10 permit tcp any gt 1023 any eq bgp 20 permit tcp any eq bgp any gt 1023 control-plane service-policy input Test6 The primary difference that I notice between the two configurations is that your allow is set to cos 7 and on mine it is not specified and yours is policing on bps and mine is policing on pps. (I am pretty certain that throughout the process I have tried it both ways). I find it a bit hard to follow that on this platform there is no way to write "just block everything that matches this class" you still have to give it a "burst" [which I obviously don't want] I understand that behind the scenes the control plane is supposed to "do the right thing" but it is just difficult to understand that from reading it. With this configuration applied: 1) Any host that NMAPs the device sees 179 open. 2) BGP sessions establish (if configured on the Nexus) for hosts that are not 192.168.1.2/32 Is there any way to find out what class traffic from a specific SRC IP or to a specific port is getting caught at in a CoPP policy? I suspect that the TCP/179 traffic is just bypassing these classes altogether and most likely are getting handled further down in the CoPP policy but I haven't been able to prove that. Thanks so much. -Drew -----Original Message- From: Tim Stevenson (tstevens) Sent: Wednesday, June 2, 2021 5:31 PM To: Drew Weaver ; 'cisco-nsp@puck.nether.net' Subject: RE: Nexus Architecture question Hi Drew, In answer to your question about BGP, the BGP process runs only on the supervisor engine, it does not run on the linecards or anywhere else. It's a single process, not a per-interface process or anything like that. Curious how exactly you are configuring CoPP to filter this? With default CoPP, I get an "open/tcpwrapped" (green) response on TCP 179; I was able to get a "filtered" (red) response by adding a CoPP class that matches on BGP and polices to a CIR of 0. I preceeded that class with a class that matches on a specific neighborship - that BGP neighborship is successfully established while nmap still returns as filtered from my host. ip access-list allow-bgp 10 permit tcp 10.1.1.1/32 gt 1023 10.1.1.2/32 eq bgp 20 permit tcp 10.1.1.2/32 eq bgp 10.1.1.1/32 gt 1023 ip access-list drop-bgp 10 permit tcp any any eq bgp 20 permit tcp any eq bgp any ! class-map type control-plane match-any allow-bgp match access-group name allow-bgp class-map type control-plane match-any drop-bgp match access-group name drop-bgp ! policy-map type control-plane test-copp-policy-strict class allow-bgp set cos 7 police cir 36000 kbps bc 128 bytes conform transmit violate drop class drop-bgp police cir 0 bps bc 32000 bytes conform transmit violate drop Hope that helps, Tim -Original Message- From: cisco-nsp On Behalf Of Drew Weaver Sent: Wednesday, June 2, 2021 6:41 AM To: 'cisco-nsp@puck.nether.net' Subject: [c-nsp] Nexus Architecture question Has anyone seen a document from Cisco that shows where various processes running on various Nexus switches actually run from? For example on a 9508 the nxapi runs in a Linux VM and in order to secure it you have to drop into the VM and use iptables. I am trying to figure out where the BGP proce
Re: [c-nsp] Nexus Architecture question
--- Begin Message --- Hi Drew, In answer to your question about BGP, the BGP process runs only on the supervisor engine, it does not run on the linecards or anywhere else. It's a single process, not a per-interface process or anything like that. Curious how exactly you are configuring CoPP to filter this? With default CoPP, I get an "open/tcpwrapped" (green) response on TCP 179; I was able to get a "filtered" (red) response by adding a CoPP class that matches on BGP and polices to a CIR of 0. I preceeded that class with a class that matches on a specific neighborship - that BGP neighborship is successfully established while nmap still returns as filtered from my host. ip access-list allow-bgp 10 permit tcp 10.1.1.1/32 gt 1023 10.1.1.2/32 eq bgp 20 permit tcp 10.1.1.2/32 eq bgp 10.1.1.1/32 gt 1023 ip access-list drop-bgp 10 permit tcp any any eq bgp 20 permit tcp any eq bgp any ! class-map type control-plane match-any allow-bgp match access-group name allow-bgp class-map type control-plane match-any drop-bgp match access-group name drop-bgp ! policy-map type control-plane test-copp-policy-strict class allow-bgp set cos 7 police cir 36000 kbps bc 128 bytes conform transmit violate drop class drop-bgp police cir 0 bps bc 32000 bytes conform transmit violate drop Hope that helps, Tim -Original Message- From: cisco-nsp On Behalf Of Drew Weaver Sent: Wednesday, June 2, 2021 6:41 AM To: 'cisco-nsp@puck.nether.net' Subject: [c-nsp] Nexus Architecture question Has anyone seen a document from Cisco that shows where various processes running on various Nexus switches actually run from? For example on a 9508 the nxapi runs in a Linux VM and in order to secure it you have to drop into the VM and use iptables. I am trying to figure out where the BGP process lives (for lack of a better word). Does it run on the line cards? In the control plane? Both? Does it vary depending on which model and which line cards? The reason I am asking is because I've noticed that no matter what I do I cannot seem to "close" the BGP port by using CoPP. It always shows up as being open when doing a port scan against the system using NMAP. I know that the switch should not establish a connection with random hosts but I really am getting hung up on it being 'scannable'/visible at all. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 9300 sflow performance
--- Begin Message --- Hi Satish, I have not worked much with this particular switch, but per the docs you need to size the TCAM regions appropriately to enable sflow. Your show command output filters any line with "0" in it - the regions in question are not carved by default, they are set to 0. To provision them, you'll need to borrow entries from some other region (ie, size down another region to size up the span/sflow regions). leaf1# show hardware access-list tcam region | eg -i sflow sFlow ACL [sflow] size =0 SPAN+sFlow ACL [span-sflow] size =0 leaf1# Please check the section "Prerequisites for sFlow" section in the documentation here for details: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/system_management/configuration/guide/b_Cisco_Nexus_9000_Series_NX-OS_System_Management_Configuration_Guide_7x/b_Cisco_Nexus_9000_Series_NX-OS_System_Management_Configuration_Guide_7x_chapter_011000.html Hope that helps, Tim -Original Message- From: Satish Patel Sent: Tuesday, April 13, 2021 9:13 AM To: Tim Stevenson (tstevens) Cc: Lasse Birnbaum Jensen ; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 9300 sflow performance Folks, I know this thread is old but I am having some strange issues, you guys may help me out. This is my config on CIsco Nexus 9396PX (NXOS 7.0(3)I7(8)) hardware access-list tcam region span-sflow 256 ! feature sflow sflow counter-poll-interval 30 sflow collector-ip 10.30.0.91 vrf management sflow collector-port 9995 sflow agent-ip 172.30.0.26 When i am trying to set a data-source interface I am not able to use 40G ports. sflow data-source interface e2/1 above command silently accepts command but when i run "show run sflow" command to verify i am not seeing data-source interface e2/1 in config, but if i use 10G interface e1/1 that does work. Cisco official document saying: Make sure that the sFlow and SPAN ACL TCAM region sizes are configured for any uplink ports that are to be configured as an sFlow data source on the following devices: Cisco Nexus 9332PQ, 9372PX, 9372TX, and 93120TX switches and Cisco Nexus 9396PX, 9396TX, and 93128TX switches with the N9K-M6PQ or N9K-M12PQ generic expansion module (GEM). I didn't find any region for SPAN ACL to carve size, what and where do I need to set SPAN ACL size? This is what i have currently. (config)# show hardware access-list tcam region | exclude 0 IPV4 PACL [ifacl] size = 512 IPV4 Port QoS [qos] size = 256 IPV4 VACL [vacl] size = 512 IPV4 RACL [racl] size = 512 Egress IPV4 VACL [vacl] size = 512 Egress IPV4 RACL [e-racl] size = 256 Ingress System size = 256 Egress System size = 256 Ingress COPP [copp] size = 256 Redirect [redirect] size = 512 NS IPV4 Port QoS [ns-qos] size = 256 NS IPV4 VLAN QoS [ns-vqos] size = 256 NS IPV4 L3 QoS [ns-l3qos] size = 256 VPC Convergence/ES-Multi Home [vpc-convergence] size = 256 ranger+ IPV4 QoS [rp-qos] size = 256 ranger+ IPV6 QoS [rp-ipv6-qos] size = 256 ranger+ MAC QoS [rp-mac-qos] size = 256 sFlow ACL [sflow] size = 256 On Mon, May 13, 2019 at 2:08 PM Tim Stevenson (tstevens) via cisco-nsp wrote: > > > > > ------ Forwarded message -- > From: "Tim Stevenson (tstevens)" > To: Lasse Birnbaum Jensen , "cisco-nsp@puck.nether.net" > > Cc: > Bcc: > Date: Mon, 13 May 2019 18:06:58 + > Subject: RE: [c-nsp] Nexus 9300 sflow performance > First gen n9k does not support Netflow at all, only sflow. 2nd gen > (EX/FX/FX2) support both, but there is the SPAN+SFlow limitation (we are > working on fixing that for FX2, which can theoretically support these > concurrently). > > For recommended sampling value, we set the rate limiters at values that we > feel the switch can handle. So you should select a sampling value where you > do NOT see HWRL drops, as then it's truly 1:n. If you drop samples, then your > sampling is 1:n up to the point where you tail drop excess packets, and that > will skew the samples you actually process and export and reduce the > statistical validity of the sample. > > Using mgmt0 should be fine for sflow export. > > Hope that helps, > Tim > > > -Original Message- > From: Lasse Birnbaum Jensen > Sent: Monday, May 13, 2019 10:58 AM > To: cisco-nsp@puck.nether.net > Subject: Re: [c-nsp] Nexus 9300 sflow perform
Re: [c-nsp] NXOS 7 apply VTY access-list to both IPv4 and IPv6
--- Begin Message --- Should be like this: tstevens-9236c-1(config)# line vty tstevens-9236c-1(config-line)# ip ip ipv6 tstevens-9236c-1(config-line)# ip access-class foo in tstevens-9236c-1(config-line)# ipv6 access-class bar in tstevens-9236c-1(config-line)# sh run | sec vty line vty access-class foo in ipv6 access-class bar in tstevens-9236c-1(config-line)# Hope that helps, Tim -Original Message- From: cisco-nsp On Behalf Of Francisco José Bernal Fernández Sent: Wednesday, January 13, 2021 9:37 AM To: Drew Weaver Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] NXOS 7 apply VTY access-list to both IPv4 and IPv6 Hi . You need to create ipv6 acces-list. Regards > El 13 ene 2021, a las 18:10, Drew Weaver escribió: > > Nevermind I actually figured this out. I had created the V6 ACL as a V4 ACL. > > Apologies for the bytes. > > -Original Message- > From: cisco-nsp On Behalf Of Drew Weaver > Sent: Wednesday, January 13, 2021 12:01 PM > To: 'cisco-nsp@puck.nether.net' > Subject: [c-nsp] NXOS 7 apply VTY access-list to both IPv4 and IPv6 > > Hello, > > I am doing basic configuration on a switch with NXOS 7. It seems to not want > to let me specify different ACLs per "address family" even though it seems to > imply that it should be possible. If I enter the command as "ip access-class > V4ACL in" and then try to enter "ipv6 access-class V6ACL in" it says: ACL > with given name exists with different type > > It is not a huge deal because CoPP filters it first, but I would like to do > it for the sake of paranoia and consistency. > > It doesn't seem to be possible to create an ACL that is both IPv4 and IPv6 on > this platform, so I am not entirely certain how you do this. > > Thanks, > -Drew > > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://urldefense.proofpoint.com/v2/url?u=https-3A__puck.nether.net_mailman_listinfo_cisco-2Dnsp=DwICAg=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw=_0xdyUFzHei0BVlt5FeNWn1NB6M1VAlq8UquBTATmY8=zdo83IO7gCmeeQPmDv_zrwEhIS5FRVbvUttgzMeSEkg= > archive at > https://urldefense.proofpoint.com/v2/url?u=http-3A__puck.nether.net_pipermail_cisco-2Dnsp_=DwICAg=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw=_0xdyUFzHei0BVlt5FeNWn1NB6M1VAlq8UquBTATmY8=HavN1jnpsgGQa4V-PZQAKKapytaGyR47SsWz81CBTsg= > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 9300 sflow performance
--- Begin Message --- First gen n9k does not support Netflow at all, only sflow. 2nd gen (EX/FX/FX2) support both, but there is the SPAN+SFlow limitation (we are working on fixing that for FX2, which can theoretically support these concurrently). For recommended sampling value, we set the rate limiters at values that we feel the switch can handle. So you should select a sampling value where you do NOT see HWRL drops, as then it's truly 1:n. If you drop samples, then your sampling is 1:n up to the point where you tail drop excess packets, and that will skew the samples you actually process and export and reduce the statistical validity of the sample. Using mgmt0 should be fine for sflow export. Hope that helps, Tim -Original Message- From: Lasse Birnbaum Jensen Sent: Monday, May 13, 2019 10:58 AM To: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 9300 sflow performance Im not totally sure about the N9300 architecture. But normally the mgmt interface is connected "directly" to the control plane cpu, thus having it process a lot of packets will take CPU resources and might impact control-plane protocols and jobs. Netflow is performed in the ASICs and I think it would be better to use an ASIC bounded interface if possible. Best regards Lasse Birnbaum Jensen Network Architect IT-services T +45 65 50 28 73 M +45 60 11 28 73 la...@sdu.dk http://www.sdu.dk/ansat/lbje University of Southern Denmark Campusvej 55 DK-5230 Odense M www.sdu.dk <http://www.sdu.dk/> Lasse Birnbaum Jensen D. 20/03/2019 18.27 skrev "cisco-nsp på vegne af Satish Patel" : Thanks Tim, Here is the output of show hardware rate-limiter. ( i believe it's 40k) This is my first time dealing with SFLOW, Can you share some configuration parameter i should use for best practice would be great, What is 1-in-N sample actually? I am planning to use mgmt0 interface for SFLOW and its 1G so i assume it will handle all the flow. do you seeing any concern there? # show hardware rate-limiter Units for Config: packets per second Allowed, Dropped & Total: aggregated since last clear counters Module: 1 R-L Class Config Allowed Dropped Total +--++---+---+-+ L3 glean 100 0 0 0 L3 mcast loc-grp3000 0 0 0 access-list-log 100 0 0 0 bfd1 0 0 0 exception 50 0 0 0 fex 3000 0 0 0 span 50 0 0 0 dpss6400 0 0 0 sflow 4 25134089890 0 25134089890 On Wed, Mar 20, 2019 at 12:07 PM Tim Stevenson (tstevens) wrote: > > Yes, this is 1st gen. The SFLOW/SPAN restriction should not apply there. > > Re: 60Gbps/24Mpps and SFLOW, SFLOW does not do aggregation of stats for flows in the switch like netflow does - it's just 1-in-n packet sampling. As such, the value of "n" should be high enough that both the switch & the collector are not overburdened. Note that we will rate limit SFLOW copies to the CPU so that's the first 'bottleneck'. If you end up tail-dropping samples, the statistical validity of your sampled set goes out the window, so you want to ensure that 1-in-n is a number that does not hit that rate limiter. > > I don't have a 1st gen switch handy to see what the defaults are for that value. It should show up in 'sh hardware rate-limiter'. In 9300-EX with 9.2.2 it's 40Kpps. > > Beyond that, you also want to make sure the collector is able to consume everything coming from all sflow enabled switches without dropping, for the same reason mentioned above. > > Hope that helps, > Tim > > > -Original Message- > From: Satish Patel > Sent: Wednesday, March 20, 2019 8:40 AM > To: Nick Cutting > Cc: Tim Stevenson (tstevens) ; cisco-nsp@puck.nether.net > Subject: Re: [c-nsp] Nexus 9300 sflow performance > > We have cisco Nexus9000 C9396PX > > 60 Gbs is data traffic, and 24Mpps ( packet per second ) not sure how > to convert it into flows. Could you please share your sflow > configuration if you don't mind? > > I had nfsen in past with 8CPU / 4GB memory but it was damn slow :( > but it could b
Re: [c-nsp] Cisco Nexus Data Broker
--- Begin Message --- Release notes have this information. EX & FX are both supported. 3600-R is not. See Table 2/Table 3 here: https://www.cisco.com/c/en/us/td/docs/net_mgmt/xnc/nexus_data_broker/release_notes/Nexus_Data_Broker_Release_Notes_371.html Hope that helps, Tim -Original Message- From: cisco-nsp On Behalf Of Robert Hass Sent: Friday, May 10, 2019 5:23 AM To: cisco-nsp@puck.nether.net Subject: [c-nsp] Cisco Nexus Data Broker Hi I cannot find information which current models of Nexus switches are supporting Cisco Nexus Data Broker. Documents on cisco.com are quite outdated - from 2014 or 2017. I'm wondering if Data Broker is supported on Nexus 9300 EX/FX and Nexus 3600-R series. Rob ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] TCAM utilization on Nexus 9396
--- Begin Message --- Please check the config guide. I am not as familiar w/the 1st gen switches as 2nd gen, but there should be at least some level of reconfigurability of the regions in gen 1. So you may be able to size up the region you want by removing entries from some other region. Yes, region resizing requires a switch reboot. Tim -Original Message- From: Satish Patel Sent: Wednesday, March 20, 2019 12:12 PM To: Tim Stevenson (tstevens) Cc: Cisco Network Service Providers ; Nick Cutting Subject: Re: TCAM utilization on Nexus 9396 Thanks for clarification, i have noticed when i add 1 rules number bump +1 but i believe you can't go above 510 right? that is hard limit if i am not wrong. also changing in resource required reload. On Wed, Mar 20, 2019 at 2:07 PM Tim Stevenson (tstevens) wrote: > > Yes, ACL lines consume space in the TCAM. TCAM can be recarved according to > the features in use/required. > > As long as the policy fits in the available TCAM space for that feature > (software will complain and fail your config if it won't), enforcement is at > full rate, no performance penalty for that. > > Tim > > -Original Message- > From: Satish Patel > Sent: Wednesday, March 20, 2019 10:46 AM > To: Cisco Network Service Providers ; Nick Cutting > ; Tim Stevenson (tstevens) > Subject: TCAM utilization on Nexus 9396 > > Folks and ( Tim/Nick ) > > I have Cisco Nexus 9396 L3 switch and running bunch of ACL ( IPv4 > Access-list to block certain traffic ) today i was reading about TCAM > and when i look at switch i found following utilization, so trying to > understand how ACL relationship with TCAM. > > - Does number of ACL impact TCAM utilization or traffic ? > > > # show hardware access-list resource utilization > > slot 1 > === > > > > INSTANCE 0x0 > - > > > ACL Hardware Resource Utilization (Mod 1) > -- > UsedFreePercent > Utilization > --- > Ingress IPv4 PACL 3 509 0.59 > Ingress IPv4 Port QoS 4 252 1.56 > Ingress IPv4 VACL 2 510 0.39 > Ingress IPv4 RACL 226 286 44.14 > Egress IPv4 VACL3 509 0.59 > Egress IPv4 RACL3 253 1.17 > SUP COPP205 51 80.08 > SUP COPP Reason Code TCAM 6 122 4.69 > Redirect2 510 0.39 > SPAN21 235 8.20 > VPC Convergence 1 255 0.39 > > LOU 2 22 8.33 > Both LOU Operands 2 > Single LOU Operands 0 > LOU L4 src port:1 > LOU L4 dst port:1 > LOU L3 packet len: 0 > LOU IP tos: 0 > LOU IP dscp:0 > LOU ip precedence: 0 > LOU ip TTL: 0 > TCP Flags 0 16 0.00 > > Protocol CAM2 244 0.81 > Mac Etype/Proto CAM 0 14 0.00 > > L4 op labels, Tcam 00 10230.00 > L4 op labels, Tcam 21 62 1.58 > L4 op labels, Tcam 60 20470.00 > > Ingress Dest info table 0 512 0.00 > > Egress Dest info table 0 512 0.00 --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] TCAM utilization on Nexus 9396
--- Begin Message --- Yes, ACL lines consume space in the TCAM. TCAM can be recarved according to the features in use/required. As long as the policy fits in the available TCAM space for that feature (software will complain and fail your config if it won't), enforcement is at full rate, no performance penalty for that. Tim -Original Message- From: Satish Patel Sent: Wednesday, March 20, 2019 10:46 AM To: Cisco Network Service Providers ; Nick Cutting ; Tim Stevenson (tstevens) Subject: TCAM utilization on Nexus 9396 Folks and ( Tim/Nick ) I have Cisco Nexus 9396 L3 switch and running bunch of ACL ( IPv4 Access-list to block certain traffic ) today i was reading about TCAM and when i look at switch i found following utilization, so trying to understand how ACL relationship with TCAM. - Does number of ACL impact TCAM utilization or traffic ? # show hardware access-list resource utilization slot 1 === INSTANCE 0x0 - ACL Hardware Resource Utilization (Mod 1) -- UsedFreePercent Utilization --- Ingress IPv4 PACL 3 509 0.59 Ingress IPv4 Port QoS 4 252 1.56 Ingress IPv4 VACL 2 510 0.39 Ingress IPv4 RACL 226 286 44.14 Egress IPv4 VACL3 509 0.59 Egress IPv4 RACL3 253 1.17 SUP COPP205 51 80.08 SUP COPP Reason Code TCAM 6 122 4.69 Redirect2 510 0.39 SPAN21 235 8.20 VPC Convergence 1 255 0.39 LOU 2 22 8.33 Both LOU Operands 2 Single LOU Operands 0 LOU L4 src port:1 LOU L4 dst port:1 LOU L3 packet len: 0 LOU IP tos: 0 LOU IP dscp:0 LOU ip precedence: 0 LOU ip TTL: 0 TCP Flags 0 16 0.00 Protocol CAM2 244 0.81 Mac Etype/Proto CAM 0 14 0.00 L4 op labels, Tcam 00 10230.00 L4 op labels, Tcam 21 62 1.58 L4 op labels, Tcam 60 20470.00 Ingress Dest info table 0 512 0.00 Egress Dest info table 0 512 0.00 --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 9300 sflow performance
--- Begin Message --- -Original Message- From: Satish Patel Sent: Wednesday, March 20, 2019 10:23 AM To: Tim Stevenson (tstevens) Cc: Nick Cutting ; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 9300 sflow performance Thanks Tim, Here is the output of show hardware rate-limiter. ( i believe it's 40k) >> Yes looks to be the same as on 9300-EX/FX/FX2. This is my first time dealing with SFLOW, Can you share some configuration parameter i should use for best practice would be great, What is 1-in-N sample actually? >> Sampling rate is controlled via config: tstevens-93180yc-ex-4(config)# sflow sampling-rate ? <4096-10> SFlow Sampling rate >> You need to calculate the PPS of the traffic on the source interfaces to >> determine the sampling rate that will keep the max number of samples to >> under 40K from all sources. I am planning to use mgmt0 interface for SFLOW and its 1G so i assume it will handle all the flow. do you seeing any concern there? >> With max of 40Kpps of sampling, it should be fine on 1G mgmt0. >> Tim # show hardware rate-limiter Units for Config: packets per second Allowed, Dropped & Total: aggregated since last clear counters Module: 1 R-L Class Config Allowed DroppedTotal +--++---+---+-+ L3 glean 100 0 0 0 L3 mcast loc-grp3000 0 0 0 access-list-log 100 0 0 0 bfd1 0 0 0 exception 50 0 0 0 fex 3000 0 0 0 span 50 0 0 0 dpss6400 0 0 0 sflow 4 25134089890 0 25134089890 On Wed, Mar 20, 2019 at 12:07 PM Tim Stevenson (tstevens) wrote: > > Yes, this is 1st gen. The SFLOW/SPAN restriction should not apply there. > > Re: 60Gbps/24Mpps and SFLOW, SFLOW does not do aggregation of stats for flows > in the switch like netflow does - it's just 1-in-n packet sampling. As such, > the value of "n" should be high enough that both the switch & the collector > are not overburdened. Note that we will rate limit SFLOW copies to the CPU so > that's the first 'bottleneck'. If you end up tail-dropping samples, the > statistical validity of your sampled set goes out the window, so you want to > ensure that 1-in-n is a number that does not hit that rate limiter. > > I don't have a 1st gen switch handy to see what the defaults are for that > value. It should show up in 'sh hardware rate-limiter'. In 9300-EX with 9.2.2 > it's 40Kpps. > > Beyond that, you also want to make sure the collector is able to consume > everything coming from all sflow enabled switches without dropping, for the > same reason mentioned above. > > Hope that helps, > Tim > > > -Original Message- > From: Satish Patel > Sent: Wednesday, March 20, 2019 8:40 AM > To: Nick Cutting > Cc: Tim Stevenson (tstevens) ; cisco-nsp@puck.nether.net > Subject: Re: [c-nsp] Nexus 9300 sflow performance > > We have cisco Nexus9000 C9396PX > > 60 Gbs is data traffic, and 24Mpps ( packet per second ) not sure how > to convert it into flows. Could you please share your sflow > configuration if you don't mind? > > I had nfsen in past with 8CPU / 4GB memory but it was damn slow :( > but it could be me.. i will set up again and see if it worth it or > not. > > On Wed, Mar 20, 2019 at 11:34 AM Nick Cutting wrote: > > > > Good point. We waited for the second Gen > > > > Regarding 60 Gbs, isn’t that is the data traffic, not the flows or sampled > > flows levels? > > > > Our NFSEn box is centos > > > > 4 vCPU and 4 GBrams > > > > Collecting flows from maybe only 30 devices, about 20Gbs and 3k flows per > > sec. > > > > -Original Message- > > From: Tim Stevenson (tstevens) > > Sent: Wednesday, March 20, 2019 11:20 AM > > To: Nick Cutting ; Satish Patel > > ; cisco-nsp@puck.nether.net > > Subject: RE: [c-nsp] Nexus 9300 sflow performance > > > > This message originated from outside your organization. > > > > Make sure you distinguish between N9300 (1st generation) and > > N9300-EX/FX/FX2 (2nd generation). The SFLOW + SPAN limitation applies only > > to the latter. It's also on the latter that Netflow is
Re: [c-nsp] Nexus 9300 sflow performance
--- Begin Message --- Yes, this is 1st gen. The SFLOW/SPAN restriction should not apply there. Re: 60Gbps/24Mpps and SFLOW, SFLOW does not do aggregation of stats for flows in the switch like netflow does - it's just 1-in-n packet sampling. As such, the value of "n" should be high enough that both the switch & the collector are not overburdened. Note that we will rate limit SFLOW copies to the CPU so that's the first 'bottleneck'. If you end up tail-dropping samples, the statistical validity of your sampled set goes out the window, so you want to ensure that 1-in-n is a number that does not hit that rate limiter. I don't have a 1st gen switch handy to see what the defaults are for that value. It should show up in 'sh hardware rate-limiter'. In 9300-EX with 9.2.2 it's 40Kpps. Beyond that, you also want to make sure the collector is able to consume everything coming from all sflow enabled switches without dropping, for the same reason mentioned above. Hope that helps, Tim -Original Message- From: Satish Patel Sent: Wednesday, March 20, 2019 8:40 AM To: Nick Cutting Cc: Tim Stevenson (tstevens) ; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 9300 sflow performance We have cisco Nexus9000 C9396PX 60 Gbs is data traffic, and 24Mpps ( packet per second ) not sure how to convert it into flows. Could you please share your sflow configuration if you don't mind? I had nfsen in past with 8CPU / 4GB memory but it was damn slow :( but it could be me.. i will set up again and see if it worth it or not. On Wed, Mar 20, 2019 at 11:34 AM Nick Cutting wrote: > > Good point. We waited for the second Gen > > Regarding 60 Gbs, isn’t that is the data traffic, not the flows or sampled > flows levels? > > Our NFSEn box is centos > > 4 vCPU and 4 GBrams > > Collecting flows from maybe only 30 devices, about 20Gbs and 3k flows per sec. > > -Original Message- > From: Tim Stevenson (tstevens) > Sent: Wednesday, March 20, 2019 11:20 AM > To: Nick Cutting ; Satish Patel ; > cisco-nsp@puck.nether.net > Subject: RE: [c-nsp] Nexus 9300 sflow performance > > This message originated from outside your organization. > > Make sure you distinguish between N9300 (1st generation) and N9300-EX/FX/FX2 > (2nd generation). The SFLOW + SPAN limitation applies only to the latter. > It's also on the latter that Netflow is supported, which can run concurrently > with SPAN sessions. > > Tim > > -Original Message- > From: cisco-nsp On Behalf Of Nick Cutting > Sent: Wednesday, March 20, 2019 6:19 AM > To: Satish Patel ; cisco-nsp@puck.nether.net > Subject: Re: [c-nsp] Nexus 9300 sflow performance > > We use sflow on 9300's, no performance hit - but you cannot use span sessions > at the same time. > > Newer code revisions support netflow, without the SPAN session limitation, > although we have not tried netflow on the 9300 yet. > > For a collector We use NFSEN - opensource, and quite a big install base, and > it seems to handle a lot of flows. > > It supports sflow and netflow as we have a mix, just make sure you add the > sflow option at build time as it’s a bit funky old linux to add it after. > > > > -Original Message- > From: cisco-nsp On Behalf Of Satish Patel > Sent: Wednesday, March 20, 2019 8:21 AM > To: cisco-nsp@puck.nether.net > Subject: [c-nsp] Nexus 9300 sflow performance > > This message originates from outside of your organisation. > > Folks, > > I have L3 Nexus 9300 switch which is running 60Gbps traffic on ISP interface > so I’m planning to run sflow on that specific interference to get flow. > > Does it going to create any performances issue on switch? > > Can I run sflow on Layer 3 LACP interface? > > Can anyone suggest free open source sflow collector? > > Sent from my iPhone > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ > > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 9300 sflow performance
--- Begin Message --- Make sure you distinguish between N9300 (1st generation) and N9300-EX/FX/FX2 (2nd generation). The SFLOW + SPAN limitation applies only to the latter. It's also on the latter that Netflow is supported, which can run concurrently with SPAN sessions. Tim -Original Message- From: cisco-nsp On Behalf Of Nick Cutting Sent: Wednesday, March 20, 2019 6:19 AM To: Satish Patel ; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 9300 sflow performance We use sflow on 9300's, no performance hit - but you cannot use span sessions at the same time. Newer code revisions support netflow, without the SPAN session limitation, although we have not tried netflow on the 9300 yet. For a collector We use NFSEN - opensource, and quite a big install base, and it seems to handle a lot of flows. It supports sflow and netflow as we have a mix, just make sure you add the sflow option at build time as it’s a bit funky old linux to add it after. -Original Message- From: cisco-nsp On Behalf Of Satish Patel Sent: Wednesday, March 20, 2019 8:21 AM To: cisco-nsp@puck.nether.net Subject: [c-nsp] Nexus 9300 sflow performance This message originates from outside of your organisation. Folks, I have L3 Nexus 9300 switch which is running 60Gbps traffic on ISP interface so I’m planning to run sflow on that specific interference to get flow. Does it going to create any performances issue on switch? Can I run sflow on Layer 3 LACP interface? Can anyone suggest free open source sflow collector? Sent from my iPhone ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Qos Statistics on the 7K
--- Begin Message --- Hi Brad, I checked this on n7700 F3 - concur that even w/'statistics per-entry', the hit count is not incrementing in 'sh ip access' output when the ACL is used for QOS classification. Same behavior in 8.3.1. >From what I see, the statistics are in fact incrementing in hardware, you can >verify by attaching to the LC via 'attach mod x' and using 'sh sys internal >access-list input entries detail' and find the block with your ACL (might be a >bit tedious doing it this way as all policies, including CoPP etc, will be >listed out there). Not sure why that is not just being exported up and >aggregated in the sup, though the 'usual' use-case for monitoring ACL hit >counts has centered around security ACLs. VDC-1 Ethernet2/1 : INSTANCE 0x0 --- Tcam 0 resource usage: -- Label_a = 0x201 Bank 1 -- IPv4 Class Policies: QoS(all-ip) Netflow profile: 0 Netflow deny profile: 0 Entries: [Index] Entry [Stats] - [0015:000b:000b] qos ip 0.0.0.0/0 10.1.1.0/24 [398869316] I guess you're hoping to figure out which specific ACEs are matching in each class (vs just seeing the total number of packets classified in each class, as seen in 'sh policy-map interface')? I can check w/our engineering team and see if there's some reason this has not been implemented. Hope that helps, Tim -Original Message- From: cisco-nsp On Behalf Of Bradley Ordner Sent: Wednesday, November 14, 2018 8:49 PM To: cisco-nsp@puck.nether.net Subject: [c-nsp] Qos Statistics on the 7K Hi, This may have been asked before, even on Cisco Support Community I have an answer but it doesn't seem to be working for me. We have a Layer 3 port with a QoS policy for marking traffic inbound. I have added the 'statistics per-entry' command in our ACL but I do not see any hits. When checking the policy and queueing, I see traffic being matched. We are only marking inbound on this port, is it not supported or do I have a bug? I am on version - 7.2(0)D1(1) Match: access-group QOSACL- BLAH 46082768 packets set dscp 56 Thanks Brad Ordner ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Fixed switch based on N9K-X9636C-RX
--- Begin Message --- There isn't one. Closest is Nexus 3636C-R, this is J+ based like the 9636C-RX but with on-chip tables only (no external TCAM). Tim -Original Message- From: cisco-nsp On Behalf Of Drew Weaver Sent: Wednesday, August 8, 2018 5:39 AM To: cisco-nsp@puck.nether.net Subject: [c-nsp] Fixed switch based on N9K-X9636C-RX Has anyone come across a 1U Nexus switch that has TCAM like the N9K-X9636C-RX? Thanks, -Drew ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] N9K ASIC port grouping
106 255 140 -13 1632 BR, Pedro Caetano On Mon, Jun 25, 2018 at 2:00 PM, Tim Stevenson (tstevens) via cisco-nsp < cisco-nsp@puck.nether.net> wrote: > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ > > > -- Forwarded message -- > From: "Tim Stevenson (tstevens)" > To: Youssef Bengelloun-Zahr , Cisco Network Service > Providers > Cc: > Bcc: > Date: Mon, 25 Jun 2018 18:00:40 + > Subject: RE: [c-nsp] N9K ASIC port grouping > Not aware of a single document that will show that for all platforms. Best > way is "show interface hardware-mappings". "Unit" and "Slice" are the main > points of reference (unit == ASIC instance, slice == pipeline). > > E.g., 93180yc-ex: > leaf1# sh int hard > Legends: >SMod - Source Mod. 0 is N/A >Unit - Unit on which port resides. N/A for port channels >HPort - Hardware Port Number or Hardware Trunk Id: >HName - Hardware port name. None means N/A >FPort - Fabric facing port number. 255 means N/A >NPort - Front panel port number >VPort - Virtual Port Number. -1 means N/A >Slice - Slice Number. N/A for BCM systems >SPort - Port Number wrt Slice. N/A for BCM systems >SrcId - Source Id Number. N/A for BCM systems >MacIdx - Mac index. N/A for BCM systems >MacSubPort - Mac sub port. N/A for BCM systems > > > > Name Ifindex Smod Unit HPort FPort NPort VPort Slice SPort SrcId > MacId MacSP > > > Eth1/1 1a00 1016255 0 -10 16324 >0 > Eth1/2 1a000200 1017255 4 -10 17344 >2 > Eth1/3 1a000400 1018255 8 -10 18364 >4 > Eth1/4 1a000600 1019255 12-10 19384 >6 > Eth1/5 1a000800 1012255 16-10 12243 >0 > Eth1/6 1a000a00 1013255 20-10 13263 >2 > Eth1/7 1a000c00 1014255 24-10 14283 >4 > Eth1/8 1a000e00 1015255 28-10 15303 >6 > Eth1/9 1a001000 108 255 32-10 8 162 >0 > Eth1/101a001200 109 255 36-10 9 182 >2 > Eth1/111a001400 1010255 40-10 10202 >4 > Eth1/121a001600 1011255 44-10 11222 >6 > Eth1/131a001800 104 255 48-10 4 8 1 >0 > Eth1/141a001a00 105 255 52-10 5 101 >2 > Eth1/151a001c00 106 255 56-10 6 121 >4 > Eth1/161a001e00 107 255 60-10 7 141 >6 > Eth1/171a002000 100 255 64-10 0 0 0 >0 > Eth1/181a002200 101 255 68-10 1 2 0 >2 > Eth1/191a002400 102 255 72-10 2 4 0 >4 > Eth1/201a002600 103 255 76-10 3 6 0 >6 > Eth1/211a002800 1060255 80-11 2040 > 140 > Eth1/221a002a00 1061255 84-11 2142 > 142 > Eth1/231a002c00 1062255 88-11 2244 > 144 > Eth1/241a002e00 1063255 92-11 2346 > 146 > Eth1/251a003000 1056255 96-11 1632 > 130 > Eth1/261a003200 1057255 100 -11 1734 > 132 > Eth1/271a003400 1058255 104 -11 1836 > 134 > Eth1/281a003600 1059255 108 -11 1938 > 136 > Eth1/291a003800 1052255 112 -11 1224 > 120 > Eth1/301a003a00 1053255 116 -11 1326 > 122 > Eth1/311a003c00 1054255 120 -11 1428 > 124 > Eth1/321a003e00 1055255 124 -11 1530 > 126 > Eth1/331a004000 1048255 128 -11 8 16 > 110 > Eth1/341a004200 1049255 132 -11 9 18 > 112 > Eth1/3
Re: [c-nsp] N9K ASIC port grouping
--- Begin Message --- Not aware of a single document that will show that for all platforms. Best way is "show interface hardware-mappings". "Unit" and "Slice" are the main points of reference (unit == ASIC instance, slice == pipeline). E.g., 93180yc-ex: leaf1# sh int hard Legends: SMod - Source Mod. 0 is N/A Unit - Unit on which port resides. N/A for port channels HPort - Hardware Port Number or Hardware Trunk Id: HName - Hardware port name. None means N/A FPort - Fabric facing port number. 255 means N/A NPort - Front panel port number VPort - Virtual Port Number. -1 means N/A Slice - Slice Number. N/A for BCM systems SPort - Port Number wrt Slice. N/A for BCM systems SrcId - Source Id Number. N/A for BCM systems MacIdx - Mac index. N/A for BCM systems MacSubPort - Mac sub port. N/A for BCM systems Name Ifindex Smod Unit HPort FPort NPort VPort Slice SPort SrcId MacId MacSP Eth1/1 1a00 1016255 0 -10 16324 0 Eth1/2 1a000200 1017255 4 -10 17344 2 Eth1/3 1a000400 1018255 8 -10 18364 4 Eth1/4 1a000600 1019255 12-10 19384 6 Eth1/5 1a000800 1012255 16-10 12243 0 Eth1/6 1a000a00 1013255 20-10 13263 2 Eth1/7 1a000c00 1014255 24-10 14283 4 Eth1/8 1a000e00 1015255 28-10 15303 6 Eth1/9 1a001000 108 255 32-10 8 162 0 Eth1/101a001200 109 255 36-10 9 182 2 Eth1/111a001400 1010255 40-10 10202 4 Eth1/121a001600 1011255 44-10 11222 6 Eth1/131a001800 104 255 48-10 4 8 1 0 Eth1/141a001a00 105 255 52-10 5 101 2 Eth1/151a001c00 106 255 56-10 6 121 4 Eth1/161a001e00 107 255 60-10 7 141 6 Eth1/171a002000 100 255 64-10 0 0 0 0 Eth1/181a002200 101 255 68-10 1 2 0 2 Eth1/191a002400 102 255 72-10 2 4 0 4 Eth1/201a002600 103 255 76-10 3 6 0 6 Eth1/211a002800 1060255 80-11 2040140 Eth1/221a002a00 1061255 84-11 2142142 Eth1/231a002c00 1062255 88-11 2244144 Eth1/241a002e00 1063255 92-11 2346146 Eth1/251a003000 1056255 96-11 1632130 Eth1/261a003200 1057255 100 -11 1734132 Eth1/271a003400 1058255 104 -11 1836134 Eth1/281a003600 1059255 108 -11 1938136 Eth1/291a003800 1052255 112 -11 1224120 Eth1/301a003a00 1053255 116 -11 1326122 Eth1/311a003c00 1054255 120 -11 1428124 Eth1/321a003e00 1055255 124 -11 1530126 Eth1/331a004000 1048255 128 -11 8 16110 Eth1/341a004200 1049255 132 -11 9 18112 Eth1/351a004400 1050255 136 -11 1020114 Eth1/361a004600 1051255 140 -11 1122116 Eth1/371a004800 1044255 144 -11 4 8 100 Eth1/381a004a00 1045255 148 -11 5 10102 Eth1/391a004c00 1046255 152 -11 6 12104 Eth1/401a004e00 1047255 156 -11 7 14106 Eth1/411a005000 1076255 160 -11 3664170 Eth1/421a005200 1077255 164 -11 3766172 Eth1/431a005400 1078255 168 -11 3868174 Eth1/441a005600 1079255 172 -11 3970176 Eth1/451a005800 1036255 176 -10 36648 0 Eth1/461a005a00 1037255 180 -10 37668 2 Eth1/471a005c00 1038255 184 -10 38688 4 Eth1/481a005e00 1039255 188 -10 39708
Re: [c-nsp] Nexus 3548 and VDCs
Hi Mike, please see inline below: At 06:06 AM 2/25/2018 Sunday, Mike Hammett gushed: We recently upgraded our Nexus 3548 from version 6.0(2)A1(1d) to version 7.0(3)I7(3). We can get in from one of the trunk ports feeding the switch, but there is no traffic passing through to other ports. I'm not fully versed on the process as I wasn't the one doing it, just putting in keyboard time looking for resolutions. 1) Cisco doesn't seem to list any documentation for NX-OS 7 on the 3548, yet 7 is the only available software download on their support site. Some of the links take me to the 3600. No idea if that's correct. N3548 is integrated into the "Nexus 3000" doc set for 7.x. The 7.x train is now a converged software release train with a single "nxos" image for all n9k and n3k, including 3548. One exception is the 3600 series (broadcom jericho+ based), the plan is to integrate that platform later this year. Definitely don't use the 3600 docs for 3548. The image management and boot model is a bit different in 7.x, there is no kickstart for example just one 'nxos' image. The process of converting/upgrading from 6.x to 7.x can be a bit tricky, if not performed according to the documentation here: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus3000/sw/upgrade/7_x/b_Cisco_Nexus_3500_Series_NX-OS_Software_Upgrade_and_Downgrade_Guide_Release_7x/b_Cisco_Nexus_3500_Series_NX-OS_Software_Upgrade_and_Downgrade_Guide_Release_7x_chapter_010.html For one thing you mentioned you went from 6.0(2)A1(1d) to version 7.0(3)I7(3). Not sure how you did that, but the doc above does state "An upgrade to Cisco NX-OS Release 7.0(3)I7(2) is supported only from Cisco NX-OS Release 6.0(2)A8(7b) or higher releases." In any case, suggest you engage TAC to help recover this switch if you have not done so already. 2) It looks like along the way, we got a vdc section added to our config, but the only instance of vdc I can find applies to 7000 series switches. Is that supposed to be there? Should I just follow the 7000's config for VDCs? This is expected, all nexus in 7.x will have a 'default VDC' config section. Only n7k allows you to configure additional VDCs. There shouldn't be much if any config modification required in the VDC config. Hope that helps, Tim - Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing & Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 93108TC-EX Breakout Support
Hi Nick, please see inline below: At 02:47 PM 1/26/2017 Thursday, Nick Cutting quipped: This is the second generation 10 gig copper leaf switch with 100 gig uplinks. The first generation did not support 40 gig x 10 SFP+ breakouts on the uplinks. I believe this Generation 2 version does - (you can run the 100's at 40, and the 40's should support breakouts) I have looked at the switch documentation - which points to the 9k breakout document - which DOES not include the generation 2 EX switches. So I cannot find any doc that says it is supported - including the cisco live PDF's. Any insight would be greatly appreciated. It is supported: 93108tc-ex-1# sh mod 1 | eg TC 154 48x10GT + 6x40G/100G Ethernet Module N9K-C93108TC-EX active 93108tc-ex-1# sh ver | eg NXOS: NXOS: version 7.0(3)I5(1) 93108tc-ex-1# sh int e1/49-54 cap | eg -i ^eth|break|speed Ethernet1/49 Speed: 1000,1,25000,4,5,10 Breakout capable: yes Ethernet1/50 Speed: 1000,1,25000,4,5,10 Breakout capable: yes Ethernet1/51 Speed: 1000,1,25000,4,5,10 Breakout capable: yes Ethernet1/52 Speed: 1000,1,25000,4,5,10 Breakout capable: yes Ethernet1/53 Speed: 1000,1,25000,4,5,10 Breakout capable: yes Ethernet1/54 Speed: 1000,1,25000,4,5,10 Breakout capable: yes 93108tc-ex-1# You can find that (buried) in the NXOS release notes here: http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/release/notes/70342_nxos_rn.html Search for "breakout cable". Hope that helps, Tim Nick ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing & Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Multicast within VLAN on Nexus7K over vPC
Any of #1,#2,#3 will work. Assuming you don't want/need this multicast traffic routed, and assuming the receivers are all sending IGMP joins, #1 is the best option: configure a snooping querier under 'vlan 100 config' on both VPC peers. #2 is the next best option, or of course required if you want to also L3 multicast route this traffic. Hope that helps, Tim At 03:27 AM 10/20/2016 Thursday, Yham asserted: Hi All, I have two cisco nexus 7K as core switches and two cisco 4500 as distribution/access switches. Nexus switches have vPC with each downstream 4500 switch and there is no connection between 4500 switch. Vlan100 exist on all four switches and all devices part of this vlan are connected to 4500 switches. I believe this is pretty standard design. Though vlan 100 has regular users and services that communicate over unicast but there are some devices that need to send and receive multicast. Both Multicast sender and receivers are in same vlan but receivers are spread across both 4500 switches. In diagram (no link below), receivers connected to switch 4K-1 (where source is connected) can receive the multicast stream but receivers connected to 4K-2 don't see anything. I believe its expected behavior due to IGMP snooping enabled on switches by default but i am trying to figure out how to make receivers on other switch able to get multicast stream. I did some research and found different ways but unfortunately i don't have non-production devices to test which one actually works. Here what i found 1) configuring IGMP querier for vlan 100 on all four switches 2) only enable 'ip pim sparse-mode' under SVIs (interface vlan100) at N7K-01 & N7K-02 3) disable IGMP snooping for vlan 100 on all four switches (which i don't think a right solution) Topology Diagram https://s22.postimg.org/3tsnta4s1/topology.png Your any help will be highly appreciated. Thanks YH ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing & Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Sup2 (Not Sup2T) on a 6513 (NON-E)
At 02:02 PM 9/9/2016 Friday, Nick Hilliard asserted: Tim Stevenson wrote: > which is 8 x 1G > connections to 8 x 8:1 oversubscribed port ASICs. the easiest way to think of a 6148 is that it's like 8 individual 1G ethernet hubs connected into a 1G switch. Now, now - a hub would flood everything to all other ports. ;) But yes, this card is not one of the architectural high points in the c6k's history... Tim Once you visualise it in these terms, you can immediately see that performance is going to be awful. The best thing to do with these boxes is retire them aggressively. It's generally cheaper to swap them out with 1U boxes than it is to pay the power (and consequently cooling) bill, if you look at it over a lifetime of several years. Nick Tim Stevenson, tstev...@cisco.com Routing & Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Sup2 (Not Sup2T) on a 6513 (NON-E)
Hi Nick, please see inline below: At 12:32 PM 9/9/2016 Friday, Nick Cutting asserted: Good afternoon Lords of the Layers, Anyone remember far back enough to answer two questions on the SUP2 supervisor on an original (NON-E) 6513 chassis? It seems the online cisco documentation doesn't go further back than the SUP 32 - it's very hard to find a datasheet for this. Mod Slot Ports Module-Type Model Sub Status --- - - --- --- 1 12 1000BaseX Supervisor WS-X6K-SUP2-2GE yes ok The layer 2 is CatOS and the layer 3 is IOS. A client has a couple of these switches I am trying to phase out, and was wondering two things, throughput related. The layer 2 switching engine trunks all traffic destined to be routed on the switch, up to an internal port on the SUP known as 15/1 -> then over to the layer 3 IOS to be routed on the SVI. No, not really. 15/1 is for traffic *destined* to the router, not traffic the router will L3 switch. That's done more or less exactly the same as in more current sups, ie, hardware-based FIB lookup. Spanning tree has a value of 4 for the cost of this link - and in spanningtree IEEE - this is 1 gig. Port Vlan Port-StateCost Prio Portfast Channel_id 15/1 11 forwarding4 32 enabled 0 Does this mean that anything that is routed - maxes out at 1 gig on this platform? Or is the spanning tree value here arbitrary - and the backplane faster than this? - I thought the backplane of the SUP 2 was 32 gig - is this for switching and routing - or just switching? Again, this is just the inband port. It's treated on the switch side as 'just another port in STP'. It looks more or less like a router on a stick - but with the very important distinction mentioned above, that transit traffic is h/w switched, not passed over this interface. Also when configuring etherchannel on the CatOS switching engine - it mentions a warning message about maximum speed being 1 gig - I imagine this is just talking about a single flow - and multiple flows will be load shared as normal? This is a totally different problem space, which as you mention in your follow up message is related to the 6148-GE-TX LC you are using. I don't think I'm allowed to talk about it in quite the same language that Gert does ;) but this card is heavily oversubscribed, and has this particular limitation because of the internal architecture, which is 8 x 1G connections to 8 x 8:1 oversubscribed port ASICs. So it is actually talking about the TOTAL port-channel bandwidth and not per flow or anything, but the limit is 1G *per port group* not per channel. Couple examples: 1. if you put 1 port in each port group in the channel then you could get 8G thruput theoretically, assuming no other ports in any of the port groups were passing traffic 2. if you put all 8 ports in one port group, you'd only get 1G of thruput Hope that helps, Tim Any insight would be greatly appreciated. Thank you, Nick Cutting ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing & Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FW: N7K F2e Module
When F2E is mixed with M, then F2E ports operate as L2 only, but in this case he is unable to configure the *M* ports with anything other than "switchport host". That's just wrong. Probably the first step is to get on decent code, and see if the issue remains. Ie, 6.2.12 or 6.2.14. Tim At 09:49 AM 9/2/2015 Wednesday, Sandor Rozsa asserted: I dag this issue and found out that if you mix M1 with f2e than on the f2e you'll have only l2 features. You can try by creating an f2e only vdc and see if the features are available. sandor On Wed, Sep 2, 2015 at 12:39 PM, Mohammad Khalil <eng_m...@hotmail.com> wrote: > Please check below > > sh vdc > Switchwide mode is m1 f1 m1xl f2 m2xl f2e > > vdc_id vdc_name state mac > typelc > -- - -- > - -- > 1 JCBank_Core1_DR active > 64:a0:e7:3f:94:41 > Ethernetm1 m1xl m2xl f2e > > sh vdc feature-set > vdc JCBank_Core1_DR allowed feature-sets: > ethernet > > SW(config-if)# interface Ethernet2/19 > SW(config-if)# switchport ? > host Set port host > > sh mod > Mod Ports Module-Type Model Status > --- - --- -- > -- > 148 10/100/1000 Mbps Ethernet XL Module N7K-M148GT-11L ok > 232 10 Gbps Ethernet XL Module N7K-M132XP-12L ok > 50 Supervisor Module-1XN7K-SUP1 active * > 60 Supervisor Module-1XN7K-SUP1 > ha-standby > 10 48 1/10 Gbps Ethernet Module N7K-F248XP-25E ok > > Mod Sw Hw > --- -- -- > 16.2(2a) 1.2 > 26.2(2a) 1.3 > 56.2(2a) 2.3 > 66.2(2a) 2.3 > 10 6.2(2a) 1.2 > > Thanks in advance > > BR, > Mohammad > Date: Wed, 2 Sep 2015 11:06:57 +0200 > Subject: Re: [c-nsp] N7K F2e Module > From: rozsa.sandor.gy...@gmail.com > To: eng_m...@hotmail.com > > Hi, > What is the vdc type (limit resources) you are using? I recall I had > something similar when missconfiguring the vdc type, but in my case I had > only f2e cards, so the workaround was easy, just modify the module type to > f2e. > > http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus7000/sw/vdc/command/reference/vdc_cmd_ref/vdc_cmds.html > > I hope my comment helped you: > sandor > On Wed, Sep 2, 2015 at 10:52 AM, Mohammad Khalil <eng_m...@hotmail.com> > wrote: > Hi all > > I have Cisco N7K with 6.2.2a Image > > I brought F2e module to be installed on my system and I have already M1xl > (30 ports fiber module ) already in place > > After installing the F2e module , most of the ports on the M1 module > (which were configured as trunk ports) shows the I cannot configure the > ports except for host mode > > > > Switchport mode host > > only > > > > Anyone faced such a case? > > > > Thanks in advance > > > > BR, > > Mohammad > > > > ___ > > cisco-nsp mailing list cisco-nsp@puck.nether.net > > https://puck.nether.net/mailman/listinfo/cisco-nsp > > archive at http://puck.nether.net/pipermail/cisco-nsp/ > > > > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ > ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing & Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Anycast GW for L2 subnet
At 01:25 AM 8/7/2015 Friday, Adam Vitkovsky quipped: ...snip... You could do that but the idea with anycast HSRP is that all participating HSRP routers equally distribute the L3 switching load. Yes I agree but in my case the DC would span across two geographic locations and so would the VLANs. So I was thinking I could enforce VMs in either location to primarily use local GWs as the closest exit from the VLAN. So kind of L2 routing or anycasting of the GWs. When I do the same in L3 i.e. anycast the VLAN prefix I should get a nice split and no tromboning of traffic between the locations. ...snip... Yes, this will work from an Anycast HSRP standpoint, ie, if you have two sites, two AC routers at each, leaf switches in each site would only vector gwy traffic to the 2 local (closest) AC routers. A couple caveats: - in theory you could have more sites, each with an AC pair, but we only support 4 AC gwys today - FP still builds the MD trees as it always has, ie you avoid tromboning of unicast, but multidestination (bc/mc/uuc) still has the potential to 'take the scenic route' through the other site, depending on the topology of each tree We are actually working on solutions to both those limitations but I am speaking only of what we have today. Hope that helps, Tim Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Anycast GW for L2 subnet
Hi Adam, please see inline below: At 03:58 PM 8/4/2015 Tuesday, Adam Vitkovsky quipped: Has anyone played with Anycast HSRP with fabric path please? Just would like to confirm I understand it correctly. So ISIS calculates best path to the anycast switch ID advertising the HSRP MAC There are no MAC advertisements in FP, routing is based on switch IDs (SID). and since I can manipulate metrics on links between spine and leaf switches I should be able to dictate which leaf switches should be using which GWs right? You could do that but the idea with anycast HSRP is that all participating HSRP routers equally distribute the L3 switching load. Because only paths to anycast switch ID with equal costs are considered for multipathing right? (i.e. thereâs no unequal cost load sharing correct?) Correct, it is ECMP only. The model is that all anycast HSRP routers have their own unique SID but also an emulated SID shared among them all. All advertise that ESID, and any FP switch with equal path cost to 2 or more of those will load balance traffic destined to the HSRP MAC among them. Typical topology is spine/leaf but any topology will work. Note that only the control-plane Active router is the one that responds to ARP sources HSRP hellos with the HSRP MAC (using the ESID as the source SID in FP frames). See section 10 here for a bit more: http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white_paper_c11-687554.html Hope that helps, Tim Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] N7k PIM Anycast RP - Do we still need MSDP to sync RPs?
This is RFC 4610. Tim At 06:41 AM 3/5/2015 Thursday, Phil Mayers murmered: On 05/03/15 10:00, Gert Doering wrote: Hi, On Wed, Mar 04, 2015 at 03:19:09PM -0800, Tim Stevenson wrote: You do not need MSDP under this configuration, Anycast w/PIM Anycast w/MSDP are two different ways to do basically do the same thing. So, pure curiousity: I know how Anycast w/MSDP works, but with classic IOS, there is no Anycast w/PIM - how is this done, protocol-wise? The PIM RPs forward the PIM register to the other PIM RPs, if it didn't come *from* one of those RPs. They do it over real IPs rather than the anycast IP, of course. It's been around on JunOS for ages, and works great. Never tried it on supporting Cisco boxen, but I imagine it works just the same. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] N7k PIM Anycast RP - Do we still need MSDP to sync RPs?
At 03:13 PM 3/4/2015 Wednesday, linux.ya...@gmail.com murmered: Dear all, I have a square of 4 x N7k running PIM Anycast RP feature. Do i need to run MSDP feature like traditionnal RPs design or does NX(OS) Anycast feature already take care of RPs sync? You do not need MSDP under this configuration, Anycast w/PIM Anycast w/MSDP are two different ways to do basically do the same thing. Tim Cheers, Manu Chao ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] PBR Limits for Nexus 7k
Hi Brian, please see inline below: At 09:06 AM 2/3/2015 Tuesday, Brian Christopher Raaen quipped: I was doing some research and found the Nexus listed a limit of 23 entries for PBR. This is a limit on number of PBR route-map sequences. Each sequence can have a match statement pointing to an ACL of arbitrary size. I have some situations that require source based routing for more than that many pairings(more like 200-300). This limitation would essentially restrict you to 23 unique sets of next-hops (ie, each sequence can set 1 or more next-hops) for each set of match criteria (ACL). Let me know if you have any questions. Thanks, Tim Does this mean I will need to look for a solution other than a Nexus 7k or am I misunderstanding what this limit means? The datasheet I found it here http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/verified_scalability/b_Cisco_Nexus_7000_Series_NX-OS_Verified_Scalability_Guide.html#reference_DF4FD746AB1145838991CE0BDE9DE621 -- Brian Christopher Raaen Network Architect Zcorum ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] PBR Limits for Nexus 7k
At 10:17 AM 2/3/2015 Tuesday, Tim Stevenson quipped: Hi Brian, please see inline below: At 09:06 AM 2/3/2015 Tuesday, Brian Christopher Raaen quipped: I was doing some research and found the Nexus listed a limit of 23 entries for PBR. This is a limit on number of PBR route-map sequences. Each sequence can have a match statement pointing to an ACL of arbitrary size. I have some situations that require source based routing for more than that many pairings(more like 200-300). This limitation would essentially restrict you to 23 unique sets of next-hops (ie, each sequence can set 1 or more next-hops) for each set of match criteria (ACL). Let me clarify/reword that: This limitation would essentially restrict you to 23 unique sets of next-hops (ie, each sequence can set 1 or more next-hops), each with its own set of match criteria (ACL). Thanks, Tim Let me know if you have any questions. Thanks, Tim Does this mean I will need to look for a solution other than a Nexus 7k or am I misunderstanding what this limit means? The datasheet I found it here http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/verified_scalability/b_Cisco_Nexus_7000_Series_NX-OS_Verified_Scalability_Guide.html#reference_DF4FD746AB1145838991CE0BDE9DE621 -- Brian Christopher Raaen Network Architect Zcorum ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] newbie questions about jumbo packets
Hi Scott, please see inline below: At 07:09 AM 12/18/2014 Thursday, Scott Voll announced: I'm working on my Nexus 5k's. I need to adjust QoS for the first time. I'm following the white papers about input QoS, Network QoS, and Queuing QoS. I currently have a very simple network QoS with MTU of 9216 to allow jumbo frames on the pair of Nexus (storage). But now that I'm adjusting QoS I will be classifying my traffic, rather than just using the default. On n5k you use the network-qos (NQ) policy to adjust the MTU for all ports in the box. This does not change any other QoS behaviors, either internal or external to the box. For example, it does not cause the switch to mark/remark packets or classify/queue any differently than before. Note that on nexus in general, there is some level of 'baseline' classification for queuing purposes, typically at least a low/high priority queue to ensure network control traffic is prioritized. More granular configs are possible but out of the box it's pretty basic. What happens if I add jumbo frames to all my network classifications and it's not enabled everywhere (other switches connected(Ethernet))? If you enable jumbos on some switches but not all, obviously if a jumbo passes to a switch with it disabled, it will be dropped. I Believe on networks where the MTU is smaller it will just down size to the standard 1500. Is that correct? Most network gear does not fragment, or only fragments in software, so this is something to avoid. Hope that helps, Tim TIA Scott ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Engineer, Technical Marketing Data Center Switching Cisco - http://www.cisco.com +1(408)526-6759 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Some basic vPC questions (nexus 3000)
Hi Drew, please see inline below: At 07:50 AM 2/7/2014 Friday, Drew Weaver remarked: Greetings, We are purchasing two Nexus 3000 switches to aggregate some 48 port 1G switches and plan on using vPC for redundancy. http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps11541/white_paper_c11-685753.html When I was reading the vPC whitepaper (referenced above) for the Nexus 3000 it mentions two different types of vPC link: #1 The vPC peer keepalive link (whitepaper suggests that this traffic can run over the mgmt. interface) #2 The vPC peer link (needs to be at least two 10G ports in a port channel) My questions are What happens to the traffic if the vPC peer keep alive communication fails between the two vPC members? Nothing. You are of course alerted to that fact, but loss of bidirectional PKA communication does not impact data plane forwarding. Depending on the answer to the question above, is it possible to make the vPC peer keepalive link redundant? You can, yes, it can be a port-channel for example. Can you add more members to the vPC peer link port channel without disrupting traffic flow? There is potential for some disruption when you add/remove links from a port-channel, as hash buckets are shuffled around. Has anyone run into any unexpected caveats or amusing issues with vPC that they would like to share? I yield the floor... ;) Hope that helps, Tim Thanks and happy Friday! -Drew ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] fabricpath and vPC+
At 09:23 AM 11/13/2013 Wednesday, Arne Larsen / Region Nordjylland quipped: Hi all What is the correct setup when one is using fabricpath and vPC+ If 2 5k are direct connected with 2 10G fabricpath interfaces, should these 2 be a channel group or doesn't it really matter, because of the equal cost routing in isis If you really want VPC+, then yes it should be a port channel and configured as the VPC peer link. PL PKA are required in VPC+ just like in VPC. If you don't need VPC+, then yes you could just do parallel FP links and use ECMP for load sharing instead of port-channel, it's your choice. Hope that helps, Tim /Arne ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FHRP selection within Nexus
At 11:03 AM 11/13/2013 Wednesday, Gert Doering pronounced: Hi, On Thu, Nov 14, 2013 at 05:51:15AM +1100, Andrew Miehs wrote: How does that work? Have a read of http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572835-00_NX-OS_vPC_DG.pdf The bottom of page 24... HSRP The use of HSRP in the context of vPC does not require any special configuration. With vPC, only the active HSRP interface answers ARP requests, but both HSRP interfaces (active and standby) can forward traffic. If an ARP request coming from a server arrives on the secondary HSRP device, it is forwarded to the active HSRP device through the peer link. forwarding to the active HSRP device and only the active HSRP interface answers ARP request doesn't particularily sound active-active to me :-) You're confusing ARP and data packets. ARP is handled by the control plane active; data packets are handled by either VPC peer. Tim *This* is what happens on any 6500 that does HSRP on a SVI... GLBP is active-active in that both L3 routers will accept packets to the world, instead of L2-forwarding them to the other one inside the SVI. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FabricPath PIM-SM Interoperation
At 08:43 AM 9/17/2013 Tuesday, Yuri Bank noted: Hi Tim, Thanks for your response. I should have phrased my question differently. I'm not suggesting that FP IS-IS actually interacts with PIM-SM directly. My question is: Will the FP device, which is running PIM-SM on some SVI, send PIM Register/Join messages for learned multicast sources/receivers in the FP domain? Yes, absolutely. Imagine the following scenario. You have DeviceA and DeviceB. DeviceA is a FP switch, with an entire FP domain behind it. DeviceB is a legacy multilayer switch, or router, with many other L3 devices behind it. DeviceA has an FP-VLAN with a PIM-SM enabled SVI in it. DeviceB simply has a layer3 interface (in the above vlan) with PIM-SM enabled. Now the RP, sits somewhere behind DeviceB. Obviously, if there is a host somewhere in the FP network that starts sending multicast traffic, DeviceA will have to send out a PIM-Register message in order for the RP to learn of the source(otherwise, only hosts in the FP network would be able to listen to that traffic). Yes, the FP switch with the SVI will attract the multicast traffic (learned as an mrouter) and once that traffic hits the SVI all the usual first hop router mechanisms for L3 multicast kick in. Likewise, if there is a multicast receiver in FP network, a PIM-Join message would need to make its way to the RP. IGMP snooping intercepts the join at the edge switch to track state on the edge port. That join will be flooded on the mrouter ports (and also handed over to FP ISIS to advertise the edge switch's interest in that group in FP via GM-LSPs.) Once the join hits the SVI all the usual last hop router mechansims kick in. Tim Does this make sense? Again, I greatly appreciate your response. -Yuri Bank On Tue, Sep 17, 2013 at 9:46 AM, Tim Stevenson mailto:tstev...@cisco.comtstev...@cisco.com wrote: Hi Yuri, please see inline below: At 03:27 AM 9/17/2013 Tuesday, Yuri Bank clamored: Hello fellow networking enthusiasts, I have a few specific questions concerning this: 1. *Will a N7k/N5k translate GM-LSPs into PIM-Join messages, on a boundary link that has a remote peer running PIM-SM? No. FP ISIS GM-LSPs are an L2 concept here, they basically track group membership state at L2 inside the FP core. FP ISIS would not peer with a router running PIM-SM; you would typically have an SVI enabled for the FP VLAN, and that would have PIM enabled to integrate to L3. 2. *Same question as it relates to sources learned, that are within the FabricPath domain. Will PIM Register messages be generated, and sent to the RP in the PIM-SM domain? Not without an SVI/router interface running PIM connected into the FP domain. 3. *Any general information regarding this topic would be greatly appreciated, as I could find very little documentation about this. Think of FP GM-LSPs as an extension of IGMP snooping. All the L3 integration is basically the same in FP and classical Ethernet. Hope that helps, Tim Regards, YuriB ___ cisco-nsp mailing list mailto:cisco-nsp@puck.nether.netcisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, mailto:tstev...@cisco.comtstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.comhttp://www.cisco.com IP Phone: tel:408-526-6759408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FabricPath PIM-SM Interoperation
Hi Yuri, please see inline below: At 03:27 AM 9/17/2013 Tuesday, Yuri Bank clamored: Hello fellow networking enthusiasts, I have a few specific questions concerning this: 1. *Will a N7k/N5k translate GM-LSPs into PIM-Join messages, on a boundary link that has a remote peer running PIM-SM? No. FP ISIS GM-LSPs are an L2 concept here, they basically track group membership state at L2 inside the FP core. FP ISIS would not peer with a router running PIM-SM; you would typically have an SVI enabled for the FP VLAN, and that would have PIM enabled to integrate to L3. 2. *Same question as it relates to sources learned, that are within the FabricPath domain. Will PIM Register messages be generated, and sent to the RP in the PIM-SM domain? Not without an SVI/router interface running PIM connected into the FP domain. 3. *Any general information regarding this topic would be greatly appreciated, as I could find very little documentation about this. Think of FP GM-LSPs as an extension of IGMP snooping. All the L3 integration is basically the same in FP and classical Ethernet. Hope that helps, Tim Regards, YuriB ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] vs anycast hsrp
All the anycast routers need to use the same switch ID for the same bundle. ie, it's a 'shared' SID. Tim At 12:34 PM 9/7/2013 Saturday, Arne Larsen / Region Nordjylland proclaimed: Hi All Can someone give me a hint what I might be doing wrong. I'm trying to get anycast hsrp running but It's complaining about the switch-id. sh hsrp anycast Anycast bundle - 1 (IPv4) Admin Status: Up Oper Status: Down Reason: Invalid switch-id Cfged, : Anycast Switch ID 166 Bundle priority 100 Bundle State Initial Tracking object ID 1 (Current status is Up) VLAN range: 2 neighbor sh hsrp anycast Anycast bundle - 1 (IPv4) Admin Status: Up Oper Status: Down Reason: Invalid switch-id Cfged, : Anycast Switch ID 165 Bundle priority 100 Bundle State Initial Tracking object ID 1 (Current status is Up) VLAN range: 2 /Arne ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] fabricpath and qos
At 01:27 AM 8/7/2013 Wednesday, Arne Larsen / Region Nordjylland stated: Hi all Does someone know about fabricpath and qos implementation How it works, is it different for normal qos on nexus, and if where can I find some doc about it. FP uses the same qos fields as normal ethernet - 1p/COS and DSCP. There's a slide discussing that in the FP technology design session from Cisco Live. Tim /Arne ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Equivalent of ip multicast boundary on N7k for blocking data packets?
At 04:12 AM 6/5/2013 Wednesday, Phil Mayers remarked: On 03/06/13 21:44, Tim Stevenson wrote: At 01:08 PM 6/3/2013 Monday, Phil Mayers clamored: How can I accomplish the equivalent of the boundary on NX-OS 5.2 for N7k, given it lacks the command? Does one just use a normal ACL, and if so, are there any caveats to doing so e.g. does boundary do *other* things that a plain ACL would miss? In n7k, you must use a combination of control plane data plane filtering to get the equivalent functionality of multicast boundary. For data plane, it's nothing more than ip access-group with matches on multicast traffic. Just to say, this does all work, but it takes a few minutes to kick in - if you add the data-plane ACL then clear ip mroute, the routes just reappear. They die off a few minutes later - presumably something hardware-related. This is expected. 'clear ip mroute' on n7k clears the MRIB everything 'below' it (down to the hardware). The MRIB then immediately queries all its clients for multicast state - ie, PIM, IGMP, MSDP, which repopulates the MRIB (and thus the h/w). You can clear the state of each client with commands like clear ip pim route, clear ip igmp route etc. Can't say I'm loving the NX-OS CLI paradigm for this particular feature though - having to merge the unicast and multicast ACEs is a pain, As you can imagine, there was considerable debate about the pros/cons. Main reasons we went this way vs multicast boundary à la c6k: - clear separation of control plane vs data plane filtering - granular per-protocol filtering control - deterministic behavior across reboots (no order-dependent ACL merge) absent any templating/include other ACL functionality :o( You might be able to do some stuff with object groups here? Eg: tstevens-7010-2# sh ip access example IP access list example 10 permit udp any addrgroup multicast-ranges 20 permit ip any 1.1.1.1/32 30 deny ip any any tstevens-7010-2# sh object-group multicast-ranges IPv4 address object-group multicast-ranges 10 239.0.0.0/8 20 225.1.1.0/24 tstevens-7010-2# Note this is just a config 'convenience', TCAM consumption is based on the expansion of the ACEs in your object groups. Hope that helps, Tim Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Equivalent of ip multicast boundary on N7k for blocking data packets?
At 01:08 PM 6/3/2013 Monday, Phil Mayers clamored: How can I accomplish the equivalent of the boundary on NX-OS 5.2 for N7k, given it lacks the command? Does one just use a normal ACL, and if so, are there any caveats to doing so e.g. does boundary do *other* things that a plain ACL would miss? In n7k, you must use a combination of control plane data plane filtering to get the equivalent functionality of multicast boundary. For data plane, it's nothing more than ip access-group with matches on multicast traffic. For control plane, there is independent filtering control for each protocol, ie, PIM, IGMP, MSDP, etc. Hope that helps, Tim Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Fabricpath and L3 on the same line card
At 10:12 AM 3/21/2013 Thursday, Chris Evans announced: Okay great, that is what I thought.. Seems like a simple feature to miss. Do you know if there are any performance limitations with it? F2/E can generally do L2 L3 at equal rate. Like could internal ports be burned for routing efforts? Absolutely not. Tim It seems that many companies have problems with the TRILL header and can't do SVI natively like we can today. How these chips can handle MPLS, GRE, and other headers with no issues and not TRILL is interesting to me. On Thu, Mar 21, 2013 at 1:02 PM, Lustgraaf, Paul J [ITNET] gr...@iastate.edu wrote: Well, I'm doing it, so I guess it can. And F2 modules must be in a VDC by themselves, so no M1 could possibly be involved. Paul Lustgraafgr...@iastate.edu Change is inevitable. Progress is not. Network Engineer, Iowa State University IT Services 515-294-0324 -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto: cisco-nsp-boun...@puck.nether.net] On Behalf Of Chris Evans Sent: Thursday, March 21, 2013 11:57 AM To: cisco-nsp Subject: [c-nsp] Fabricpath and L3 on the same line card Can anyone tell me if Cisco F2/F2e line modules can run Fabricpath and L3 (SVI's) on the same line module. Is it line rate as well or does it proxy through an ASIC burning ports, etc. Is an M1 module required? Someone has told me it cannot, but I believe it can. Are there any limitations with it? ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7K NX-OS Upgrade
correctly updates the boot variables for slot 5: boot kickstart bootflash:/n7000-s1-kickstart.5.2.3a.bin sup-1 boot system bootflash:/n7000-s1-dk9.5.2.3a.bin sup-1 boot kickstart bootflash:/n7000-s1-kickstart.5.1.3.bin sup-2 The install all script assumes that if there is only one supervisor, it should be on slot 5. Above we can see that the boot system is missing for sup-2. In summary, is there any problem if I simply update the boot variables and reload ? May I end up with the supervisor running the new NX-OS release and the modules the old NX-OS release ? Regards, Antonio Soares, CCIE #18473 (RS/SP) amsoa...@netcabo.pt http://www.ccie18473.net http://www.ccie18473.net/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7K NX-OS Upgrade
/datacenter/sw/5_x/nx-os/upg ra de/gui de/b_Cisco_Nexus_7000_Series_NX-OS_Software_Upgrade_and_Downgrade_Gu id e__Rel ease_5.x_chapter_00.html#task_39E26688E1204F8CAAE876450A575E73 I had a bad experience in the past with the install all script. I was doing an upgrade to a 7010 with only 1 supervisor that was installed in slot 6. The install all script has a problem, may a bug, it only correctly updates the boot variables for slot 5: boot kickstart bootflash:/n7000-s1-kickstart.5.2.3a.bin sup-1 boot system bootflash:/n7000-s1-dk9.5.2.3a.bin sup-1 boot kickstart bootflash:/n7000-s1-kickstart.5.1.3.bin sup-2 The install all script assumes that if there is only one supervisor, it should be on slot 5. Above we can see that the boot system is missing for sup-2. In summary, is there any problem if I simply update the boot variables and reload ? May I end up with the supervisor running the new NX-OS release and the modules the old NX-OS release ? Regards, Antonio Soares, CCIE #18473 (RS/SP) amsoa...@netcabo.pt http://www.ccie18473.net http://www.ccie18473.net/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7K NX-OS Upgrade
At 09:36 AM 11/8/2012, Antonio Soares mused: Thanks Tim, I will follow that procedure, it's the one that makes perfect sense. The documentation should be more clear about this kind of situations, don't you think ? There are important things that are omitted between steps 10 and 11: You mean specific to also upgrading the DRAM? This particular procedure is not intended to cover also upgrading DRAM at the same time, that's not really something we assume you're doing every time you upgrade. BTW, Sukumar does make a good point about the install script - it will potentially make some changes to the config based on updated features, CoPP being a prominent example. An alternative in your case would be to just power off, upgrade DRAM, reboot, and then install all. Clearly that involves 2 reboots with a single sup. Tim http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/upgrade/gui de/b_Cisco_Nexus_7000_Series_NX-OS_Software_Upgrade_and_Downgrade_Guide__Rel ease_5.x_chapter_00.html#task_304731 Regards, Antonio Soares, CCIE #18473 (RS/SP) amsoa...@netcabo.pt http://www.ccie18473.net -Original Message- From: Tim Stevenson [mailto:tstev...@cisco.com] Sent: quinta-feira, 8 de Novembro de 2012 15:51 To: Antonio Soares; 'Dirk Woellhaf' Cc: 'cisco-nsp'; 'Charles Spurgeon' Subject: Re: [c-nsp] Nexus 7K NX-OS Upgrade At 07:18 AM 11/8/2012, Antonio Soares mused: I just have one SUP... You are talking about dual supervisors setup, right ? Ah. In that case, clearly, the box is going to go offline when you upgrade. You might want to consider buying another sup. IMO, there is no huge benefit in using the install all script in a single sup system - in the end, all it will do for you is a little sanity checking and maybe save you from fat fingering a bootstring. In your situation, I would copy over the new images you want; manually change the bootstrings save to startup; power off the box, yank the sup add the DRAM; and then power it all back on. Tim Regards, Antonio Soares, CCIE #18473 (RS/SP) amsoa...@netcabo.pt http://www.ccie18473.net -Original Message- From: Dirk Woellhaf [mailto:dirk.woell...@gmail.com] Sent: quinta-feira, 8 de Novembro de 2012 14:10 To: Antonio Soares Cc: Charles Spurgeon; cisco-nsp Subject: Re: [c-nsp] Nexus 7K NX-OS Upgrade Hi Antonio, You should be able to do the memory-upgrade without rebooting the box. I've never done it on my I own but I know a few which did without any problem. I believe they first upgraded the memory and then did the update! Dirk Sent from my iPhone On 08.11.2012, at 13:42, Antonio Soares amsoa...@netcabo.pt wrote: Thanks, I don't know if you noticed but somewhere in the thread the bug was mentioned and it is resolved in 5.1.5 and later. Bug CSCtn61286 - Boot variables are not set up correctly on Sup-2 after ISSU So in my case, it should not give me problems (5.2.3a to 5.2.7). But since I also need to upgrade the SUP1 RAM from 4G to 8G, I have no other option than doing the traditional upgrade. It's the only way to just send the box down 1 time: - update the boot variables - power off and upgrade the RAM - power on The install all script has another limitation: it won't let us to reboot when we chose to do it. This is what happened to me last time: + Switch will be reloaded for disruptive upgrade. Do you want to continue with the installation (y/n)? y Install is in progress, please wait. (..) A few minutes later: Finishing the upgrade, switch will reboot in 10 seconds. + I don't see how to upgrade the RAM and upgrade the NX-OS with the install script in just one shot... Regards, Antonio Soares, CCIE #18473 (RS/SP) amsoa...@netcabo.pt http://www.ccie18473.net -Original Message- From: Charles Spurgeon [mailto:c.spurg...@austin.utexas.edu] Sent: quinta-feira, 8 de Novembro de 2012 00:50 To: Antonio Soares Cc: 'Tóth András'; 'cisco-nsp' Subject: Re: [c-nsp] Nexus 7K NX-OS Upgrade While doing some more testing this aft I also removed the sup from slot 5 and did a disruptive single sup ISSU upgrade from 5.1(5) to 5.2(7) on the slot 6 sup without issues. -Charles On Tue, Nov 06, 2012 at 11:48:35PM +, Antonio Soares wrote: Great, I must confess that I searched a lot and I didn't find this bug. So I suppose the install all script will work well this time. I will come back to the list next week with the good news. I hope :) Thanks. Regards, Antonio Soares, CCIE #18473 (RS/SP) amsoa...@netcabo.pt http://www.ccie18473.net -Original Message- From: Tóth András [mailto:diosbej...@gmail.com] Sent: terça-feira, 6 de Novembro de 2012 23:35 To: Antonio Soares Cc: cisco-nsp Subject: Re: [c-nsp] Nexus 7K NX-OS Upgrade Hi Antonio, In general, doing a traditional upgrade (changing boot variables) will not update
Re: [c-nsp] Nexus 7K NX-OS Upgrade
At 06:05 AM 11/7/2012, Pete Templin mused: On 11/7/12 6:02 AM, Alexander Lim wrote: Do you know what caused the 3 secs blip? How can Cisco claims that it is non-disruptive then? Thanks for sharing. From what I've learned from others, the 'install all' unpacks the new files which run the processes, and then the processes are stopped/started. The blip aligns with the card that's actively being upgraded, as shown by the 'install all' or 'show install all status' if run on another login session/console. There are no software processes that affect hardware/data plane forwarding, any process can be statefully restarted without impacting data flow (in theory, ignoring bugs). We do claim it is non-disruptive and we can easily demonstrate that and have many times. It is unexpected and not per design to lose data traffic during an ISSU, provided you are ISSU'ing to/from supported releases (as per the ISSU matrix in the user documentation), all your data traffic is being hardware switched, and assuming no software defects (such as the specific one cited earlier in the thread). 2 cents, Tim pt ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7K NX-OS Upgrade
Hi Antonio, The difference between the two procedures is that install all will perform an in-service software upgrade, ie, this should be non disruptive to the data plane while the sups and all the modules will upgrade to the new version. Versus just changing the boot strings and rebooting, which is clearly disruptive to the entire system. In the end, both should result in all sups modules running the new release. Not sure what issues you ran into with ISSU, would at a minimum suggest you check the release notes to make sure the starting and target releases are compatible etc. Hope that helps, Tim At 03:04 PM 11/6/2012, Antonio Soares mused: Thanks, I appreciate your feedback. Since it is a lab environment, may I ask you to see what happens when you upgrade with the install all script and with the sup in slot 6 ? I had the problem when upgrading from 5.1.3 to 5.2.3a. Now I need to upgrade to 5.2.7 and I want to avoid the issue. Regards, Antonio Soares, CCIE #18473 (RS/SP) amsoa...@netcabo.pt http://www.ccie18473.net -Original Message- From: Charles Spurgeon [mailto:c.spurg...@austin.utexas.edu] Sent: terça-feira, 6 de Novembro de 2012 22:39 To: Antonio Soares Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 7K NX-OS Upgrade On Tue, Nov 06, 2012 at 10:38:46AM +, Antonio Soares wrote: Hello group, Anyone knows the difference between using the install all script or just update the boot system flash command when upgrading NX-OS on a Nexus 7K ? In summary, is there any problem if I simply update the boot variables and reload ? May I end up with the supervisor running the new NX-OS release and the modules the old NX-OS release ? I was just testing that this aft and it works fine in my lab tests, with the caveat that I have a dual-sup 7010. Manually configuring the boot strings and then typing reload resulted in sups and mods all coming up on the new code. -Charles Charles E. Spurgeon / UTnet UT Austin ITS / Networking c.spurg...@its.utexas.edu / 512.475.9265 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cat 6500 - uRPF - FIB TCAM
At 04:50 PM 8/14/2012, Brandon Applegate vociferated: Hello, I know this has been mentioned over the years here and there, but I don't know that I fully understand the exact behavior. I've always read 'urpf halves your tcam...'. It applies only to sup2. Sup720 later don't suffer this limitation. So this only applies to the interface on which it's configured, correct ? No. If you turn on uRPF check on sup2 on any interface, the available FIB TCAM for IP prefixes becomes 50% of what it is without uRPF check. So for example, in a single switch with the full routing table (using ipv4 for examples, and using simple even numbers not counting any built-in entries): uplink 1 - 400k routes uplink 2 - 400k routes customer interface 1 - 2 routes customer interface 2 - 2 routes So this is 400,004 entries. Adding (strict) urpf to the customer interfaces (not the uplinks) would make this 400,008 ? Well this whole discussion is moot, since you're probably not using sup2, especially if you have 400K prefixes. I guess I'm just unsure of if urpf is added to a single interface (even a customer interface with 1 or 2 prefixes) - does this have some 'global' effect ? You're probably confusing the sup2 limit described above, and the sup720 limitation that all interfaces w/uRPF check have to be in the same mode (strict or loose) and last configured wins. Tim Thanks in advance. -- Brandon Applegate - CCIE 10273 PGP Key fingerprint: 8779 B023 7637 CEC8 C5C6 4052 664D 7E08 3CBB 1739 SH1-0151. This is the serial number, of our orbital gun. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] WRR Confusion on 6748 blades
Hi John, please see inline below: At 08:58 AM 6/27/2012, John Neiberger pronounced: On Wed, Jun 27, 2012 at 8:24 AM, Janez Novak jnovak...@gmail.com wrote: 6748 can't do shaping. Would love to have them do that. So you must be experiencing drops somewhere else and not from WRR BW settings or WRED settings. They both kick in when congestion is happening (queues are filling up). For exaple linecard is oversubscribed etc Look at second bullet (http://www.cisco.com/en/US/docs/routers/7600/ios/12.2SR/configuration/guide/qos.html#wp1728810). Kind regards, Bostjan This is very confusing and I'm getting a lot of conflicting information. I've been told by three Cisco engineers that these queue bandwidths limits queue-limit and bandwidth values (ratios/weights) are *different* things. The queue-limit physically sizes the queue. It says how much of the total physical buffer on the port is set aside exclusively for each class (where class is based on DSCP or COS). Traffic from other classes can NEVER get access to the buffer set aside for another class, ie, there could be plenty of available buffer in other queues even as you're dropping traffic in one of the queues. The bandwidth ratios, on the other hand, determine how frequently each of those queues is serviced, ie, how often the scheduler will dequeue/transmit a frame from the queue. If there is nothing sitting in one queue, other queues can get access to that bandwidth, ie, bandwidth is not a hard limit, you can think of it as a minimum guarantee when there is congestion/contention. are fairly hard limits. That is in line with what we were experiencing because we were seeing output queue drops when the interface was not fully utilized. Increasing the queue bandwidth got rid of the output queue drops. What this should be doing is just causing us to service the queue more frequently. That could certainly reduce/eliminate drops in the event of congestion, but only if there is traffic in the other queues that is also contending for the bandwidth. In other words, if there is only one active queue (ie only one queue has traffic in it), then it can should get full unrestricted access to the entire link bandwidth. Can you confirm whether there's traffic in the other queues? For one particular application traversing this link, that resulted in a file transfer rate increase from 2.5 MB/s to 25 MB/s. That's a really huge difference and all we did was increase the allocated queue bandwidth. At no point was that link overutilized. We frequently see 'microburst' situations where the avg rate measured over 30sec etc is well under rate, but at some instantaneous moment there is a burst that exceeds line rate and can cause drops if the queue is not deep enough. Having a low bandwidth ratio, with traffic present in other queues, is another form of the queue not being deep enough, ie, the queue may have a lot of space but if packets are not dequeued frequently enough that queue can still fill drop. In fact, during our testing of that particular application, the link output never went above 350 Mbps. We used very large files so that the transfer would take a while and we'd get a good feel for what was happening. Doing nothing but increasing the queue bandwidth fixed the problem there and has fixed the same sort of issue elsewhere. This suggests to me that there is traffic in other queues contending for the available bandwidth, and that there's periodically instantaneous congestion. Alternatively you could try sizing this queue bigger and using the original bandwidth ratio. Or a combination of those two (tweaking both bandwidth queue-limit). Is there some issue with changing the bandwidth ratio on this queue (ie, are you seeing collateral damage)? Else, seems like you've solved the problem already ;) Hope that helps, Tim I'm still researching this and trying to get to the bottom of it. I think we're missing something important that would make this all make more sense. I appreciate everyone's help! John ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] WRR Confusion on 6748 blades
At 09:20 AM 6/27/2012, Phil Mayers pronounced: note that queues don't have bandwidth, they have size and weight. Yes, I've always disliked this term, bandwidth - I think weight would have been better, but that's water under the bridge. Tim Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] WRR Confusion on 6748 blades
Hi John, please see inline below: At 10:05 AM 6/27/2012, John Neiberger pronounced: ...snip... What this should be doing is just causing us to service the queue more frequently. That could certainly reduce/eliminate drops in the event of congestion, but only if there is traffic in the other queues that is also contending for the bandwidth. In other words, if there is only one active queue (ie only one queue has traffic in it), then it can should get full unrestricted access to the entire link bandwidth. Can you confirm whether there's traffic in the other queues? I'm not certain whether or not we have traffic in the other queues. In nearly all cases, the output drops are all in one queue with zero in the other queues. That seems to indicate that either all of our traffic is one queue or there just isn't a lot of traffic in the other queues. Unfortunately, there are no 'absolute' per queue counters, only per queue drop counters. So no easy way to determine if other queues are being utilized unless you just 'know' (based on your classification policies and known application mix) or those queues overflow drop. ...snip... This suggests to me that there is traffic in other queues contending for the available bandwidth, and that there's periodically instantaneous congestion. Alternatively you could try sizing this queue bigger and using the original bandwidth ratio. Or a combination of those two (tweaking both bandwidth queue-limit). Is there some issue with changing the bandwidth ratio on this queue (ie, are you seeing collateral damage)? Else, seems like you've solved the problem already ;) Nope, we don't have a problem with it. That's what we've been doing. We haven't really been adjusting the queue limit ratios, though. In most cases, we were just changing the bandwidth ratio weights. I'm looking at an interface right now where the 30-second weighted traffic rate has never gone above around 150 Mbps but I'm still seeing OQDs in one of the queues only. How do you think we should be interpreting that? In my opinion, it indicates that: 1. there is traffic in the other queues contending for the link bandwidth 2. there is instantaneous oversubscription that causes the problem queue to fill as it's not being serviced frequently enough and/or is inadequately sized 3. the other queues are sized/weighted appropriately to handle the amount of traffic that maps to them (ie, even under congestion scenarios, there is adequate buffer to hold enough packets to avoid drops) If #1 was not true, then I don't see how changing the bandwidth ratio would make any difference at all - if there is no traffic in the other queues, then the single remaining active queue would get full unrestricted access to the full bandwidth of the link and no queuing would be necessary in the first place. Supposing there is no traffic in the other queues - in that case, you could certainly still have oversubscription of the single queue and drops, but changing the weight should have no effect on that scenario at all (while changing the q-limit certainly could). 2 cents, Tim Hope that helps, Tim It helps a lot! thanks! John Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] WRR Confusion on 6748 blades
At 10:53 AM 6/27/2012, Peter Rathlev pronounced: On Wed, 2012-06-27 at 10:46 -0700, Tim Stevenson wrote: Unfortunately, there are no 'absolute' per queue counters, only per queue drop counters. Any chance of that ever showing up on the Cat6500 platform? :-D Not on 67xx cards; not sure if 69xx cards have capable hardware, have been off the c6k platform for quite a while. Tim Or as my lolcat would say: i can haz absolute counters plz kthxby -- Peter Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7k BGP Redistribute into EIGRP
Hi Skeeve, Out of curiousity, why are you running that (very early) engineering build of 6.0? I'm surprised TAC would even entertain debugging it until you move at least to the released 6.0(1) and we have a 6.0(3) maint release with numerous fixes on top of the initial feature release. Clearly route redistribution between BGP an IGP should work just fine, before anything else I'd get this box running customer-released code. 2 cents, Tim At 11:32 PM 4/26/2012, Lee pronounced: On 4/21/12, Skeeve Stevens skeeve+cisco...@eintellego.net wrote: Hey all, Got an odd problem with a Nexus 7010. We're taking in a default route from an upstream and we're wanting to redistribute the default route into EIGRP. The route is getting in just fine, but for some reason it isn't redistributing into EIGRP. We've tried OSFP as well, but no go. We're running Image version: 6.0(1) [build 6.0(0.66)] The TAC has been looking into it, but they've gone off to with a confused look on their face. If anyone has experienced anything like this and had a resolution, please let me know. Nexus 7010 running 5.2(3a) - redistribute default route from bgp to eigrp: ip prefix-list default_route_prefix permit 0.0.0.0/0 le 1 route-map bgp_default_to_eigrp permit 10 match ip address prefix-list default_route_prefix match as-number 65103 set metric 1000 255 1 1500 router eigrp 1 redistribute bgp 65103 route-map bgp_default_to_eigrp Regards, Lee *Skeeve Stevens, CEO* eintellego Pty Ltd ske...@eintellego.net ; www.eintellego.net http://www.eintellego.net.auhttp://www.eintellego.net.au Phone: 1300 753 383 ; Fax: (+612) 8572 9954 Cell +61 (0)414 753 383 ; skype://skeeve facebook.com/eintellego twitter.com/networkceoau ; www.linkedin.com/in/skeeve PO Box 7726, Baulkham Hills, NSW 1755 Australia The Experts Who The Experts Call Juniper - Cisco Brocade - IBM ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] NX-OS MAC-MOVE notifications, no vlan shown ??
Hi Jeff, CSCtw82129 introduces the change to include VLAN ID, 5.2(4) and 6.0(2) have the integration. Hope that helps, Tim At 10:12 AM 3/21/2012, Jeffrey G. Fitzwater proclaimed: I am running NX 5.2.1 on 7018 and have set logging level L2FM to 5 (notifications) in order to see the MAC-MOVES in logs. The problem I see is that VLAN associated with the MAC is not part of the error message as it is with 6500 IOS NX-OS %L2FM-4-L2FM_MAC_MOVE: Mac 0014.4f82.9a60 has moved from Po1 to Eth3/33 IOS %MAC_MOVE-SP-4-NOTIF: Host 0014.4f82.9a60 in vlan 128 is flapping between port Gi10/6 and port Po16 Is there a way to have the NEXUS show VLAN or would this be an Enhancement request. Thanks for any help; Jeff Fitzwater OIT Network Systems Princeton University ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7000 MPLS
At 01:37 AM 1/6/2012, Phil Mayers noted: On 01/06/2012 07:26 AM, Tim Stevenson wrote: Correct. No EoMPLS, no VPLS as yet, it's roadmapped. Tim, do you happen to know / can you tell us if the L2 MPLS features will be covered by the same feature license, or will people have to shell out for an MPLS2 feature license? Current plan of record is the L2VPN feature set will fall under the existing license. Of course, I am neither a PM nor the person with whom the buck stops, so this is perfectly subject to change prior to shipping those features. Thanks, Tim ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7000 MPLS
At 09:46 PM 1/5/2012, Justin M. Streiner stated: On Thu, 5 Jan 2012, Kris Price wrote: I see the Nexus 7000 does MPLS now (perhaps for some time?). Since Oct. 2011 (5.2.1 release). Is there anyone out there using MPLS on these and cares to comment about their experience? I'm particularly interested in RSVP, L3VPN support using OSPF as the PE/CE protocol, any scalability issues, possibly some interop w/ Juniper MX, and of course stability. All on and off list replies very much appreciated. :) Actually, I'd be interested in hearing about peoples' experience with this as well. The last time I looked, the L3 stuff was there, but EoMPLS was still off in the future. Correct. No EoMPLS, no VPLS as yet, it's roadmapped. Tim jms ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus debug output
Hi Bob, What are you pinging? I assume it's something out the mgmt0 interface. In this case, debug ip packet detail will not generate any output - many debugs are applicable to the inband interface only (as in this case). One alternative if you want to capture mgmt0 traffic is to use the ethanalyzer, eg something like: ethan local int mgmt capture-filter icmp limit-captured-frames 20 detail bootflash:foo etc. Or, if you're just trying to see how the debug logfile works, ping something connected to something other than mgmt0. Hope that helps, Tim At 12:46 PM 1/3/2012, Bob Sinclair stated: Hi, I am not able to see any debug output on my Nexus 7K. I am logged into the default VDC as netadmin. I have set up a debug logfile. But when, for example, I turn on 'debug ip packet' then try a ping I get nothing from 'show debug logfile mydebugs' I have searched command references, config guides and googled, to no avail. Any help would be most appreciated. C1-Default# sh debug Output forwarded to file mydebugs (size: 4194304 bytes) Debug level is set to Minor(1) default for new sessions logging level: 3 debug ip packet `end` C1-Default# sh debug logfile mydebugs C1-Default# ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus debug output
Hi Bob, Try using the debug-filter options - eg, if you're debugging ospf in vrf foo: debug-filter ip ospf vrf foo debug ip ospf adj The debug-filters are really pretty powerful, it helps you narrow down the debugs to just what you're looking for. Filters are available for most L3 protocols infrastructure. Hope that helps, Tim At 02:37 PM 1/3/2012, Bob Sinclair stated: Thanks Tim! That was it. Huge help! I can also now debug activity in a non-default VDC, but only for activity on default vrf interfaces. That seems like a real limitation, but I cannot find any vrf keywords on debug commands. I also tried changing routing-context to the non-default vrf, but that did not help. Any way to debug activity in the non-default vrfs? For example, debug ip ospf events? Thanks! -Original Message- From: Tim Stevenson [mailto:tstev...@cisco.commailto:tstev...@cisco.com] Sent: Tuesday, January 03, 2012 4:32 PM To: b...@bobsinclair.net; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus debug output Hi Bob, What are you pinging? I assume it's something out the mgmt0 interface. In this case, debug ip packet detail will not generate any output - many debugs are applicable to the inband interface only (as in this case). One alternative if you want to capture mgmt0 traffic is to use the ethanalyzer, eg something like: ethan local int mgmt capture-filter icmp limit-captured-frames 20 detail bootflash:foo etc. Or, if you're just trying to see how the debug logfile works, ping something connected to something other than mgmt0. Hope that helps, Tim At 12:46 PM 1/3/2012, Bob Sinclair stated: Hi, I am not able to see any debug output on my Nexus 7K. I am logged into the default VDC as netadmin. I have set up a debug logfile. But when, for example, I turn on 'debug ip packet' then try a ping I get nothing from 'show debug logfile mydebugs' I have searched command references, config guides and googled, to no avail. Any help would be most appreciated. C1-Default# sh debug Output forwarded to file mydebugs (size: 4194304 bytes) Debug level is set to Minor(1) default for new sessions logging level: 3 debug ip packet `end` C1-Default# sh debug logfile mydebugs C1-Default# ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.ne ther.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.comhttp://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. - No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.1901 / Virus Database: 2109/4720 - Release Date: 01/03/12 - No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.1901 / Virus Database: 2109/4720 - Release Date: 01/03/12 - No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.1901 / Virus Database: 2109/4720 - Release Date: 01/03/12 - No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.1901 / Virus Database: 2109/4720 - Release Date: 01/03/12 Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] N7K fabric behavior
Hi Tim, please see inline below: At 08:35 AM 12/8/2011, Tim Durack submitted: Trying to get something clear in my mind: N7K, 2x FAB-2, fabric redundancy, 220G capacity. N7K, 3x FAB-2, fabric redundancy, 330G capacity. ... N7K, 5x FAB-2, fabric redundancy, 550G capacity. Cisco recommend a minimum of 3 FAB-2 cards. Why? The origin of this recommendation is around M1 10G modules, which are 80G/slot cards. So 2 fabrics give you full b/w the 3rd gives you N+1 redundancy. This rule however doesn't apply to some of the higher performing cards (F1, F2). If I choose to ignore this recommendation, what is the impact? Is the fabric behavior different for M1/F1/F2 generation line cards? The key is that Fab 2 in the chassis does not change the b/w capabilities of modules with a local fab 1. M1 10G cards are still 80G/slot, F1 10G cards are still 230G/slot (with 5 fabrics). I understand the implications of over-subscription. If a 2x FAB-2 chassis is not over-subscribed due to the mix of 1G and 10G ports, will the fabric perform correctly? (I'm thinking yes, as this is a simple math equation. There is no other fabric-magic going on. Yes, you are correct - every card works and any port can talk to any port in the system even with a *single* fabric module. You (obviously) just won't get full bandwidth. Hope that helps, Tim Sales-Engineering is trying to convince me otherwise :-) Thanks for humoring me. -- Tim: ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus enabling pathcost method long - documentation
Mark, What documents are you referring to? In the N7K L2 config guide, the default port costs are shown appear to be correct. Please provide pointers to any incorrect docs and we can have them checked/fixed. http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/layer2/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_Layer_2_Switching_Configuration_Guide_Release_5.x_chapter7.html http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/layer2/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_Layer_2_Switching_Configuration_Guide_Release_5.x_chapter8.html#con_1706125 Tim At 11:24 AM 12/1/2011, Mark Mason murmered: In all the Nexus doc's for method long, I am seeing 20,000 as the 1x10Gb (10Gb total) cost and it should be 2,000. Furthermore, 2x10Gb (20Gb total) is 1,000 and in the doc's it shows as 10,000. Can someone from Cisco please clear up the design and configuration guides sooner than later? Documents should be corrected with the following for long: 10Gb - 2000 20Gb - 1000 40Gb - 500 Anyone else see this in the doc's and have questions about it? Mark NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus enabling pathcost method long - documentation
Ok thanks. Not sure who owns that document, it's definitely not part of the offical user documentation, but we'll track it down. Tim At 01:48 PM 12/1/2011, Mark Mason murmered: Tim- Below is the reference. Google has some other folks tying to that same page. Fix that one, and fix all I guess. http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572834-00_STDG_NX-OS_vPC_DG.pdfhttp://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572834-00_STDG_NX-OS_vPC_DG.pdf - this has the incorrect data Mark Mason | Network Engineer, Adv | JHA Communications Infrastructure Jack Henry Associates, Inc.® | 417.235.6652 x1520 | mma...@jackhenry.com Decisions without actions are pointless. Actions without decisions are reckless. -Original Message- From: Tim Stevenson [mailto:tstev...@cisco.commailto:tstev...@cisco.com] Sent: Thursday, December 01, 2011 3:26 PM To: Mark Mason; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus enabling pathcost method long - documentation Mark, What documents are you referring to? In the N7K L2 config guide, the default port costs are shown appear to be correct. Please provide pointers to any incorrect docs and we can have them checked/fixed. http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/layer2/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_Layer_2_Switching_Configuration_Guide_Release_5.x_chapter7.htmlhttp://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/layer2/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_Layer_2_Switching_Configuration_Guide_Release_5.x_chapter7.html http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/layer2/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_Layer_2_Switching_Configuration_Guide_Release_5.x_chapter8.html#con_1706125 Tim At 11:24 AM 12/1/2011, Mark Mason murmered: In all the Nexus doc's for method long, I am seeing 20,000 as the 1x10Gb (10Gb total) cost and it should be 2,000. Furthermore, 2x10Gb (20Gb total) is 1,000 and in the doc's it shows as 10,000. Can someone from Cisco please clear up the design and configuration guides sooner than later? Documents should be corrected with the following for long: 10Gb - 2000 20Gb - 1000 40Gb - 500 Anyone else see this in the doc's and have questions about it? Mark NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisc o-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether .net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/h ttp://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pip ermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.comhttp://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. NOTICE: This electronic mail message and any files transmitted with it are intended exclusively for the individual or entity to which it is addressed. The message, together with any attachment, may contain confidential and/or privileged information. Any unauthorized review, use, printing, saving, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email and delete all copies. Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7K - Multicast Question
Hi Antonio, Just would like to be perfectly clear what's going on here: sh ip mroute on NXOS shows the MRIB. The MRIB is specifically for multicast *routing* information. In the original scenario you described, where the join-group is configured on the upstream (RPF) interface and the 7k is not the PIM DR, no *mrouting* is required or being performed - the 7k is behaving as a host, ie, sending IGMP joins to pull the traffic. So you'll only see that state in sh ip igmp groups, not sh ip mroute. Note that you should still see the multicast traffic arriving on the n7k RPF interface (thru sh int). In the case of the loopback, mrouting *is* required, ie, in order to multicast route the multicast from the RPF interface to the loopback - so IGMP feeds the group information and OIF to the MRIB to enable the mrouting. This is all just a side effect of the modular architecture of the NXOS software - IGMP, the MRIB, PIM, MSDP etc etc are all independent processes and each maintains its own state based on what it needs to know; and each only tells other process(es) about that state when its actually necessary. Hope that helps, Tim At 04:56 PM 10/4/2011, Antonio Soares contended: Youre right, it works if the N7K is the DR is that segment. Now the mroute table shows what I was expecting: N7K12(config-if)# sh ip mroute IP Multicast Routing Table for VRF default (*, 232.0.0.0/8), uptime: 00:13:29, pim ip Incoming interface: Null, RPF nbr: 0.0.0.0 Outgoing interface list: (count: 0) (*, 239.1.2.3/32), uptime: 00:04:55, pim ip igmp Incoming interface: Ethernet1/27, RPF nbr: 10.12.2.2 Outgoing interface list: (count: 1) Ethernet1/27, uptime: 00:00:30, igmp, (RPF) (10.12.1.1/32, 239.1.2.3/32), uptime: 00:04:53, ip mrib pim Incoming interface: Ethernet1/27, RPF nbr: 10.12.2.2 Outgoing interface list: (count: 1) Ethernet1/27, uptime: 00:00:30, mrib, (RPF) N7K12(config-if)# Thank you for clarifying this difference between IOS and NXOS. Regards, Antonio Soares, CCIE #18473 (RS/SP) mailto:amsoa...@netcabo.ptamsoa...@netcabo.pt http://www.ccie18473.net From: Tim Stevenson [mailto:tstev...@cisco.com] Sent: domingo, 2 de Outubro de 2011 01:35 To: Antonio Soares; Phil Mayers; cisco-nsp@puck.nether.net Subject: RE: [c-nsp] Nexus 7K - Multicast Question Interface e1/27 on n7k12 appears to be connected to g2/2 on the 6500. I rest my case. ;) Tim At 12:40 PM 10/1/2011, Antonio Soares contended: Hello Tim, Very simple setup: N7K11===CAT6500===N7K12 The RP is the CAT6500. The relevant configs bellow: CAT6500 (The RP): ip multicast-routing ! interface Loopback0 ip address 3.3.3.3 255.255.255.255 ip pim sparse-mode ! interface GigabitEthernet2/1 ip address 10.12.1.2 255.255.255.0 ip pim sparse-mode ! interface GigabitEthernet2/2 ip address 10.12.2.2 255.255.255.0 ip pim sparse-mode no ip mroute-cache ! ip pim rp-address 3.3.3.3 ! N7K11 (The source): feature ospf feature pim interface Ethernet1/27 ip address 10.12.1.1/24 ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown ip pim rp-address 3.3.3.3 group-list 224.0.0.0/4 ip pim ssm range 232.0.0.0/8 N7K12 (The destination) feature ospf feature pim interface Ethernet1/27 ip address 10.12.2.1/24 ip router ospf 1 area 0.0.0.0 ip pim sparse-mode ip igmp join-group 239.1.2.3 no shutdown ip pim rp-address 3.3.3.3 group-list 224.0.0.0/4 ip pim ssm range 232.0.0.0/8 It works if the ip igmp join is moved to the loopback interface on the N7K12. Thanks. Regards, Antonio Soares, CCIE #18473 (RS/SP) mailto:amsoa...@netcabo.ptamsoa...@netcabo.pt http://www.ccie18473.net From: Tim Stevenson [ mailto:tstev...@cisco.com] Sent: sábado, 1 de Outubro de 2011 16:06 To: Antonio Soares; Phil Mayers; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 7K - Multicast Question Hi Antonio, Can you please describe the exact topology here? Which interfaces you're using, what they're connected to, and where is the RP? One thing to be aware of with NXOS is that things only show up in sh ip mroute if the client protocol/process (eg IGMP in this case) has reason to feed them there. sh ip mroute is basically looking at the MRIB view of the world and nothing else. WRT the ip igmp join-group command, it does two things: one is it causes that interface to send IGMP joins for that group out that interface as if it were a host. The other is that, if the router is PIM DR on that interface, it feeds the *G the OIF to the MRIB. It's only at that point you'll see the entry in sh ip mroute. Otherwise, you'll only see it in the IGMP group membership table, ie, sh ip igmp group. My guess here is you're not DR on this interface, so I assume there's another router on the segment that IS the DR. If you check the mrouting there I suspect you'll see the *G entry joined to the RPT (driven by the IGMP joins sent on that segment from the router w/the join-group command). WRT the loopback working
Re: [c-nsp] Nexus 7K - Multicast Question
Hi Antonio, Can you please describe the exact topology here? Which interfaces you're using, what they're connected to, and where is the RP? One thing to be aware of with NXOS is that things only show up in sh ip mroute if the client protocol/process (eg IGMP in this case) has reason to feed them there. sh ip mroute is basically looking at the MRIB view of the world and nothing else. WRT the ip igmp join-group command, it does two things: one is it causes that interface to send IGMP joins for that group out that interface as if it were a host. The other is that, if the router is PIM DR on that interface, it feeds the *G the OIF to the MRIB. It's only at that point you'll see the entry in sh ip mroute. Otherwise, you'll only see it in the IGMP group membership table, ie, sh ip igmp group. My guess here is you're not DR on this interface, so I assume there's another router on the segment that IS the DR. If you check the mrouting there I suspect you'll see the *G entry joined to the RPT (driven by the IGMP joins sent on that segment from the router w/the join-group command). WRT the loopback working, what you're seeing is that this router is now the only means by which to reach that 'network segment' (ie, it's obviously going to be DR on its own loopback) so it will report the *G OIF to the MRIB and then you'll see the entry in sh ip mroute. Hope that helps, Tim At 03:24 AM 10/1/2011, Antonio Soares contended: This is lab environment, I'm just testing basic multicast features with nexus. The command reference says the following: When you enter this command, the traffic generated is handled by the device CPU, not the hardware. http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/multicast/chttp://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/multicast/c ommand/reference/mcr_cmds_i.html#wp1230243 The ip igmp static-group was replaced by the command: ip igmp static-oif http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/multicast/chttp://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/multicast/c ommand/reference/mcr_cmds_i.html#wp1034808 When I have the igmp joing on the physical interface, the (*,G) entry is created then it disappears. But I see the (*,G) entry on the RP and I verified that the traffic is actually sent to the nexus. Thanks. Regards, Antonio Soares, CCIE #18473 (RS/SP) amsoa...@netcabo.pt http://www.ccie18473.nethttp://www.ccie18473.net -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.netmailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Phil Mayers Sent: sábado, 1 de Outubro de 2011 10:49 To: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 7K - Multicast Question On 10/01/2011 01:31 AM, Antonio Soares wrote: Hello group, Anyone knows why the ip igmp join-group does not work on a physical interface but it works fine on a loopback interface ? ip igmp join-group is a CPU command; it makes the CPU join the group and receive the packets. I imagine this might not work on a hardware platform, with an interface which will be processed in hardware. Are you sure you don't want ip igmp static-group? What's your use case? ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7K - Multicast Question
Interface e1/27 on n7k12 appears to be connected to g2/2 on the 6500. I rest my case. ;) Tim At 12:40 PM 10/1/2011, Antonio Soares contended: Hello Tim, Very simple setup: N7K11===CAT6500===N7K12 The RP is the CAT6500. The relevant configs bellow: CAT6500 (The RP): ip multicast-routing ! interface Loopback0 ip address 3.3.3.3 255.255.255.255 ip pim sparse-mode ! interface GigabitEthernet2/1 ip address 10.12.1.2 255.255.255.0 ip pim sparse-mode ! interface GigabitEthernet2/2 ip address 10.12.2.2 255.255.255.0 ip pim sparse-mode no ip mroute-cache ! ip pim rp-address 3.3.3.3 ! N7K11 (The source): feature ospf feature pim interface Ethernet1/27 ip address 10.12.1.1/24 ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown ip pim rp-address 3.3.3.3 group-list 224.0.0.0/4 ip pim ssm range 232.0.0.0/8 N7K12 (The destination) feature ospf feature pim interface Ethernet1/27 ip address 10.12.2.1/24 ip router ospf 1 area 0.0.0.0 ip pim sparse-mode ip igmp join-group 239.1.2.3 no shutdown ip pim rp-address 3.3.3.3 group-list 224.0.0.0/4 ip pim ssm range 232.0.0.0/8 It works if the ip igmp join is moved to the loopback interface on the N7K12. Thanks. Regards, Antonio Soares, CCIE #18473 (RS/SP) mailto:amsoa...@netcabo.ptamsoa...@netcabo.pt http://www.ccie18473.net From: Tim Stevenson [mailto:tstev...@cisco.com] Sent: sábado, 1 de Outubro de 2011 16:06 To: Antonio Soares; Phil Mayers; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 7K - Multicast Question Hi Antonio, Can you please describe the exact topology here? Which interfaces you're using, what they're connected to, and where is the RP? One thing to be aware of with NXOS is that things only show up in sh ip mroute if the client protocol/process (eg IGMP in this case) has reason to feed them there. sh ip mroute is basically looking at the MRIB view of the world and nothing else. WRT the ip igmp join-group command, it does two things: one is it causes that interface to send IGMP joins for that group out that interface as if it were a host. The other is that, if the router is PIM DR on that interface, it feeds the *G the OIF to the MRIB. It's only at that point you'll see the entry in sh ip mroute. Otherwise, you'll only see it in the IGMP group membership table, ie, sh ip igmp group. My guess here is you're not DR on this interface, so I assume there's another router on the segment that IS the DR. If you check the mrouting there I suspect you'll see the *G entry joined to the RPT (driven by the IGMP joins sent on that segment from the router w/the join-group command). WRT the loopback working, what you're seeing is that this router is now the only means by which to reach that 'network segment' (ie, it's obviously going to be DR on its own loopback) so it will report the *G OIF to the MRIB and then you'll see the entry in sh ip mroute. Hope that helps, Tim At 03:24 AM 10/1/2011, Antonio Soares contended: This is lab environment, I'm just testing basic multicast features with nexus. The command reference says the following: When you enter this command, the traffic generated is handled by the device CPU, not the hardware. http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/multicast/chttp://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/multicast/c ommand/reference/mcr_cmds_i.html#wp1230243 The ip igmp static-group was replaced by the command: ip igmp static-oif http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/multicast/chttp://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/multicast/c ommand/reference/mcr_cmds_i.html#wp1034808 When I have the igmp joing on the physical interface, the (*,G) entry is created then it disappears. But I see the (*,G) entry on the RP and I verified that the traffic is actually sent to the nexus. Thanks. Regards, Antonio Soares, CCIE #18473 (RS/SP) amsoa...@netcabo.pt http://www.ccie18473.nethttp://www.ccie18473.net -Original Message- From: cisco-nsp-boun...@puck.nether.net [ mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Phil Mayers Sent: sábado, 1 de Outubro de 2011 10:49 To: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Nexus 7K - Multicast Question On 10/01/2011 01:31 AM, Antonio Soares wrote: Hello group, Anyone knows why the ip igmp join-group does not work on a physical interface but it works fine on a loopback interface ? ip igmp join-group is a CPU command; it makes the CPU join the group and receive the packets. I imagine this might not work on a hardware platform, with an interface which will be processed in hardware. Are you sure you don't want ip igmp static-group? What's your use case? ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http
Re: [c-nsp] 6500 dbus usage
Hi Jon, please see inline below: At 07:23 AM 9/20/2011, Jon Marshall contended: Thanks Peter If they do have dbus connectors then why are they not supported with a sup32. The connection to the dbus is a control path only on 67xx cards, there is no data path there. 67xx cards always use the fabric for data plane forwarding. It's just a question of whether the lookup happens centrally via the dbus or locally via a DFC. Sup32 does not have any fabric connection, so there would be no way for traffic ingressing a 67xx card to reach the sup, or for sup traffic to egress a 67xx front panel port. I thought it was a physical connectivity issue but apparently not. So is it just a performance decision made by Cisco Don't worry, unlike most other things on this list seemingly, there's not a conspiracy here. :P Hope that helps, Tim ? Jon Subject: Re: [c-nsp] 6500 dbus usage From: pe...@rathlev.dk To: jms@hotmail.co.uk CC: cisco-nsp@puck.nether.net Date: Tue, 20 Sep 2011 16:15:38 +0200 On Tue, 2011-09-20 at 14:12 +0100, Jon Marshall wrote: My confusion is when i read on here that when the 67xx modules are running in CFC mode they send a portion of the packet (depending on compact/truncated mode) across the 32Gbps bus to the supervisor for a forwarding decision to be made. But if they don't have connectors to the 32Gbps bus then how do they access it ? They do have DBus connectors. Even the WS-X6708-10G-3C, even though the diagram doesn't show it. http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd80673385.html#wp9000639http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd80673385.html#wp9000639 As a further question, assuming they do send lookups across the shared bus then having classic linecards using the same bus could have a direct impact on the performance of the 67xx modules ? AFAIK it will not impact the performance of traffic staying on fabric-enabled cards. (I'm not aware of any changes the Sup2T might introduce here.) -- Peter ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Common uRPF setting on all interfaces
Hi Ross, This is a 'well-known' limitation of uRPF checking on sup720. It's documented here (3rd bullet): http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configuration/guide/secure.html#wp1099693 Hope that helps, Tim At 12:04 PM 7/25/2011, Ross Halliday commented: Hello list, We recently did a forklift upgrade of a 6509 from a SUP2 unit to a SUP720-3B box. At the same time I also plunked over a few VRFs which had been living on an external router due to lack of VRF support on the SUP2s. To my surprise one of the moved customers reported lack of Internet connectivity (VPN was fine - they collocate a firewall) at sites hanging off of the upgraded box. I determined that, though I thought I copied everything properly, an SVI's uRPF got messed up and was dropping packets from the Internet. In troubleshooting I added allow-default to the ip verify ... line on the SVI and it worked. Being connected to an internal VLAN that peers with other switches in that VPN (we're not MPLS yet) where all other ingress traffic is filtered I figured it was a redundant step so removed the line completely. Well, this afternoon I saw RANCID email me a list of changes from that box. Every single SVI that used to have some incantation of uRPF now have ip verify unicast source reachable-via rx allow-default allow-self-ping on them. Explains how the allow-default got removed in the first place; the next SVI I pasted in doesn't have that bit. Has anyone seen this before? I did a couple of quick searches but my Google-fu is letting me down. Is there some secret that only one possible stanza for uRPF is allowed on this box, unless the line isn't present? Running 12.2(33)SXI4a on SUP720-3B in a 6509. Thanks Ross ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus 7010 SVI issues
VLAN created interfaces assigned is not good enough. To confirm, at least one interface (access or trunk) assigned to that vlan is in the UP and STP forwarding state? You can do a sh spanning vlan X to check. Tim At 11:25 AM 7/9/2011, Renelson Panosky opined: Yes, the vlan are created on the layer 2 and have interfaces assign to them. Renelson On Sat, Jul 9, 2011 at 12:54 PM, Quinn Snyder snyd...@gmail.com wrote: depending on code version, i've seen the n7k not create the layer-2 vlan associated with the svi, even allowing you to place it on a trunk. can you confirm that the layer-2 vlan is in place and created? regards, q. -= sent via ipad. please excuse brevity, spelling, and grammar =- On Jul 9, 2011, at 8:52, Renelson Panosky panocisc...@gmail.com wrote: I have a couple nexus pod up and running so i just created two more SVI in my Nexus 7010 with the following configuratons. All my other SVIs are configured exactly the same way and all of them are UP UP but the two new one i just add. They are all added to all my trunks and all my trunks are UP UP. I do know on some devices in the IOS platform the SVI will not come up until you put a node on it (plug something in oe of the ports assign to that vlan.) but int he same token some the other SVIs have no nodes on them and they are UP UP and i can ping them. Any input would be greatly apprecisted interface Vlan2 no shutdown description XXX no ip redirects ip address 10.100.XX.XX/25 ip router eigrp 100 ip passive-interface eigrp 100 hsrp 2 preempt delay minimum 30 priority 110 ip 10.XXX.XX.XX ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Upgrading Software using TFTP server in a Nexus 7000
And if you're trying over mgmt0 that you specify the management vrf. Tim At 01:57 PM 6/12/2011, Chris Evans pronounced: Make sure that your source ping IP is the same as the tftp source IP? On Jun 12, 2011 4:33 PM, Renelson Panosky panocisc...@gmail.com wrote: I am trying to upgrade the IOS in a new Nexus 7000 that i am working on. I keep getting this crazy error: TFTP get operation failed:connection timed out. I can ping my TFTP server from the switch and i can ping the switch so i know i have two way communication. Have Anyone of you guys ran into this situation or any other machine? ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus N7K-M132XP-12 powerup
Hi Renelson, My first suspicion would be that these are in fact N7K-M132XP-12L cards (note the trailing L). The L version of this card is only supported from 5.1, I'd recommend 5.1(3). Otherwise, I'd open a TAC case, as support for the non-L version is there from day 1. Hope that helps, Tim At 01:39 PM 6/11/2011, Renelson Panosky pronounced: I am working on a N7K with two of the above cards. they're showing power down and everytime i tried to power them back up i got the following errors. switch(config)# 2011 Jun 12 01:22:36 switch %$ VDC-1 %$ %PLATFORM-2-PFM_MODULE_POWER_ON: Manual power-on of Module 7 from Command Line Interface 2011 Jun 12 01:22:39 switch %$ VDC-1 %$ %PLATFORM-2-MOD_PWRIDPROM_SW_CARD_ID_UNKNOWN: Module 7 failed to power up. (Unknown card. Could not get software-card-id) I am running Software version 5.0 (2a) and i moved one the card in slot 7 and still getting the same error. sho mod Mod Ports Module-Type Model Status --- - -- 10 Unknown Module powered-dn 248 10/100/1000 Mbps Ethernet Module N7K-M148GT-11 ok 348 10/100/1000 Mbps Ethernet Module N7K-M148GT-11 ok 50 Supervisor module-1X N7K-SUP1 active * 60 Supervisor module-1X N7K-SUP1 ha-standby 70 Unknown Module powered-dn Mod Power-Status Reason --- --- 1powered-dn Unknown card (Could not get software-card-id) 7powered-dn Unknown card (Could not get software-card-id) Mod Sw Hw --- -- -- 25.0(2a) 1.7 35.0(2a) 1.7 55.0(2a) 1.8 65.0(2a) 1.8 Mod MAC-Address(es) Serial-Num --- -- -- 250-3d-e5-b3-35-5c to 50-3d-e5-b3-35-90 JAF1502BDGH 350-3d-e5-b9-81-a0 to 50-3d-e5-b9-81-d4 JAF1502BDGF 510-8c-cf-1d-e5-d0 to 10-8c-cf-1d-e5-d8 JAF1453BKBK 68c-b6-4f-e8-ee-60 to 8c-b6-4f-e8-ee-68 JAF1453BKCF Mod Online Diag Status --- -- 2Pass 3Pass 5Pass 6Pass Xbar Ports Module-Type Model Status --- - -- 10 Fabric Module 1 N7K-C7010-FAB-1ok 20 Fabric Module 1 N7K-C7010-FAB-1ok 30 Fabric Module 1 N7K-C7010-FAB-1ok Xbar Sw Hw --- -- -- 1NA 1.1 2NA 1.1 3NA 1.1 Xbar MAC-Address(es) Serial-Num --- -- -- 1NA JAF1453BPHE 2NA JAF1453BPHH 3NA JAF1453BPAP ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] off-topic NMS Suggestion
I'll put a plug in for PRTG, which personally I've been using to run visual demos of some switching features for our product; but I've been quite impressed with the depth of capability (custom scripts with XML, full SNMP with custom MIB compilation, NFv5/v9 collection, auto device discovery sensor creation, and tons of other stuff). The map function, which allows you to overlay live data/graphs on a static image (network topology, geo map, etc) is pretty cool. They have an operational demo here: https://prtg.paessler.com/group.htm?id=0 Hope that helps, Tim At 06:41 PM 5/24/2011, Dan Letkeman mumbled: Intermapper has worked well for me for the past few years, easy to setup, not expensive, and has the ability to make a nice graphical map of all your devices any which way you please. Dan. On Tue, May 17, 2011 at 9:38 PM, omar parihuana omar.parihu...@gmail.com wrote: Hi List, Please could you suggest me a NMS for WAN/LAN? the WAN is a MPLS/VPN (300 remote offices) and the Switching is a campus LAN (aprox 1000 Network Devices) and three remote buildings (aprox Network 200 devices in each building). Before I tried Cisco Works but I faced some issues; HP Openview was difficult also. We need a easy web interface for monitoring and reporting (unfortunately no open source solutions are accepted). Thank you for your suggestions. Rgds. -- Omar E.P.T - Certified Networking Professionals make better Connections! ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] NX-OS 5.1 proxy l3
It gives you more proxy L3 forwarding capacity in a mixed module chassis, ie your M1 modules doing routing on behalf of your F1 modules. Tim At 06:52 PM 5/2/2011, Tim Durack mused: What does an increased number of layer-3 forwarders (16 to 128) do for me in NX-OS 5.1? -- Tim: ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] NX-OS 5.1 proxy l3
The number of SVIs is not a factor (or no more so than for any system) - the question is how much bandwidth/thruput do you need for inter-vlan routing for those SVIs. Each M1 module you add to the system adds up to 80G of proxy L3 (assuming 10G M1 cards, and assuming you allow all M1 ports to participate). Tim At 08:08 PM 5/2/2011, Tim Durack mused: Okay, so 1 proxy forwarder = 1 SVI? I guess I'm trying to figure out how many SVIs I can proxy-route with an F1/M1 mix. I can't find any numbers anywhere. On Mon, May 2, 2011 at 10:46 PM, Tim Stevenson tstev...@cisco.com wrote: It gives you more proxy L3 forwarding capacity in a mixed module chassis, ie your M1 modules doing routing on behalf of your F1 modules. Tim At 06:52 PM 5/2/2011, Tim Durack mused: What does an increased number of layer-3 forwarders (16 to 128) do for me in NX-OS 5.1? -- Tim: ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.comhttp://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. -- Tim: Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Understanding 10G line card oversubscription
It is targeted for the 5.2 release (internal name Delhi). Tim At 08:08 AM 3/31/2011, Peter Rathlev uttered: On Thu, 2011-03-31 at 08:09 -0500, Tony Varriale wrote: Phil, looks like Cisco is launching (has launched) their marketing for MPLS phase 1 on N7K. http://www.cisco.com/en/US/prod/switches/ps9441/ps9402/nexus_7000.html#~MPLS Anybody know what software version will support MPLS on N7k? I can't seem to see that from the above. -- Peter ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Fabricpath on Nexus
At 12:41 PM 3/28/2011, Asbjorn Hojmark - Lists uttered: We are considering deploying a pair of Nexus 7010 switches using fabricpath for L2 and HSRP for Layer 3. If it really is only two boxes, FabricPath provides *no* benefits, For one thing you could provide up to 256 10G links between two boxes, something you could not do with STP. only more complexity... FP config is certainly no more complex than STP. What makes you think it's complex? Have you ever configured it? Tim If you want to use FabricPath for anything useful today, you have to use Nexus 7000 access switches as well, and more than two aggregation switches. -A ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Fabricpath on Nexus
Please see inline below: At 06:43 PM 3/28/2011, Tony Varriale uttered: On 3/28/2011 3:09 PM, Tim Stevenson wrote: For one thing you could provide up to 256 10G links between two boxes, something you could not do with STP. Is this 16 links active per path? If so, what's the LACP game being played? Tim and/or Lincoln, I was hoping you could comment on a couple of other things. 1) The configuration guide recommends that all L2 FP gateways be configured as 8192 priority but do not specify them needing to be root. I'm guessing the docs want the gateways to be root. Correct. I'll follow up with the doc team. I think that would have some design implications (FHRP, as well as having to actively take root away from your existing bridge(s)). Would you comment on why this is? All ports need to be designated? I assumed it was an optimal traffic flow to/from the FP domain and the ability to proactively stop loops. Basically, the FP 'cloud' appears as a single unified root switch to any connected STP domains. Not only do the edge switches need to be root, but they should ideally use the same bridge priority as well. Clearly this is a transitional benefit, being able to connect any arbitrary STP domains to your FP network. The end-goal would be to have no STP bridges, just FP switches, and hosts connected directly to FP edge ports. We have the STP interop, and features like VPC+, to ease that transition and not make you rip replace your entire network before you get any benefit. 2) In the docs, it states that the FP network automatically presents a single bridge to all CE devices. I assume when you enable FP on the gateways, it plays a vPC peer switch game of the single bridge ID presentation. Or, something similar? It's basically just a static BID (c84c.75fa.6000). All edge ports must be designated. If an edge switch is not root we receive a superior BPDU, we block the port automatically. 3) In the docs it recommends not to use the vPC peer switch feature. Is this because of the multipath calc in the FP domain? Any implications from a STP role change here? Hm. Don't know offhand where that recommendation comes from, I can check into it. Maybe Lincoln knows. 4) QoS? FP requires new hardware (F1). That hardware is designed to be able to look at the appropriate offsets in order to make intelligent decisions based on COS/DSCP as well as the full L2/L3/L4 headers, regardless of whether FP encap is present or not. 5) PLVAN type feature? You can configure FP edge ports into PVLANs. A few guidelines here: http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/fabricpath/configuration/guide/fp_advanced.html#wp1928818 Hope that helps, Tim Thanks! tv ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] 6500/BGP/full route tables [even more] confusing...
Hi Jeff, Removing the DFC from a 65xx card letting the PFC do the work is HIGHLY preferable to letting traffic punt for your shortest mask prefixes, which are the ones that won't get into the h/w if you blow out the FIB TCAM. In order of preference: * get an XL DFC, list price on a DFC3CXL is like $15000 * remove the DFC from the card. You can do this easily on a 65xx card (note that with 67xx cards you'd need a CFC around to replace it with). 65xx without DFC will use the bus to get central lookups by the PFC, will use the fabric for sending data traffic to the egress module. * just blow out the TCAM randomly and punt to the CPU. This is not just least preferred, it's bordering on downright insane. 2 cents, Tim At 12:05 PM 3/26/2011, Jeff Kell quipped: On 3/26/2011 2:32 PM, Jon Lewis wrote: On Sat, 26 Mar 2011, Jeff Kell wrote: I intended to upgrade this to something suitable for full routing tables, and went for a Sup720/PFC3CXL. A million routes, right? Not really. The million routes thing is highly misleading. It at least depends on a whole lot of things. I'll settle for half. Just not 192K. Received another WS-X6516-GBIC but with a DFC3A. Powers up, but switches everything to PFC3A mode [192K routes]: If you're not doing that much traffic, is removing the DFC from the WS-X6516-GBIC an option? I don't know, that's why I'm asking, but if that's a viable option, will certainly give it a try. Jeff ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] 6500/BGP/full route tables [even more] confusing...
DFC is optional on CEF256 cards. CEF256 is just a family name. The dCEF under CEF mode indicates whether it's doing central or distributed (d = distributed, ie, DFC). sh module will clearly tell you whether a DFC is present or not, under the Sub-Module heading. Tim At 12:22 PM 3/26/2011, Andriy Bilous quipped: As far as I'm aware there is no DFC in 6516, it's a legacy card with a single 8Gb (duplex) connection to the fabric. Your output actually confirms that, saying CEF256. Are there any other modules in the chassis? On Sat, Mar 26, 2011 at 8:05 PM, Jeff Kell jeff-k...@utc.edu wrote: On 3/26/2011 2:32 PM, Jon Lewis wrote: On Sat, 26 Mar 2011, Jeff Kell wrote: I intended to upgrade this to something suitable for full routing tables, and went for a Sup720/PFC3CXL. A million routes, right? Not really. The million routes thing is highly misleading. It at least depends on a whole lot of things. I'll settle for half. Just not 192K. Received another WS-X6516-GBIC but with a DFC3A. Powers up, but switches everything to PFC3A mode [192K routes]: If you're not doing that much traffic, is removing the DFC from the WS-X6516-GBIC an option? I don't know, that's why I'm asking, but if that's a viable option, will certainly give it a try. Jeff ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] SPAN on 6500
Hi Chuck, You're correct, if you first configure the span dest as a .1q trunk, the SPANned traffic will go out vlan tagged (regardless of whether the original traffic actually had a vlan tag or not). You might want to check this (old but still relevant) doc on cco: http://bock-bock.cisco.com/~tstevens/White_Papers/virtual-span/virtual-span.pdf Hope that helps, Tim At 08:16 AM 1/13/2011, Church, Charles muttered: All, I'm running into some issues with SPAN session limitations on 6500 (SXI on a VSS pair). After reading this doc: http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configuhttp://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configu ration/guide/span.html I'm lead to believe that if I make the destination interface a trunk, a span source of say VLANs 10 and 20 will leave the destination port with those VLAN tags intact. This appears to match the 'encapsulation replicate' that is present on the 3560s. My end goal is to use 2 3560 switches off of the 6500s to distribute SPAN sessions to 4 separate entities. Switch A will get a SPAN session off of the 6500 consisting of VLAN groups X and Y. Switch B will get a SPAN session off of the 6500 consisting of VLAN groups X and Z. Switch A will span VLAN group X to a certain destination port, and group Y to another. Switch B will do a similar thing with VLAN groups X and Z. I'm assuming normal local SPAN. I think the relies on the SPAN off of the 6500 to keep the VLAN tags intact. Can anyone confirm if my assumption is correct? Thanks, Chuck ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] SPAN on 6500
Arg, I've done it again. Don't bother clicking, this link is internal to cisco. Sorry about that. The basic information you need though is in the user docs: http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_0/nx-os/system_management/configuration/guide/sm_span.html#wp1184503 Hope that helps, sorry for the confusion. Tim At 08:42 AM 1/13/2011, Tim Stevenson muttered: Hi Chuck, You're correct, if you first configure the span dest as a .1q trunk, the SPANned traffic will go out vlan tagged (regardless of whether the original traffic actually had a vlan tag or not). You might want to check this (old but still relevant) doc on cco: http://bock-bock.cisco.com/~tstevens/White_Papers/virtual-span/virtual-span.pdf Hope that helps, Tim At 08:16 AM 1/13/2011, Church, Charles muttered: All, I'm running into some issues with SPAN session limitations on 6500 (SXI on a VSS pair). After reading this doc: http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configuhttp://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configu ration/guide/span.html I'm lead to believe that if I make the destination interface a trunk, a span source of say VLANs 10 and 20 will leave the destination port with those VLAN tags intact. This appears to match the 'encapsulation replicate' that is present on the 3560s. My end goal is to use 2 3560 switches off of the 6500s to distribute SPAN sessions to 4 separate entities. Switch A will get a SPAN session off of the 6500 consisting of VLAN groups X and Y. Switch B will get a SPAN session off of the 6500 consisting of VLAN groups X and Z. Switch A will span VLAN group X to a certain destination port, and group Y to another. Switch B will do a similar thing with VLAN groups X and Z. I'm assuming normal local SPAN. I think the relies on the SPAN off of the 6500 to keep the VLAN tags intact. Can anyone confirm if my assumption is correct? Thanks, Chuck ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Fun problem affecting only even-numbered ports
On the fiber (SFP) card, ports are indeed mapped in groups of 12 consecutive odd/12 consecutive even ports per rohini, with 24 odd ports on one janus 24 even ports on the other. I'd guess you're just running out of replication bandwidth. A bit more detail about the traffic pattern etc would help. How many L3 replications are occurring here, and for what input rate? Ingress or egress replication? As a general rule you should spread the receivers as evenly as possible among odd even ports to spread the replication load. Hope that helps, Tim At 08:37 AM 10/28/2010, Ben Lovell (belovell) uttered: Nothing that I can think of. One Super Santa Anna and Janus run ports 1-24 and another pair of them run ports 25-48. One Rohini runs each consecutive group of 12 ports. The only thing I could guess at is the path take across the switchbar fabric but I don't recall how we select FPOE right now. -Ben On Oct 27, 2010, at 12:54 PM, John Neiberger wrote: This is a good one. I'm working with TAC on it, but I thought i'd share it here, too, just because it's so unusual. We're seeing intermittent drops on a multicast video stream and we haven't been able to determine why. This is the second time we've seen this bizarre behavior. At the urging of the TAC engineer, we tried moving receivers to different ports and noticed that we only see the drops when the receiver is connected to even-numbered interfaces! Odd-numbered interfaces are not affected. This is on a 7600 with SUP 720-3BXL and a 6748 linecard. What part of the 6748 linecard architecture is responsible for a behavior difference between odd- and even-numbered ports? This is a WS-6748-SFP with a DFC-3BXL. Any thoughts? ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Fun problem affecting only even-numbered ports
At 09:00 AM 10/28/2010, Mack McBride uttered: The 6748-SFP use one Rohini for even and one Rohini for odd ports not consecutive, IIRC. This could vary by revision. I don't have one handy to check against. The Janus and SSA are for ports 1-24 and 25-48. It's even/odd all the way back to the fabric on the 6748-SFP, one janus/SSA handles odd ports, the other even ports. Tim There are four Rohini on the board, two connected to each Janus/SSA pair. You can use 'sh int port cap | inc Ports to list the ports. And 'sh asic slot slot to list the asics. This is example output of a 6724 which is consecutive. #sh int Gi1/1 cap | inc Ports Ports-in-ASIC (Sub-port ASIC) : 1-24 (1-12) The first port list is the Janus and SSA. The second is the Rohini. #sh asic-version slot 1 Module in slot 1 has 3 type(s) of ASICs ASIC Name Count Version JANUS 1 (1.0) SSA 1 (9.0) ROHINI 2 (1.5) Mack -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.netmailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Benjamin Lovell Sent: Thursday, October 28, 2010 9:38 AM To: John Neiberger Cc: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] Fun problem affecting only even-numbered ports Nothing that I can think of. One Super Santa Anna and Janus run ports 1-24 and another pair of them run ports 25-48. One Rohini runs each consecutive group of 12 ports. The only thing I could guess at is the path take across the switchbar fabric but I don't recall how we select FPOE right now. -Ben On Oct 27, 2010, at 12:54 PM, John Neiberger wrote: This is a good one. I'm working with TAC on it, but I thought i'd share it here, too, just because it's so unusual. We're seeing intermittent drops on a multicast video stream and we haven't been able to determine why. This is the second time we've seen this bizarre behavior. At the urging of the TAC engineer, we tried moving receivers to different ports and noticed that we only see the drops when the receiver is connected to even-numbered interfaces! Odd-numbered interfaces are not affected. This is on a 7600 with SUP 720-3BXL and a 6748 linecard. What part of the 6748 linecard architecture is responsible for a behavior difference between odd- and even-numbered ports? This is a WS-6748-SFP with a DFC-3BXL. Any thoughts? ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] CEF Tuning with ECMP?
Hi John, Let's get everyone agreed on the terminology first, then we can try to solve the problem. * ECMP = Equal cost multipath, it is most typically a term used around unicast routing where for a given IP prefix you have multiple equal cost next hops and you load share traffic among them based on a hash (or less commonly per packet). The hash can be based on several criteria, ie, IPs L4 ports in various combinations. * CEF = it's interchangeable with 'ECMP' - CEF-based load sharing ECMP mean the same thing. * Multicast multipath = Uses a hash to select the RPF interface for a particular multicast flow when there are ECMP paths back to the source/RP. There are options to determine the input values (G, S,G, S,G+NH). This feature is not on by default in IOS. If it is not enabled, then IOS will choose ONE of the ECMP paths as the RPF (highest neighbor IP) and ALL multicast will be pulled over that link. * Polarization = In a 'multi-tier' network, using the same hash input on each tier results in traffic after the 1st tier polarizing to a subset of the available links. It's accomodated for by adding a unique ID at each hop to the hash input for unicast; for multicast multipath, by including the next hop IP as hash input. Whether this really comes into play depends on the depth of the network routing topology. Ok - so given all of the above, with ECMP routing between the 7600s the 4948s, and with multicast multipath already enabled on the 7600 and using S,G basic hashing: if the traffic flow is from the 4k-7600, the only option you have to improve things is to use S,G + next-hop. I'm not entirely convinced it will have a major impact, it depends on whether you have multiple levels of routing, one which is getting RPF hash selection pretty evenly but then at this layer, polarization is occurring since only a subset of traffic is reaching it and the hash input is the same (so only a subset of links is being selected as RPF). Based on your description I can't tell if that's a possibility in your setup. Regardless of all that, changing CEF/ECMP hash input on the 4948 will not have any significant impact, since that wouldn't affect multicast traffic at all, any particular S,G will still have only ONE of those four interfaces as an OIF, and that is driven by where the PIM join came in from the 7600, which in turn is driven by whether mcast multipath is enabled, and what hash is used to select the RPF interface. Also, clearly, changing CEF/ECMP hash input on the 7600 would have not any impact since you're worried about traffic flowing the other direction anyway. Hope that helps, Tim At 09:09 AM 10/3/2010, John Neiberger remarked: I'm starting another thread because the topic is migrating. To simplify, we have a 7600 with SUP720-3BXL connected via four routed links to a 4948. The bulk of the traffic on this network is multicast traffic flowing from the 4948 to the 7600 (and onward from there). In order to get the best load sharing over those four links, what is the recommended CEF tuning and ECMP configuration? I ask because we seem to be running into ECMP polarization and/or CEF polarization. We have already decided that we need to be using s-g-hash next-hop-based for ECMP. We're using s-g-hash basic right now. But what about CEF? Do we need to tune CEF along with tuning ECMP for this to work properly? We want the most even distribution of traffic possible. As it is right now, we're seeing serious unequal load sharing. In some cases all of the traffic is going over one link and not even using the other three. Do any of you know the recommended CEF parameters in a situation like this? Thanks, John ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] CEF Tuning with ECMP?
Hi John, For 4948-7600, I don't see why you would have channel replication issues. 4K is a shared memory buffer system, so there's no ingress/egress replication considerations or suboptimal distribution across the fabric of the MD packet for egress replication. There's only one copy of the packet in shared memory up to the point where a channel member is selected. In this situation I'd think a single 4 port routed EC would be preferred, with all mroutes having the same OIF (the EC) but then allowing the EC hash to select the link based on L3+L4. But that's theoretical - if you had issues with that then I'm not necessarily suggesting you go back to it. Of course, if you have heavy mcast going back the other direction as well, then egress L3 replication w/EC behavior is a consideration. 2 cents, Tim At 10:16 AM 10/3/2010, John Neiberger remarked: Thanks, Tim. I've only heard of ECMP in relation to multicast. Thanks for clearing up the terminology. It sounds like chanting the multicast multipath hash is our only option. We used to use etherchannels but we switched to this to help work around some multicast replication issues. If changing the hash doesn't work, we may have to go back to etherchannels. I really hope we don't have to do that. Thanks again! On Oct 3, 2010 11:08 AM, Tim Stevenson mailto:tstev...@cisco.comtstev...@cisco.com wrote: Hi John, Let's get everyone agreed on the terminology first, then we can try to solve the problem. * ECMP = Equal cost multipath, it is most typically a term used around unicast routing where for a given IP prefix you have multiple equal cost next hops and you load share traffic among them based on a hash (or less commonly per packet). The hash can be based on several criteria, ie, IPs L4 ports in various combinations. * CEF = it's interchangeable with 'ECMP' - CEF-based load sharing ECMP mean the same thing. * Multicast multipath = Uses a hash to select the RPF interface for a particular multicast flow when there are ECMP paths back to the source/RP. There are options to determine the input values (G, S,G, S,G+NH). This feature is not on by default in IOS. If it is not enabled, then IOS will choose ONE of the ECMP paths as the RPF (highest neighbor IP) and ALL multicast will be pulled over that link. * Polarization = In a 'multi-tier' network, using the same hash input on each tier results in traffic after the 1st tier polarizing to a subset of the available links. It's accomodated for by adding a unique ID at each hop to the hash input for unicast; for multicast multipath, by including the next hop IP as hash input. Whether this really comes into play depends on the depth of the network routing topology. Ok - so given all of the above, with ECMP routing between the 7600s the 4948s, and with multicast multipath already enabled on the 7600 and using S,G basic hashing: if the traffic flow is from the 4k-7600, the only option you have to improve things is to use S,G + next-hop. I'm not entirely convinced it will have a major impact, it depends on whether you have multiple levels of routing, one which is getting RPF hash selection pretty evenly but then at this layer, polarization is occurring since only a subset of traffic is reaching it and the hash input is the same (so only a subset of links is being selected as RPF). Based on your description I can't tell if that's a possibility in your setup. Regardless of all that, changing CEF/ECMP hash input on the 4948 will not have any significant impact, since that wouldn't affect multicast traffic at all, any particular S,G will still have only ONE of those four interfaces as an OIF, and that is driven by where the PIM join came in from the 7600, which in turn is driven by whether mcast multipath is enabled, and what hash is used to select the RPF interface. Also, clearly, changing CEF/ECMP hash input on the 7600 would have not any impact since you're worried about traffic flowing the other direction anyway. Hope that helps, Tim At 09:09 AM 10/3/2010, John Neiberger remarked: I'm starting another thread because the topic is migrating. To simplify, we have a 7600 with SUP720-3BXL connected via four routed links to a 4948. The bulk of the traffic on this network is multicast traffic flowing from the 4948 to the 7600 (and onward from there). In order to get the best load sharing over those four links, what is the recommended CEF tuning and ECMP configuration? I ask because we seem to be running into ECMP polarization and/or CEF polarization. We have already decided that we need to be using s-g-hash next-hop-based for ECMP. We're using s-g-hash basic right now. But what about CEF? Do we need to tune CEF along with tuning ECMP for this to work properly? We want the most even distribution of traffic possible. As it is right now, we're seeing serious unequal load sharing. In some cases all of the traffic is going over one link and not even using
Re: [c-nsp] CEF Tuning with ECMP?
Hi Mack, An example is shown in figure 16 here: http://www.cisco.com/application/pdf/en/us/guest/netsol/ns431/c649/ccmigration_09186a008093b876.pdf Hope that helps, Tim At 03:18 PM 10/3/2010, Mack O'Brian remarked: Thanks Tim for explaining the terminologies. That was really beneficial. I have a question on your comments under polarization In a 'multi-tier' network, using the same hash input on each tier results in traffic after the 1st tier polarizing to a subset What is 'multi-tier' network? Can you shed some light or point to a URL etc. Thanks again. Mack On Sun, Oct 3, 2010 at 10:08 AM, Tim Stevenson mailto:tstev...@cisco.comtstev...@cisco.com wrote: Hi John, Let's get everyone agreed on the terminology first, then we can try to solve the problem. * ECMP = Equal cost multipath, it is most typically a term used around unicast routing where for a given IP prefix you have multiple equal cost next hops and you load share traffic among them based on a hash (or less commonly per packet). The hash can be based on several criteria, ie, IPs L4 ports in various combinations. * CEF = it's interchangeable with 'ECMP' - CEF-based load sharing ECMP mean the same thing. * Multicast multipath = Uses a hash to select the RPF interface for a particular multicast flow when there are ECMP paths back to the source/RP. There are options to determine the input values (G, S,G, S,G+NH). This feature is not on by default in IOS. If it is not enabled, then IOS will choose ONE of the ECMP paths as the RPF (highest neighbor IP) and ALL multicast will be pulled over that link. * Polarization = In a 'multi-tier' network, using the same hash input on each tier results in traffic after the 1st tier polarizing to a subset of the available links. It's accomodated for by adding a unique ID at each hop to the hash input for unicast; for multicast multipath, by including the next hop IP as hash input. Whether this really comes into play depends on the depth of the network routing topology. Ok - so given all of the above, with ECMP routing between the 7600s the 4948s, and with multicast multipath already enabled on the 7600 and using S,G basic hashing: if the traffic flow is from the 4k-7600, the only option you have to improve things is to use S,G + next-hop. I'm not entirely convinced it will have a major impact, it depends on whether you have multiple levels of routing, one which is getting RPF hash selection pretty evenly but then at this layer, polarization is occurring since only a subset of traffic is reaching it and the hash input is the same (so only a subset of links is being selected as RPF). Based on your description I can't tell if that's a possibility in your setup. Regardless of all that, changing CEF/ECMP hash input on the 4948 will not have any significant impact, since that wouldn't affect multicast traffic at all, any particular S,G will still have only ONE of those four interfaces as an OIF, and that is driven by where the PIM join came in from the 7600, which in turn is driven by whether mcast multipath is enabled, and what hash is used to select the RPF interface. Also, clearly, changing CEF/ECMP hash input on the 7600 would have not any impact since you're worried about traffic flowing the other direction anyway. Hope that helps, Tim At 09:09 AM 10/3/2010, John Neiberger remarked: I'm starting another thread because the topic is migrating. To simplify, we have a 7600 with SUP720-3BXL connected via four routed links to a 4948. The bulk of the traffic on this network is multicast traffic flowing from the 4948 to the 7600 (and onward from there). In order to get the best load sharing over those four links, what is the recommended CEF tuning and ECMP configuration? I ask because we seem to be running into ECMP polarization and/or CEF polarization. We have already decided that we need to be using s-g-hash next-hop-based for ECMP. We're using s-g-hash basic right now. But what about CEF? Do we need to tune CEF along with tuning ECMP for this to work properly? We want the most even distribution of traffic possible. As it is right now, we're seeing serious unequal load sharing. In some cases all of the traffic is going over one link and not even using the other three. Do any of you know the recommended CEF parameters in a situation like this? Thanks, John ___ cisco-nsp mailing list mailto:cisco-nsp@puck.nether.netcisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, mailto:tstev...@cisco.comtstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000
Re: [c-nsp] Multicast replication over GRE on 7600s
With a tunnel you don't know which is the egress card until the encap is done. That's why tunnel OIFs are always ingress replicated. Tim At 09:43 AM 9/28/2010, Ben Lovell (belovell) declared: You would not have to force the box back to ingress. These packet would take the ingress forwarding path instead of egress. Other groups would still function in egress. I agree. it's hard to see how this would work in egress as the idea of replication is all packets are getting the same rewrite(on ingress) and egress card just needs to make copies. I suppose you could replicate in the normal fashion to each egress LC plus one more copy for the GRE tunnel would would then loop through lookup process again for GRE encap but this is purely conjecture on my part. -Ben ~ .. Benjamin Lovell || AS Video Practice ||| ||| Cisco Customer Advocacy .|. .|. Research Triangle Park, NC .:|:..:|:.Email: belov...@cisco.com ciscodesk:919.392.8255 cell:203.509.1562 ~ On Sep 28, 2010, at 12:34 PM, John Neiberger wrote: Now that I think about it, I bet that egress mode isn't allowed in this scenario. It would make sense that only ingress mode would work, that way the ingress Janus/Metro would take care of replicating out to all the receivers, including the GRE tunnel. I'm having trouble visualizing how that would work in egress mode. It was worth checking into, though. We have a situation where this might be useful temporarily. But since we're running egress on our 7600s, moving back to ingress is just not an option. Once again, thanks for your help! John On Tue, Sep 28, 2010 at 10:25 AM, Benjamin Lovell belov...@cisco.com wrote: The same hardware(janus/metro) is responsible for the replication(no punt to CPU) but due to the GRE ecanp required the packet will have to go through a longer forwarding process(more lookups) and performance will be reduced. I don't have any solid numbers but my guess is that forwarding rate would be approx 1/2. The part I am not sure about is if egress replication is still possible. In the mVPN scenario only ingress replication is possible due to the GRE encap/decap but I am not sure if this same limitation applies to P2P GRE tunnels. Let me know if this piece would be important to you and I can look into it. The one caveat to be careful of here(applies to unicast as well) is that each GRE tunnel must be sourced from a unique IP address on the box. Using the same source IP on more than one GRE tunnel will cause all traffic in GRE decap path to be punted to CPU and maybe multicast on encap path in some scenarios. -Ben ~ .. Benjamin Lovell || AS Video Practice ||| ||| Cisco Customer Advocacy .|. .|. Research Triangle Park, NC .:|:..:|:.Email: belov...@cisco.com ciscodesk:919.392.8255 cell:203.509.1562 ~ On Sep 28, 2010, at 10:02 AM, John Neiberger wrote: I have an architectural question. As an example, let's say you have two 7600s directly connected via routed links running PIM and you have primarily multicast traffic. If you're running egress replication mode, you either have a Janus ASIC or Metropolis ASIC responsible for the multicast replication, which happens on the line card. But how would things change if you were not running multicast directly on the routed link, but instead used a GRE tunnel between the two routers? I guess I have a couple of questions: 1. How is GRE traffic processed in this scenario? Can it be forwarded at high rates on the line card or is it punted to the CPU? 2. Which hardware is responsible for multicast replication over GRE tunnels? Any ideas? Thanks! John ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759
Re: [c-nsp] Multicast replication over GRE on 7600s
I'm gonna have to admit I'm getting rusty on some of these details, but IIRC, for standard GRE OIF, it is ingress replicated and recirculated to do a lookup on the outer IP header; but other non-tunnel OIFs can still be egress replicated. For MVPN configurations, the entire box is forced to ingress mode. Hope that helps, Tim At 10:34 AM 9/28/2010, Ben Lovell (belovell) declared: Tim, Just to make sure I am understanding. For a certain group, non-tunnel OIFs would still use egress replication and only tunnel OIFs would be ingress or the whole group falls back to ingress? -Ben ~ .. Benjamin Lovell || AS Video Practice ||| ||| Cisco Customer Advocacy .|. .|. Research Triangle Park, NC .:|:..:|:.Email: mailto:belov...@cisco.combelov...@cisco.com ciscodesk:919.392.8255 cell:203.509.1562 ~ On Sep 28, 2010, at 1:13 PM, Tim Stevenson wrote: With a tunnel you don't know which is the egress card until the encap is done. That's why tunnel OIFs are always ingress replicated. Tim At 09:43 AM 9/28/2010, Ben Lovell (belovell) declared: You would not have to force the box back to ingress. These packet would take the ingress forwarding path instead of egress. Other groups would still function in egress. I agree. it's hard to see how this would work in egress as the idea of replication is all packets are getting the same rewrite(on ingress) and egress card just needs to make copies. I suppose you could replicate in the normal fashion to each egress LC plus one more copy for the GRE tunnel would would then loop through lookup process again for GRE encap but this is purely conjecture on my part. -Ben ~ .. Benjamin Lovell || AS Video Practice ||| ||| Cisco Customer Advocacy .|. .|. Research Triangle Park, NC .:|:..:|:.Email: mailto:belov...@cisco.combelov...@cisco.com ciscodesk:919.392.8255 cell:203.509.1562 ~ On Sep 28, 2010, at 12:34 PM, John Neiberger wrote: Now that I think about it, I bet that egress mode isn't allowed in this scenario. It would make sense that only ingress mode would work, that way the ingress Janus/Metro would take care of replicating out to all the receivers, including the GRE tunnel. I'm having trouble visualizing how that would work in egress mode. It was worth checking into, though. We have a situation where this might be useful temporarily. But since we're running egress on our 7600s, moving back to ingress is just not an option. Once again, thanks for your help! John On Tue, Sep 28, 2010 at 10:25 AM, Benjamin Lovell mailto:belov...@cisco.combelov...@cisco.com wrote: The same hardware(janus/metro) is responsible for the replication(no punt to CPU) but due to the GRE ecanp required the packet will have to go through a longer forwarding process(more lookups) and performance will be reduced. I don't have any solid numbers but my guess is that forwarding rate would be approx 1/2. The part I am not sure about is if egress replication is still possible. In the mVPN scenario only ingress replication is possible due to the GRE encap/decap but I am not sure if this same limitation applies to P2P GRE tunnels. Let me know if this piece would be important to you and I can look into it. The one caveat to be careful of here(applies to unicast as well) is that each GRE tunnel must be sourced from a unique IP address on the box. Using the same source IP on more than one GRE tunnel will cause all traffic in GRE decap path to be punted to CPU and maybe multicast on encap path in some scenarios. -Ben ~ .. Benjamin Lovell || AS Video Practice ||| ||| Cisco Customer Advocacy .|. .|. Research Triangle Park, NC .:|:..:|:.Email: mailto:belov...@cisco.combelov...@cisco.com ciscodesk:919.392.8255 cell:203.509.1562 ~ On Sep 28, 2010, at 10:02 AM, John Neiberger wrote: I have an architectural question. As an example, let's say you have two 7600s directly connected via routed links running PIM and you have primarily multicast traffic. If you're running egress replication mode, you either have a Janus ASIC or Metropolis ASIC responsible for the multicast replication, which happens on the line card. But how would things
Re: [c-nsp] Nexus evolution
Next major s/w release (Cairo, release # most likely to be 5.1) supports 2248 to n7k directly. 2232 comes a bit later (within 6-8 months). Hope that helps, Tim At 09:45 AM 9/27/2010, David Freedman declared: I believe that this direct-to-7k support is only just being released in s/w (aug/sep) and it will be limted to 32 FEX per 7k (and fex must be 2248 or 2232, 2148 not supported) Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Operational impact of switching from ingress to egress replication mode
Hi John, Switching replication modes basically purges the hardware of all mroutes and those will be reprogrammed based on the current software state. It will be potentially disruptive for all mroutes, but the exact amount of traffic loss/blackholing would depend on the rate of each stream at the time, and the overall amount of time it takes to reprogram. For a few 100 mroutes, I would not expect much impact. Hope that helps, Tim At 11:30 AM 9/21/2010, John Neiberger averred: We're running 12.2(33)SRC2, I believe. It's actually experimental code and the exact version is overwritten with another code. On Tue, Sep 21, 2010 at 12:04 PM, Jeffrey Pazahanick jeff...@gmail.com wrote: John, Having switched back and forth a few times, I never noticed more than a 1-2 second outage. What version of code are you on? On Tue, Sep 21, 2010 at 11:59 AM, John Neiberger jneiber...@gmail.com wrote: We're going to be doing a whole bunch of maintenance tonight during a maintenance window. One of the many things on our plate is to switch from ingress replication mode to egress on some 7600s that have a few hundred multicast routes on them. We know there is going to be at least a minor blip while things settle down after making the change, but I wanted to see if anyone on the list has done this and what the operational impact was. I've heard there will be slight interruption in traffic, but what sort of interruption are we talking about? Are we speaking about a second or two? I'm asking because we're trying to decide if we want to split this out to another night. If the disruption is minor and the risk is low then we'll do it tonight. Otherwise, we might choose to do it on a separate night. Any thoughts? ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] N7k: Attaching a service-policy to an SVI
Hi Matt, Arie, Yes, the documentation is incorrect in that section, I will work to get that updated. Arie is right, you apply qos service policies to the vlan itself, not the SVI, in NXOS. That's to decouple SVI creation and policy application for vlans. For an L2 switchport, L3 interface, or L3 subinterface: int x/y[.z] service-policy in|out foo For an entire VLAN: vlan x service-policy in|out foo Note however that in an upcoming release (5.1) we will be changing that to be in line with the new IOS convention of putting VLAN config under the vlan-configuration mode, like this: vlan configuration x service-policy in|out foo That change is to decouple VLAN creation and policy application for VLANs (specifically to support VTP client, where you may want to tie a policy to a VLAN that has not been learned through VTP yet). Hope that helps, Tim At 06:43 AM 9/2/2010, Arie Vayner (avayner) vociferated: Matt, You should be able to apply the qos policy on the vlan (as opposed to interface vlan): Let me know if it works for you. This is the doc, but I think it might be wrong (for the syntax) - let me know, and I will see internally. http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/qos/conhttp://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/qos/con figuration/guide/cpl.html#wp1056677 Arie -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.netmailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Matthew Melbourne Sent: Thursday, September 02, 2010 14:18 To: cisco-nsp@puck.nether.net Subject: [c-nsp] N7k: Attaching a service-policy to an SVI Hi, Is it possible to attach a QoS service policy (in this case a simple ICMP policier) to a VLAN interface on the N7k platform (NX-OS 5.0(2a))? The docs suggest it is possible, but the service-policy command doesn't appear to be available in interface configuration mode for an SVI (the command is present under a physical interface). Cheers, Matt -- Matthew Melbourne ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] incoming queue
If you are using COS-Q mapping, the frame *must* carry COS to be prioritized on ingress. If using DSCP-Q (available on certain 10G modules) then we can prioritize on the ingress port based on DSCP, which of course would always be carried in an IP packet. An input service-policy is applied in the PFC/DFC - ie, 'long after' the packet was ingress queued. Clearly ingress/egress service policies, trust statements, etc affect how egress queueing is handled and what's carried in the final packet. In the vast majority of cases, input queuing just isn't that useful as queueing only happens when there's congestion - and congestion is more likely by far on egress where you can have n-to-1 traffic patterns. Tim At 12:39 PM 8/20/2010, P.A contended: Yeah I have the same config, but donât think that incoming packets will be mapped to the priority queue. I think all the config below is going to do is set the internal dscp with the policy to be used on the outgoing queue. From what I understand from the link below all incoming packets 1st are assigned to an incoming queue before anything else. So unless the packet already has a COS marking it will not use the priority queue. -Original Message- From: Peter Rathlev [mailto:pe...@rathlev.dkmailto:pe...@rathlev.dk] Sent: Friday, August 20, 2010 2:47 PM To: P.A Cc: cisco-nsp Subject: Re: [c-nsp] incoming queue Hi Paul, (I've Cc'ed the list again since other people's eyes can spot errors I might have made.) On Fri, 2010-08-20 at 12:51 -0400, P.A wrote: Actually Peter, I have come to determine that even now with the correct cos 5 mapped to the incoming priority queue, unless itâs a trunk it will not pass any packets to that queue. See, http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SXF/native/configuration/guide/qos.html#wp1740736http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SXF/native/configuration/guide/qos.html#wp1740736 A service policy is applied outside of this diagram (afterwards). Take a look at figure 41-6 on the same page (anchor wp1744018 on same URL). As far as I know a service-policy will always override port trust (and CoS/DSCP mappings). And I see there's even a very simple way to trust DSCP via a policy-map. I tested this configuration successfully: ! *** Router *** policy-map Trust-DSCP-pmap class class-default trust dscp exit ! exit ! interface GigabitEthernet1/1 description Towards test host switchport switchport access vlan 3 switchport mode access mtu 9216 load-interval 30 mls qos trust cos spanning-tree portfast edge service-policy input Trust-DSCP-pmap exit ! And it works fine, everything comes through with correct DSCP/ToS. Removing the service-policy makes everything CoS 0 / DSCP 0; trust DSCP always lets things through. (Verified with ping -Q tos ... and a tcpdump on the other end.) Thus you can trust DSCP on an access port and still be able to configure a priority-queue on input (i.e. have mls qos trus cos on the interface). This test was on a PFC3B, module is a WS-X6748-GE-TX without DFC and the system is running SXI/AIS. The WS-X6748-GE-TX doesn't have a priority-queue on input (it's 1q8t) so I couldn't test that part, but I can see no reason why it wouldn't work exactly the same way on other modules and other software. I tested it on a WS-X6516-GBIC (1p1q4t) and SXF/AES, but I don't have a host on that box to verify with. It didn't complain about the configuration though.) P.S.: If you want to trust only some DSCP values (or for some other reason want to keep the class-map model) you can bundle DSCP values: class-map match-any Trust-DSCP-cmap match ip dscp 0 1 2 3 4 5 6 7 match ip dscp 8 9 10 11 12 13 14 15 match ip dscp 46 63 exit ! policy-map Trust-DSCP-pmap class Trust-DSCP-cmap trust dscp exit ! class class-default set ip dscp 0 exit ! ! interface GigabitEthernet1/1 [...] service-policy input Trust-DSCP-pmap exit ! This trusts only DSCP 0-15, 46 and 63. -- Peter ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] internal DSCP
Hi PA, Not sure where your quotes are coming from. The 2nd one is a bit misleading IMO, it sort of mangles the concepts of PFC classification egress port QoS. It refers to a specific behavior around matching remarked DSCP in an egress ACL. That is not possible w/o a recirculation. This section of the cfg guide should help: http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configuration/guide/qos.html#wp1766015 The point is, if you remark DSCP on ingress, an egress ACL will not match on the remarked DSCP without first recirculating the packet. Regardless of that, the remarked DSCP (or derived COS) will definitely be used to map the packet to an egress port queue and that remarked DSCP will be carried in the final packet. Hope that helps, Tim At 08:41 AM 8/12/2010, P.A averred: I have a questing about internal DSCP on a 6500 that I'm not really sure about. I know that it's used to identify the priority of a frame/packet as it transits the switch but I have read on some sites that the internal DSCP is copied to the frame/packet as it leaves the switch. On other sites that when the packet arrives at the egress port the original ToS will be used. So im confused to which is true, see below. Upon output, the internal DSCP settings are copied to the IP header. If the outbound frame is being given a trunking header (ISL or 802.1Q) then the appropriate CoS bits are also set. 7.12 Egress ACL Support for Remarked DSCP When a frame transits the switch, classification performed by the PFC can change the ToS priority setting (IP Precedence or DSCP) in the packet. When the packet arrives at the egress port however, the default action will result in the original ToS priority (not the PFC remarked ToS priority) that was present on the arriving packet to be used for classification purposes. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] internal DSCP
Hi Paul, please see inline below: At 12:40 PM 8/12/2010, P.A averred: Tim thanks for your response, I think I'm starting to get it. So basically if you do ingress marking with an ACL by default it will not use that marking on egress unless you use an egress acl for remarked packets using 'platform ip features sequential'? No. It *WILL* use that remarked value to rewrite the packet regardless. The issue is, the fwding engine does certain things in parallel. In this case, there is no signalling in the ASIC of the remarked DSCP value to the logic that does egress ACL classification/enforcement (remember, the INGRESS fwd engine does both ingress egress policy enforcement). So that match will be based on the original value. However, the egress interface will always use the remarked DSCP/COS to enqueue the packet, and the packet will always be tx'd using that remarked value. This knob is SPECIFIC only to the case where you want to remark on ingress AND match on the remarked value, not the original value, in an egress ACL. It doesn't affect/change the egress q'ing behavior nor what is rewritten into the packet. What about for the following, would the set dscp value be used? class-map match-any voip_traffic match protocol rtp audio Policy Map incoming_voip_policy Class voip_traffic trust dscp police cir percent 20 conform-action set-dscp-transmit ef This will work exactly as you expect. What would NOT work, without the knob, is if you then have an egress ACL on the egress L3 interface that tries to match on the remarked DSCP value (eg, match on packets that were marked down by the policer). Note that recirculation incurs a performance hit, it requires 2 passes thru the fwding engine. Hope that helps, Tim thanks, Paul -Original Message- From: Tim Stevenson [mailto:tstev...@cisco.commailto:tstev...@cisco.com] Sent: Thursday, August 12, 2010 2:34 PM To: P.A; cisco-nsp@puck.nether.net Subject: Re: [c-nsp] internal DSCP Hi PA, Not sure where your quotes are coming from. The 2nd one is a bit misleading IMO, it sort of mangles the concepts of PFC classification egress port QoS. It refers to a specific behavior around matching remarked DSCP in an egress ACL. That is not possible w/o a recirculation. This section of the cfg guide should help: http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configuhttp://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configu ration/guide/qos.html#wp1766015 The point is, if you remark DSCP on ingress, an egress ACL will not match on the remarked DSCP without first recirculating the packet. Regardless of that, the remarked DSCP (or derived COS) will definitely be used to map the packet to an egress port queue and that remarked DSCP will be carried in the final packet. Hope that helps, Tim At 08:41 AM 8/12/2010, P.A averred: I have a questing about internal DSCP on a 6500 that I'm not really sure about. I know that it's used to identify the priority of a frame/packet as it transits the switch but I have read on some sites that the internal DSCP is copied to the frame/packet as it leaves the switch. On other sites that when the packet arrives at the egress port the original ToS will be used. So im confused to which is true, see below. Upon output, the internal DSCP settings are copied to the IP header. If the outbound frame is being given a trunking header (ISL or 802.1Q) then the appropriate CoS bits are also set. 7.12 Egress ACL Support for Remarked DSCP When a frame transits the switch, classification performed by the PFC can change the ToS priority setting (IP Precedence or DSCP) in the packet. When the packet arrives at the egress port however, the default action will result in the original ToS priority (not the PFC remarked ToS priority) that was present on the arriving packet to be used for classification purposes. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.n ether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net /mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.ne t/pipermail/cisco-nsp/http://puck.nether.net/piperma il/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.comhttp://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential
Re: [c-nsp] Erspan on 7600
It most certainly is done in h/w - as I said, both encap decap (ie source dest sessions) are hw based. At the dest box, the FE strips off the GRE/ERSPAN header dumps the original packet on the SPAN dest port. Note that certain misconfigurations can cause a punt, particularly on the ERSPAN destination box. I suggest carefully following the config guide for the proper configuration. One key point is configuring identical IPs and ERSPAN IDs on the source destination sessions. That can all be avoided by just pointing directly to the IP of a host w/wireshark etc that can decode the packets with the GRE/ERSPAN still on it. Hope that helps, Tim At 01:02 AM 8/11/2010, Mikael Abrahamsson averred: On Wed, 11 Aug 2010, Phil Mayers wrote: We've used ERSPAN on some truly phenomenal bitrates; it *is* done in hardware. Sending ERSPAN is done in hw, I've done multigigabit ERSPAN. ERSPAN reception is not done in HW (at least not when I tested) and it killed the box when I tried :P -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Erspan on 7600
Hi Martin, ERSPAN is handled by the hardware, either the central replication engine on the sup, or by the REs on the linecards themselves (depends on which sup LCs you have). In no case do we use the sup CPU to perform ERSPAN encap/decap. Tim At 07:10 AM 8/10/2010, Martin Moens averred: Hi list, Does someone have experience with erspan on a 7600? Is this loading the CPU (rsp720 / ws-x6748-ge-tx) or is it handled in hardware? Martin ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Linecard performance w/ SUP32's
Hi Graham, please see inline below: At 09:17 AM 8/9/2010, Graham P. Wooden averred: Hi there, I am trying to decide if swapping a few x6348s to x6548 (fabric-enabled/CEF256 .. no DFC) will be a huge performance gain with SUP32s (IOS is SXI). Nope. From what I am reading, it looks like they will somewhat - but appears that I will bump up against the 256 mark until the SUPs themselves get upgraded. There is no 256 with sup32. It is a bus-based sup, no xbar fabric whatsoever. Even if the line cards have the DFCs, can the SUP32-3B take advantage of them? DFC + sup32 = not supported/possible. Basically you're trading one bus-attached 48 port 1G card for another. There are some benefits perhaps - qos capabilities, possibility to u/g to sup720 in future get fabric connections add DFCs for more performance - but for now, they are pretty equivalent. Hope that helps, Tim Currently, this chassis doesn't see a whole-lot of traffic and is very light on the QoS side - but I have the funds to get this particular chassis upgraded and thought that spending a few bucks for new line cards would be a step in the right direction. Appreciate any real-world comments on the older 6348 vs. 6548 w/ a SUP32. Thanks, -graham ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Multicast issues on 7600s with WS-6748-sfp blades
Moving from port 24 to 25 moves you to the other Janus regardless of whether there's a DFC or not. John, if as you say moving to the other replication engine (RE) solves the problem, it's possible you are simply exceeding the replication capacity of the Janus. Metro based cards (6708, 6716) have superior replication capacity, but that doesn't help you for GE, there is no shipping GE card for c6k that uses metro. Note that changing the DFC has no impact whatsoever on the RE hardware, the RE is on the linecard itself. More recent code does provide a CLI to monitor drops in the replication engine, show platform hardware capacity rewrite-engine, this was added in 12.2(33)SXI. That could help you figure out whether drops are happening due to oversubscription of the Janus. Hope that helps, Tim At 08:00 AM 7/29/2010, Ben Lovell (belovell) submitted: John, Is your 6748 a DFC line card? If so you are correct, moving from port 24 to 25 would have moved you to the other Janus ASIC. Janus is the fabric/mcast replication ASIC. If your problem is ONLY with mcast then you can safely ignore the Rohini. We have addressed a number of issues with mcast replication on the Janus in recent years. So if you are running older 12.2(18)SXF or even earlier SX code you may want to consider an IOS update. Also not knowing your network I can't say but it's not impossible to hit the scale limits with high rates mcast pps and mroute counts. The newer LCs with the 3CXL use a newer replication ASIC that preforms significantly better than the Janus. -Ben On Jul 28, 2010, at 6:04 PM, John Neiberger wrote: We have a weird problem on some 7606s with WS-6748-SFP blades. We have a whole bunch of multicast streams running through these routers and there are multicast receivers directly attached. We had a problem where one particular multicast stream would occasionally have dropped packets resulting in MPEG CC errors on the receiver. We were able to prove that the source was clean, as were the paths between the source and the receiver. The receiver was not seeing MPEG CC errors on any other stream, which is really odd. Here's where it gets even stranger. When we moved the receiver to another port, like from 23 to 24, the receiver still saw the errors. We moved it to port 25 and the errors apparently went away. Our only guess is that this could potentially be an issue with the ASICs on the blade. Ports 13-24 are controlled by one ASIC (I believe a Rohini ASIC), and the four Rohini ASICs connect to two Janus ASICs. That is as I understand it. I may be wrong. When we moved the receiver from port 24 to 25, we moved to a different Rohini ASIC and possibly to another Janus ASIC. Regardless, according to our video guys the problem cleared up after the move. Have any of you ever experienced anything like this? Could this really be a problem on the Rohini or Janus ASICs? What sort of problem would only affect certain multicast groups and not others? Thanks ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Multicast issues on 7600s with WS-6748-sfp blades
It never worked, when I sent that reply I had mistakenly thought it was an internal email. The closest public thing you'll probably find is the 6500 h/w architecture decks from networkers. That, or talk to your Cisco account team. Tim At 01:17 PM 7/29/2010, Mikhail submitted: Googling about it gave me link to Tim's page, which doesn't work anymore: http://bock-bock.cisco.com/~tstevens/FAQs/module-asic-mappings.htmlhttp://bock-bock.cisco.com/~tstevens/FAQs/module-asic-mappings.html Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] SXI3 strange issue, Loose mode uRPF jumps to strict by itself
CSCec39733 added just such a warning ages ago, back in the 12.1 days - but I just checked a c6k running 12.2(33)SXH and it's not there any more, so there seems to be a regression. Tim At 08:25 PM 7/29/2010, Church, Charles submitted: I got bit by this just a couple weeks ago. Building a new core router for a location, couldn't ping up through the Sidewinder gateways I'm only a little familiar with. Blaming it on my lack of Sidewinder experience, turns out my default had changed to strict mode after changing the inward facing ints to strict. Doh! Seems like a warning message would be nice, like they do with portfast. Chuck Church Network Planning Engineer, CCIE #8776 Southcom Harris IT Services 1210 N. Parker Rd. Greenville, SC 29609 Office: 864-335-9473 Cell: 864-266-3978 E-mail: charles.chu...@harris.com Southcom E-mail: charles.church@hq.southcom.mil -Original Message- From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-boun...@puck.nether.netmailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Jared Mauch Sent: Thursday, July 29, 2010 3:32 PM To: bas Cc: Cisco Subject: Re: [c-nsp] SXI3 strange issue, Loose mode uRPF jumps to strict by itself On the SUP720/EARL7 unicast-rpf is a global setting on the device. If someone changes *any* interface to strict, all interfaces with u-rpf enabled will change to strict. - jared On Jul 29, 2010, at 3:21 PM, bas wrote: Hi All, Yesterday we had a strange issue. Our monitoring tool alerted that one of our boxes (SUP720-3BXL - 6506 running SXI3) became unreachable. When we logged in everything looked ok. BGP was up, OSPF was up and nothing special in logging. Still traffic had dropped to near zero. With debug ip cef drop we immediately saw that traffic was dropped due to uRPF feature. All upstream interfaces had strict mode uRPF configured, before the problems started it was loose mode uRPF. After manually changing them back too loose mode traffic was restored. A couple of minutes before the problems started an engineer had configured a customer facing interface with strict mode uRPF. Apparently this configuration changed triggered a bug that caused upstream interface loose mode to be automagically turned to strict mode. So, hereby a heads up. If your SXI3 boxes show strange behavior, quickly check uRPF. Cya, Bas ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Distinguished Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] LTL?
Hi Ray, You're right, LTLs are also known as port selects, it's basically a table in various ASICs that identify which ports should receive a frame. For a unicast frame, the LTL would select a single port; for multicast, could be many or all. What the destination LTL of a packet is inside the box is typically dictated by the forwarding engine lookup. It's a bit more complex than that when you dig into it, but that's it in a nutshell. Note that every physical port in the system has an LTL (destination index) set aside for it so you can't really run out in that case. But in other cases LTLs are a dynamic, limited resource, eg for multicast, there's a fixed number available, each describing a particular fanout or receiver set, so if there are many unique receiver sets then each will burn an LTL from that pool. Some platforms can share those LTLs so if two mcast groups share the same fanout they can share an LTL. Others don't have that capability. The specific capacities capabilities etc are very hardware dependent. The primary shipping platforms that use LTL today that I know of are 6500/7600, and Nexus 7000. What leads you to think you have run into an overflow condition, some error msg I take it? Hope that helps, Tim At 01:13 PM 6/30/2010, Ray Van Dolson contended: Newbie question here. LTL -- in the context of the Cisco world -- what is it and what does it do? Sounds like it's some sort of an index to track which ports to forward frames to. However, I believe we may have run into an LTL overflow (older hardware, 32-bit data-size?). Anyone know what might trigger this? Just wanting to get a little background / understanding. What I've read makes it sound like the LTL wouldn't contain all that much data. Thanks, Ray ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] policy-maps on dCEF platforms
archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] CPP and MAC address population
Hi Chris, The SMAC will be learned in that case. Hope that helps, Tim At 06:55 PM 5/21/2010, Chris Woodfield opined: Odd question, but there's a reason for it, I assure you... On the 6500/Sup720 platform, will an ethernet frame that is dropped due to hitting a control-plane policer rule (or an inbound policer, or inbound access list, ...) still cause its source MAC address to be entered into the switch's MAC address table? Or will the fact that the frame is dropped prevent the MAC table population from taking place? Thanks, -C ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] N7K tcam handling
Hi Tim, please see inline below: At 08:01 AM 3/9/2010, Tim Durack clamored: Anyone know if the N7K handles tcam exhaustion more gracefully than the 6500? (If you've lived through that experience, you'll know why I'm asking.) Yes, it does. I say that because n7k will reject your configuration if it won't fit within the constraints of the hw resources. C6K will instead punt to software to let the RP CPU enforce the ACL (and you can probably guess the result - inband saturated CPU pegged). Other improvements on n7k WRT ACLs: - we rarely merge polices, the ACL TCAM is carved bank-wise mostly on a per feature basis (you can chain the banks if you have enormous ACLs) - also, we don't try a bunch of different merge strategies to try to make things fit, driving up the CPU util - we have a verify/commit option using config sessions, ie, you make all your ACL changes in a scratch area, then use the verify cmd to make sure it will fit in the hardware. Only then do you commit it. - we have atomic ACL commits, ie, non traffic disruptive by default (versus a default result (deny by default) on c6k while the old entries are removed the new installed). Docs suggest the N7K is generally smarter about handling tcam than the 6500. Or maybe NX-OS is smarter. (IMHO,) yes, both. :P Heres an idea for Cisco: how about porting NX-OS to the 6500? No committed plans. Or release a new Sup that makes the C6K an N6.5K? C6K will continue to evolve and they do have a roadmap to a new sup fabric. Hope that helps, Tim I think you would make a lot of customers happy. -- Tim: ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsphttps://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/http://puck.nether.net/pipermail/cisco-nsp/ Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] N7K tcam handling
Hi Tim, Sorry about that, assumed you were talking about ACL TCAM, but you are referring to FIB TCAM. In the scenario you mention, prefixes are installed in the FIB TCAM on a first come first served basis. Packets not matching a prefix in the FIB TCAM are punted to the CPU, but such traffic is heavily rate limited (to protect the inband/CPU), so your routing will be considerably hosed. Obviously we syslog such events. As you probably know, n7k today has a 128K FIB TCAM, inadequate to hold full routes anyway. Near-term we will have an XL card that holds 900K prefixes. In that case, you should not run out of FIB TCAM in the case you describe, but as always, you should be sure not to miss configuring route limits filters to avoid issues, that's clearly best practice. Hope that helps, Tim At 09:31 AM 3/9/2010, Tim Durack clamored: Good to know. I was actually thinking more along the lines of: BGP peering, missing max-prefix, provider dumps 300k routes on me. What does the N7K do? (Unfortunately I know what a 6500 does.) Tim Stevenson, tstev...@cisco.com Routing Switching CCIE #5561 Technical Marketing Engineer, Cisco Nexus 7000 Cisco - http://www.cisco.com IP Phone: 408-526-6759 The contents of this message may be *Cisco Confidential* and are intended for the specified recipients only. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
