Re: [c-nsp] ASR920 Opinions
Hi, > On Dec 19, 2017, at 8:52 PM, James Jun wrote: > > Hey, > > We have about 40 of ASR920's, mostly 24SZ-M and 24SZ-IM variants. We're > running mainly 03.16.04S and 03.16.05S. … > For layer-2 services, we use LDP signalled L2CKT and VPLS. We tried testing > layer-3 use case, but last time we > tested (it was on early SW versions though.. like 3.14.x something), > control-plane protection like didn't even > work as we expected. Are you saying that whatever L3 issues you had have been resolved in the versions you cited above? > My overall experience with ASR920 are as follows. ... > The Bad Stuff: > - Weird behavior with 10G ports and optics: Sometimes when upgrading SW, > some of the SFPs (e.g. Bidi ones) > fail to come up. Bouncing the interface with shut/no shut does nothing; > dispatching field service crew to > remove and re-insert the optic solves the problem. > > We also had issues with 10G ports going admin-down upon upgrade as well. > Long story short, OOB is highly > desirable to have if SW upgrade is required on this platform. Did you happen to catch a bug ID for either of these two 10G port issues? > - Shallow buffers - 12MB for the whole box; and default values are > ridiculously small. > I'm not sure what Cisco was thinking regarding buffers on this box. ASIC > speed has nothing to do with > buffering requirements when you're downstepping from 10G to 1G -- you > either have buffers to make up for > the Tx/Rx rate difference or you tail drop, it's as simple as that. Are you referring partly to this? https://www.cisco.com/c/dam/en/us/td/docs/routers/asr920/design/Cisco-ASR920-Microburst-whitepaper.pdf > We applied 100% shared buffers with policy-map, but we did run into buffer > exhaustion when several customers > are doing heavy inbound traffic. It's fine for putting in typical end-user > / retail users, but for placing > lot of enterprise 1GE internet customers on the box, I don't know.. We > ended up configuring fixed 512KB queue > on every 1GE port (so we don't really oversubscribe the 12MB buffer space) > to absorb up to ~2ms worth of burst, > but this now brings back lot of tail drops on long distance TCP flows. So, > we're now having upstream IP > transit routers at head-end sites provide traffic-shaping with very low > burst on customer EVCs terminating > on ASR920s. It's not ideal, as this means I'll need -SE line card at > upstream side to deal with the increased > queueing requirements, but it is a decent compromise. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR920 Opinions
Hey, We have about 40 of ASR920's, mostly 24SZ-M and 24SZ-IM variants. We're running mainly 03.16.04S and 03.16.05S. We're using ASR920s only for layer-2 transport -- pseudowires and VPLS; For internet customers, we establish an L2 pseudowire to transport the user from ASR920 to an IP transit hub router (ASR9K or MX480). For layer-2 services, we use LDP signalled L2CKT and VPLS. We tried testing layer-3 use case, but last time we tested (it was on early SW versions though.. like 3.14.x something), control-plane protection like didn't even work as we expected. My overall experience with ASR920 are as follows. The Good Stuff: - Most layer-2 features behave more or less like a real router (ASR1K) than the traditional Catalyst SW model. You get real EFP infrastructure and you can rewrite vlan on ingress on EFPs and such. Getting these features at low price point is very cool. - For core-facing L3 interfaces (OSPF, MPLS w/ RSVP-TE, LDP tunneling, BGP), no problems at all. Works very well as expected. Overall, ASR920 is like the perfect Metro-E switch but configuration wise, behaves much like a router than a switch. I think this makes 920 much, much more attractive platform when compared to similar MetroE/packet backhaul boxes from say.. Ciena, etc. - The device appears to perform well as advertised on traffic and packet forwarding rates. The Bad Stuff: - Weird behavior with 10G ports and optics: Sometimes when upgrading SW, some of the SFPs (e.g. Bidi ones) fail to come up. Bouncing the interface with shut/no shut does nothing; dispatching field service crew to remove and re-insert the optic solves the problem. We also had issues with 10G ports going admin-down upon upgrade as well. Long story short, OOB is highly desirable to have if SW upgrade is required on this platform. - Shallow buffers - 12MB for the whole box; and default values are ridiculously small. I'm not sure what Cisco was thinking regarding buffers on this box. ASIC speed has nothing to do with buffering requirements when you're downstepping from 10G to 1G -- you either have buffers to make up for the Tx/Rx rate difference or you tail drop, it's as simple as that. We applied 100% shared buffers with policy-map, but we did run into buffer exhaustion when several customers are doing heavy inbound traffic. It's fine for putting in typical end-user / retail users, but for placing lot of enterprise 1GE internet customers on the box, I don't know.. We ended up configuring fixed 512KB queue on every 1GE port (so we don't really oversubscribe the 12MB buffer space) to absorb up to ~2ms worth of burst, but this now brings back lot of tail drops on long distance TCP flows. So, we're now having upstream IP transit routers at head-end sites provide traffic-shaping with very low burst on customer EVCs terminating on ASR920s. It's not ideal, as this means I'll need -SE line card at upstream side to deal with the increased queueing requirements, but it is a decent compromise. - FAT-PW is not supported on ASR920s, and last I checked, is not even on the roadmap. ASR90x (902, 903, 907) with RSP-3C have FAT-PW support starting with Everest release SW. Long story short, ASR920 is meant to aggregate 1GE end-users. The built-in 4x10GE ports are really meant to form uplink ring architecture; not for customer facing ports by design.. I think. Lack of FAT-PW makes terminating burstable 10G customer interface for IP backhaul extremely undesirable. In conclusion, given the economical price point of the ASR920, this is really geared to go after competing vendors for packet networking/MetroE/cell backhaul of 1GE revenue ports. For the given price point, ASR920 works really well and does its job as advertised, so we are very happy with them for our use case. Anything more advanced, well I guess that's where the expensive routers come in.. James ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR920 Opinions
Stephen, We have about 20 ASR920's deployed some are 24 Port Copper and some are 24 Port Fiber Running version: asr920-universalk9_npe.03.16.04.S.155-3.S4-ext.bin Advance metro IP license with 10G Ports Core Facing ISIS+LDP+BFD,IPv4,IPv6,BGP Customer Services: Internet (Non BGP Customers), L3VPN, EoMPLS, VPLS/BridgeGroups, ENNI's, QOS Shaping and Policing We will deploy these in a 10G ring of 6 devices then 2x 10G back to our core Routers One down side is the 20K IPv4/IPv6 Route limit. So no full routes and we also place a RT Filter on our VPNv4 sessions back to the core. We did hit a bug where sometime after you upgrade the 10G ports were admin down. I forget what Image this occurred on. I am very happen with them the only down side is they don't have a 48x1G, 4x10Gport model. -Original Message- From: cisco-nsp [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Stephen Fulton Sent: Tuesday, December 19, 2017 12:38 PM To: cisco-nsp@puck.nether.net Subject: Re: [c-nsp] ASR920 Opinions Hi Jason, We're running several, primarily as PE's facing external networks, with ISIS, LDP, BGP, VPNv4, IPv6 (not 6VPE) and EoMPLS. So far, no major issues, we're running 03.16.04.S or 03.16.05.S. Core facing interfaces are IP only, not trunks attached to BDI's. My only concern up to this point is the buffer size. The PDF "Handling Microburst on Cisco ASR920" outlines steps to mitigate it but the commands do not work on the versions I'm running. It hasn't been a problem yet, but we'll see. -- Stephen On 2017-12-19 1:31 PM, Jason Lixfeld wrote: > Hey all, > > With the ME3600 EOL, we’re looking to start deploying ASR920s. These boxes > would run 100% L3 on the core facing sides (at 10 or 20Gbps), and aside from > the odd corner case, 100% L3 on the customer facing side. > > Some of the more major features they’d run would be: > ISIS > LDP > BFD > BGP-VPNv4 > BGP-VPNv6 (6VPE) > BGP Selective Route Download > IPv6* > ACL (ingress and egress)* > Per-VRF label mode > EoMPLS > FAT-PW > VRF aware DHCP Relay w/option 82 stamping (device, port (EFP?), VLAN) > VRF aware DHCP Server > > Corner cases would include BGP signalled VPLS w/BGP-AD, and l2protocol > support for peer/forward/tunnel primarily on CDP and STP-type frames, as > required. > > *ME3600s cannot support simultaneous configuration of egress ACLs and IPv6. > I’ve heard that the ASR920 resources are carved up differently, where this is > no longer a problem. > > My understanding is that the ASR920 behaves more like an ASR1000 than an > ME3600 in terms of how L2 is implemented (ie: no more global vlan table, vlan > database, etc and all EFP/bridge-domain based). Also, I understand that > these boxes have Netflow to some degree, but a cursory look at the > documentation seems to suggest that you need to set the SDM profile to video > (which affects the device scale as it re-configured the TCAM) if you want to > enable Netflow? > > I know this isn't the first time a “what are your experiences with these > boxes like?” thread has made the rounds, but I wanted to throw it out again > to see how much has changed since the last time it circulated. So, while we > wait for some of these guys for the lab, I’m looking for some feedback on > what to expect from these boxes in terms of reliability (hardware and > software), feature limitations/gotchas, a good, reliable code version, and > anything else someone might want to share about these guys, good, bad or > indifferent. > > Thanks again, in advance. > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ > ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR920 Opinions
Hi Jason, We're running several, primarily as PE's facing external networks, with ISIS, LDP, BGP, VPNv4, IPv6 (not 6VPE) and EoMPLS. So far, no major issues, we're running 03.16.04.S or 03.16.05.S. Core facing interfaces are IP only, not trunks attached to BDI's. My only concern up to this point is the buffer size. The PDF "Handling Microburst on Cisco ASR920" outlines steps to mitigate it but the commands do not work on the versions I'm running. It hasn't been a problem yet, but we'll see. -- Stephen On 2017-12-19 1:31 PM, Jason Lixfeld wrote: > Hey all, > > With the ME3600 EOL, we’re looking to start deploying ASR920s. These boxes > would run 100% L3 on the core facing sides (at 10 or 20Gbps), and aside from > the odd corner case, 100% L3 on the customer facing side. > > Some of the more major features they’d run would be: > ISIS > LDP > BFD > BGP-VPNv4 > BGP-VPNv6 (6VPE) > BGP Selective Route Download > IPv6* > ACL (ingress and egress)* > Per-VRF label mode > EoMPLS > FAT-PW > VRF aware DHCP Relay w/option 82 stamping (device, port (EFP?), VLAN) > VRF aware DHCP Server > > Corner cases would include BGP signalled VPLS w/BGP-AD, and l2protocol > support for peer/forward/tunnel primarily on CDP and STP-type frames, as > required. > > *ME3600s cannot support simultaneous configuration of egress ACLs and IPv6. > I’ve heard that the ASR920 resources are carved up differently, where this is > no longer a problem. > > My understanding is that the ASR920 behaves more like an ASR1000 than an > ME3600 in terms of how L2 is implemented (ie: no more global vlan table, vlan > database, etc and all EFP/bridge-domain based). Also, I understand that > these boxes have Netflow to some degree, but a cursory look at the > documentation seems to suggest that you need to set the SDM profile to video > (which affects the device scale as it re-configured the TCAM) if you want to > enable Netflow? > > I know this isn't the first time a “what are your experiences with these > boxes like?” thread has made the rounds, but I wanted to throw it out again > to see how much has changed since the last time it circulated. So, while we > wait for some of these guys for the lab, I’m looking for some feedback on > what to expect from these boxes in terms of reliability (hardware and > software), feature limitations/gotchas, a good, reliable code version, and > anything else someone might want to share about these guys, good, bad or > indifferent. > > Thanks again, in advance. > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ > ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] ASR920 Opinions
Hey all, With the ME3600 EOL, we’re looking to start deploying ASR920s. These boxes would run 100% L3 on the core facing sides (at 10 or 20Gbps), and aside from the odd corner case, 100% L3 on the customer facing side. Some of the more major features they’d run would be: ISIS LDP BFD BGP-VPNv4 BGP-VPNv6 (6VPE) BGP Selective Route Download IPv6* ACL (ingress and egress)* Per-VRF label mode EoMPLS FAT-PW VRF aware DHCP Relay w/option 82 stamping (device, port (EFP?), VLAN) VRF aware DHCP Server Corner cases would include BGP signalled VPLS w/BGP-AD, and l2protocol support for peer/forward/tunnel primarily on CDP and STP-type frames, as required. *ME3600s cannot support simultaneous configuration of egress ACLs and IPv6. I’ve heard that the ASR920 resources are carved up differently, where this is no longer a problem. My understanding is that the ASR920 behaves more like an ASR1000 than an ME3600 in terms of how L2 is implemented (ie: no more global vlan table, vlan database, etc and all EFP/bridge-domain based). Also, I understand that these boxes have Netflow to some degree, but a cursory look at the documentation seems to suggest that you need to set the SDM profile to video (which affects the device scale as it re-configured the TCAM) if you want to enable Netflow? I know this isn't the first time a “what are your experiences with these boxes like?” thread has made the rounds, but I wanted to throw it out again to see how much has changed since the last time it circulated. So, while we wait for some of these guys for the lab, I’m looking for some feedback on what to expect from these boxes in terms of reliability (hardware and software), feature limitations/gotchas, a good, reliable code version, and anything else someone might want to share about these guys, good, bad or indifferent. Thanks again, in advance. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
