Re: [c-nsp] Thoughts on the ASR9902?
On 10/11/24 17:52, Drew Weaver wrote: -- Yeah, it sometimes almost feels as though traditional vendors are hastening the [for lack of a nicer term] enshittification of the Internet to their own detriment in a short term vs long term sense. It has to suck for them that the aforementioned 5-6 content networks basically side stepped them entirely but we know why they did it. They are desperately clinging on to a model that does not work outside of 3 or 4 customers... the AT&T's, Verizon's, NTT's and Deutsche Telekom's of the world. The model they use for those customers does not work for the majority of the ISP market. I had a call with a Juniper employee yesterday who wanted to understand whether modern ISP's are for opex or capex models. I told him opex makes sense for small ISP's, or ISP's just starting out. But at some point (and that is arbitrary for each ISP), opex models become inferior to capex models. The problem with capex models is that vendors don't know how to do them well, and will maintain a large opex revenue base with each customer through TAC support fees, which negates any capex model gains. Like large telco's in a market of shrinking transit and DIA margins, traditional OEM vendors of old are struggling to maintain all the overhead they have accrued when the few customers buying from them are reducing their spend YoY, and the massive but low-spend customers are moving on to more sensible OEM's. But I also don't think Broadcom is here to save anyone. So we're kind of stuck. Sadly. Broadcom have certainly done their bit to democratize silicon packet forwarding. The true benefit will come when we have a multitude of well-supported, open-source NOS's that are compatible with the emerging white box vendors. Considering what is happening in the optical world with MSA, OpenROADM, OIF, OpenZR and OpenXR, options that challenge traditional models are well on their way to creaming to the top. And I don't think the IP/MPLS world shall be spared. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Thoughts on the ASR9902?
On 10/10/24 18:20, Drew Weaver via cisco-nsp wrote: Hello, We bought one and regret it mightily every single day. Ours specifically had bad memory in it, it took a year before they/we figured that out, lost our SNT over that year while it was acting insane [and we couldn't deploy it] and then even after they RMA'd the router because it was faulty the entire time we owned it we were provided an insane quote to renew our SNT. Had it 15 months now, it's routed exactly zero packets. It's basically a 8 port 100GE router, all we wanted to do was configure it for 3x100GE +10x10GE per 'slice' and that was impossible, instead they advised us that if we want 6x100GE ports we should configure the slices asymmetrically [i.e. one slice 4x100GE and the other slice 2x100GE+10x10g+10x10g] but that of course reduces redundancy if you planned on using port channels across the slices [assuming that the slices fail independently of one another which wasn't our case with the bad memory as all of the ports went down at the same time even though we were advised that the two slices were independent by TAC]. If I had it to do all over again I probably would've just purchased two Arista 30x100GE switches [which with the right model can also do full tables] for the same price as one 9902. If I couldn't do that I would probably just get whatever the smallest ASR99 is with 2x4 port 100GE line cards in it and just be done with it. The value that one used to get from buying tin with vendor silicon from Cisco and Juniper seems to be on the rapid decline. Given the ultra-high pressure on transit and DIA margins, and with the bulk of the Internet failing when the large 5 - 6 content networks have a sneeze, do we really need all the smarts these traditional vendors have been pushing, in 2024? On the one hand, I think not. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Serious Bug in Cisco's 6500 & 6800 Platforms
On 4/9/24 15:29, Gert Doering wrote: I'm so glad our single box with SUP-2T has been retired many years ago... (We still do have one (1) Sup720-10G 6500 running, but that is being migrated away from right now) You are the first person I thought about, when I saw this advisory... Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] Serious Bug in Cisco's 6500 & 6800 Platforms
https://sec.cloudapps.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-ios-dos-Hq4d3tZG Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] IOS XR filter route from OSPF?
On 11/28/23 17:02, Nick Hilliard via cisco-nsp wrote: prefix filtering is a defining feature of a policy routing protocol. OSPF is a link-state protocol, and doesn't support the concept of having different visibility of prefixes inside the same area. If you want that with OSPF, you'll need to divide your network into different areas, which is messy. Probably better off using bgp for this. Filtering in link state routing protocols is a bit of a misnomer, technically speaking... but, you can use import/export filters on routers with OSPF and IS-IS. It would not necessarily limit the LSA/LSP flooding scope, but you end up with the desired outcome (all manner of caveats apply). All that said, the usefulness of an IGP is in its homogeneous view of the network from and by all participating nodes. Bad things can happen when one partitions IGP's, especially in an unintended way. As you say, BGP is better for this kind of thing, as typically, IGP's should carry infrastructure prefixes, and you don't really want to filter those as they provide basic router-to-router connectivity. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Midpoint RSVP LSP stats
On 9/28/23 09:10, Mohammad Khalil via cisco-nsp wrote: Greetings I am looking for similar command to obtain forwarding information at the midpoint (no te interfaces) https://www.juniper.net/documentation/us/en/software/junos/mpls/topics/ref/command/show-mpls-lsp.html This is on NCS5500 therefore “traffic collector” is not supported. Is this for p2mp RSVP-TE? Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Extended Route Target Community Bug - Solved!
On 9/27/23 13:23, Nathan Ward wrote: In JunOS you can’t use regexes or wildcards for “target:” communities. You can use wildcards in IOS-XR RT sets - so if your RPL has something like the following, without defining the RTs you care about in the VRF, you’ll generate a bunch of rtfilter[1] routes. Perhaps it could optimise and generate rtfilter routes using prefix lengths other than 0 and 96 - though I’ve never actually tested that before, so I’d be wary of how well it is implemented… :-) route-policy make-lots-of-rtfilter-routes if extcommunity rt matches-any (64496:*) then pass else drop endif end-policy IOS-XR has a more complex language than JunOS for policy, too. Without the above being an issue, it feels like jumping through parameterised RPL could make it pretty difficult to figure out all the possible RTs you might want to accept. Noting that communities can be built up from parameters. Impossible? No. More difficult? Yeah. If sticking all the possible RTs in a VRF definition is the cost of getting RPL, I think that’s a fair trade. I recall IOS and IOS XE also had a number of options with which you could create RT permutations. As with IOS XR, a lot of them seem nice-to-have, for us anyway, in the context of where the Internet is today. I’m a Juniper guy mostly, but, RPL is pretty good I’ve got to admit. I think it's a bit over-the-top, but that's just me :-). -- Nathan Ward [1] I have forgotten what Cisco calls constrained route distribution. rtfilter is a JunOS term and is what I call it. From memory vendors all call it different things. RTC (Route Target Constraint), as defined in RFC 4684. I looked into implementing it back in 2008, but decided it wasn't worth the effort for the network I was engineering at the time. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Extended Route Target Community Bug - Solved!
On 9/24/23 03:43, Nathan Ward wrote: Further than that, in JunOS if you define an RT in a VRF with an export/import policy it has no effect. Import/export RT is just a shortcut for creating and applying a policy if no other policy exists. It doesn’t (so far as I am aware) do anything else behind the scenes. I have seen this catch out folks in the past, who either expected it to pre-filter like Cisco, or expected it to permit RTs additional to the policy. Agreed - the Juniper option makes more sense to me, even though I first interacted with VRF's in IOS. Cisco have always been like this, where the VRF import/export RTs are an additional layer of policy. I wonder if they see some performance benefit. My only assumption was that early versions of VRF implementation in IOS did not expect that operators would require more fine-grained use of import/export policies, and may just mostly rely on the RT defined in the VRF. Perhaps in XR RPL where it would maybe be more difficult to generate a list of expected RTs? Why would that be more difficult? It would certainly make it a lot faster to generate the list of RTs to advertise with rtfilter - though given that’s only at config commit time perhaps it’s not a big deal. I can't think of a reason why implementation in IOS XR would be more onerous. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Extended Route Target Community Bug - Solved!
So I eventually figured this out... for the router to apply the extended community on inbound routes, one has to configure the export RT in the VRF itself. Originally, I had used only import and export maps, without defining the RT explicitly in the VRF. Turns out that even if you use import and export maps for fine-grained community management, you still need to define the RT in the VRF. That sort of acts like a "first step" in telling the router what communities to allow, and then the import/export maps are the "second step" in further being granular about what communities are allowed into and out of the VRF. This documentation is nowhere in the wild that I could find, but hope it helps someone else that runs into the issue. This is different from how Junos does it, where import/export maps can be used without having to explicitly define the RT in the VRF. Mark. On 9/21/23 14:27, Mark Tinka wrote: Hi all. I have a simple inbound route-map on a VPNv4 PE-CE BGP session that does the below: route-map TEST deny 10 match rpki invalid ! route-map TEST permit 20 match ip address prefix-list test-in set metric 0 set local-preference 120 set extcommunity rt 65200:5 ! route-map TEST deny 65535 The outcome of that policy works correctly for setting MED to 0 and LOCAL_PREF to 120. However, I can't get it to set the extended RT community value to 65200:5. Nothing happens. If I update that sequence with the below... route-map TEST permit 20 match ip address prefix-list test-in set metric 0 set local-preference 120 set community 65200:5 set extcommunity rt 65200:5 ... the regular community value is applied to the route. Of course, this does not work for me since I need the extended RT community applied to the route for it work further down the core. Am I doing something wrong, or is this a bug? System is an ASR1002-X running IOS XE 17.03.04a. For completeness, doing this on Junos works flawlessly. All help appreciated. Thanks. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] Extended Route Target Community Bug
Hi all. I have a simple inbound route-map on a VPNv4 PE-CE BGP session that does the below: route-map TEST deny 10 match rpki invalid ! route-map TEST permit 20 match ip address prefix-list test-in set metric 0 set local-preference 120 set extcommunity rt 65200:5 ! route-map TEST deny 65535 The outcome of that policy works correctly for setting MED to 0 and LOCAL_PREF to 120. However, I can't get it to set the extended RT community value to 65200:5. Nothing happens. If I update that sequence with the below... route-map TEST permit 20 match ip address prefix-list test-in set metric 0 set local-preference 120 set community 65200:5 set extcommunity rt 65200:5 ... the regular community value is applied to the route. Of course, this does not work for me since I need the extended RT community applied to the route for it work further down the core. Am I doing something wrong, or is this a bug? System is an ASR1002-X running IOS XE 17.03.04a. For completeness, doing this on Junos works flawlessly. All help appreciated. Thanks. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] IOS XE BGP Add-Paths Support for VPNv4 + VPNv6 AFI's
On 8/30/23 18:24, Mark Tinka wrote: Actually, different RD's are not a solution for VRF routes leaked into the global table. It will only work for traffic carried inside the VRF domain. If IOS XE can't support Add-Paths for VPN traffic, direct iBGP sessions may be necessary to workaround this. Ouch! So this turned out to be a known limitation as described in CSCvn41817. Basically, IOS XE is unable to advertise capability for Add-Paths for VPNv4 and VPNv6 AFI's. From IOS XE 17.9.x and later, VPNv4 and VPNv6 supports the "bgp additional-paths select" and "bgp additional-paths install" features. However, neither "bgp additional-paths send receive" nor "advertise additional-paths" are supported, making it a fractured and not very useful feature. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] BGP Extended Communities
On 9/10/23 21:22, Mohammad Khalil via cisco-nsp wrote: Greetings Hope all is well. I need to check if Juniper's BGP extended community settings are compatible with Cisco's BGP extended community settings. Is it possible to intercommunicate Juniper's BGP extended community with Cisco BGP extended community ? We use them for l3vpn VRF's between both vendors. Works standard. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] "next-table" Equivalent for IOS XR - Default Route into Global Routing Table
On 9/3/23 02:05, Phil Bedard wrote: Some Junos platforms won't do this either BTW, it's somewhat dependent on the forwarding hardware. I was wondering whether anyone running Junos on a current Broadcom chip has tested this. Trio spoils us. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] IOS XE BGP Add-Paths Support for VPNv4 + VPNv6 AFI's
On 8/30/23 18:14, Mark Tinka via cisco-nsp wrote: Hi all. Does anyone have any definitive info per subject? We don't see support in our CSR1000v units, and my SE seems to have gone fishing. Anyone who has deployed Cat8000v know if there is support there? It's what we are moving to, but we aren't there yet. Using different RD's per site is a workaround, but... Actually, different RD's are not a solution for VRF routes leaked into the global table. It will only work for traffic carried inside the VRF domain. If IOS XE can't support Add-Paths for VPN traffic, direct iBGP sessions may be necessary to workaround this. Ouch! Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] IOS XE BGP Add-Paths Support for VPNv4 + VPNv6 AFI's
Hi all. Does anyone have any definitive info per subject? We don't see support in our CSR1000v units, and my SE seems to have gone fishing. Anyone who has deployed Cat8000v know if there is support there? It's what we are moving to, but we aren't there yet. Using different RD's per site is a workaround, but... Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] "next-table" Equivalent for IOS XR - Default Route into Global Routing Table
On 8/29/23 18:22, Daniël Verlouw wrote: slightly different approach, but I’ve had some success with ACL-based VRF select, but it really depends on your use-case: https://community.cisco.com/t5/service-providers-knowledge-base/asr9000-xr-abf-acl-based-forwarding/ta-p/3153403 Something like: ipv4 access-list FOO remark Don’t perform FBF on intra-VRF traffic permit ipv4 remark Forward everything else to VRF of your choice, default or non-default permit ipv4 any any nexthop1 ! int x/y/z vrf YOURVRF ipv4 access-list FOO ingress ! Also works on NCS: https://xrdocs.io/ncs5500/tutorials/acl-based-forwarding-and-object-tracking-for-ncs5xx-and-ncs55xx/ I did come across a suggestion about using ABF for this, but it immediately stood out as a 3-legged stool. If it is working for you, that's good to bank for the archives. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] "next-table" Equivalent for IOS XR - Default Route into Global Routing Table
On 8/29/23 15:17, Gert Doering wrote: So, yes, I would be interested what exactly happens inside the box, and why it does not work / how hard it would be with existing ASR9k NPUs to make it work (technically) but I expect there will be no answer on this. I didn't even bother asking our SE. One could probably find out, with enough energy, but I've given up on IOS XR... also because BGP Add-Paths setup in IOS XR is unnecessarily complicated if you compare it to IOS XE. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] "next-table" Equivalent for IOS XR - Default Route into Global Routing Table
On 8/29/23 11:40, Nathan Ward wrote: We were learning a default from an eBGP peer on the same node, so we were able to leak that in to the other VRF and get more or less what we wanted - but it wasn’t ideal. I tested the same by pointing 0/0 to another PE via the default VRF, and that worked. But the traffic was being forwarded through the remote PE, which is not sensible. I can’t recall if it would do a route lookup in the VRF that had the eBGP peer or not, I have a funny feeling it did what we wanted it do, though. From my tests, unknown destination traffic was be pulled into the global table, and sent to the remote PE configured as the default route's next hop in the VRF, which worked. I am guessing that was label switched traffic toward default route next hop. Obviously for packets coming from other PEs following that default, it would work as desired (we were running per-VRF labels). Perhaps you could experiment with that. If it does a route lookup, you could run a BGP session over a hand-bag cable so that you have an eBGP default you can leak - and in theory only traffic that doesn’t match your DFZ routes would go down that link. Messy, but, might work? Yeah, the site is remote. Don't have the energy for sexiness :-). From memory, if you create a static default and leak that, it follows wherever that default goes, and doesn’t follow the logic you would expect for |label mode per-vrf| - so if it’s a default to null, the packets get dropped. Default to a vrf with a next-hop - packets go out to that next-hop. Interesting, I always wondered what the equivalent for Junos' "vrf-table-label" was. Thanks for this :-). So yes, our default routes point to Null0. I changed that to something useful and it still didn't work. It's almost as if the traffic exiting the VRF toward the global table wanted to follow a label switched path, and not an IP-based path. Not sure whether "label mode per-vrf" would have helped to obfuscate the fact that the global table default routes pointed to Null0, but it's too late to test now. The box has been swapped out. But this is a good tip. I'll ask the next person who runs into this to update this post with their experience. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] "next-table" Equivalent for IOS XR - Default Route into Global Routing Table
On 8/29/23 12:43, Arie Vayner wrote: Would something like this work? https://learningnetwork.cisco.com/s/question/0D53i0KstGrCAJ/ios-xr-leaking-the-routes-between-vrf-and-global-rib That very thread was the last thing I tried this morning. It didn't work either. I suspected that it could be because we have 0/0 and ::/0 pointing to Null0 in our DFZ on the box. So I changed that and pointed it to something useful, and still, the default route would not import into the VRF. I then decided to walk away from IOS XR and move to Junos. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] "next-table" Equivalent for IOS XR - Default Route into Global Routing Table
On 8/29/23 11:05, Fraser McGlinn wrote: Would this be a case where vasi-left and vasi-right interfaces are appropriate? Essentially same as an LT in Junos. Not as elegant for sure, but should function. IIRC, VASI support was only on the MSB (Multi Service Blade) on the XR 12000 platform. There was no support for the ASR9000 (I was working on a rather ancient ASR9001). If someone with more ASR9000 clue knows if support is now enabled for that platform, please chime in. We do use "virtual-router" routing instances in Junos to basically make a whole new router inside an existing router. But we don't use that feature too often because it impacts ultimate PFE performance in a way that a regular "vrf" routing instance does not. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] "next-table" Equivalent for IOS XR - Default Route into Global Routing Table
Hi all.
I've been racking my brain trying to implement an equivalent feature in
IOS XR 6.7.1 similar to Junos' "next-table" feature.
Essentially, I am trying to point all unknown destinations from within a
VRF toward the local global table for resolution. In Junos, it's as easy as:
static {
route 0.0.0.0/0 next-table inet.0;
}
rib VRF.inet6.0 {
static {
route ::/0 next-table inet6.0;
}
In IOS XR, we have:
vrf VRF
address-family ipv4 unicast
0.0.0.0/0 vrf default
vrf VRF
address-family ipv6 unicast
::/0 vrf default
The Junos one works straight out of the box.
IOS XR is a bit more petulant.
And if you look at the VRF FIB, the route exists:
S* 0.0.0.0/0 [1/0] via 0.0.0.0 (nexthop in vrf default), 00:15:06
But CEF says it is a "drop", likely because there is no associated
output interface.
The most I could do was to point the static route out via "vrf default",
but to another PE device that carries the DFZ (with its Loopback address
as the target). This works well, but surely cannot be expected to be a
lasting solution, because the router on which this VRF is configured
already carries the DFZ. So why send the traffic to another PE router
and cause it to handle traffic it should not be handling?
I've given up and just swapped the IOS XR box with a Junos one, and
fixed the issue that way.
But for posterity, has anyone ran into this and came up with a working
solution, or is IOS XR just deficient in this regard? I don't understand
the point of being able to point default to the global table in a VRF,
and then not be able to actually use it. Thanks.
Mark.
___
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] add-path on XR
On 9/9/22 11:06, Sebastian Neuner via cisco-nsp wrote:
Hi all,
I got no replies and that might be because nobody cares, or it might
be because nobody knows how to do it on XR. Googling for something and
finding posts without solution is always annoying, so here's what I
found. This is all on 7.5.2.
route-policy monitoring-out
set path-selection all advertise
end-policy
=> can't be attached at the neighbor-out attach point
Possible since IOS XR 7.3.1, but convoluted. See here:
https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/217097-bgp-path-selection-in-ios-xr.html#anc16
You can use "destination" to filter for the destination prefix, but
you can also use it to match a path type (best, backup, multipath,
all), like this: "if destination is-best-external then ...". Not sure
why they chose a syntax with two meanings for "destination". Weird.
But: you can only advertise paths to neighbors that are marked for
RIB-out. You can set a policy with "set path-selection all advertise"
to mark all paths for RIB-out:
router bgp X
address-family ipv6 unicast
additional-paths send
additional-paths selection route-policy send-all-paths
As soon as a neighbor negotiates the capability (neighbor has
"receive" enabled), it will receive *all* paths by default, for all
the prefixes you advertise. To filter this, you'd have something like
this in your neighbor-out policy:
route-policy CUSTOMER-out
if destination is-all then
drop # "is-all" matches all *additional* paths, not the best-path
else
pass
endif
end-policy
Yes, very convoluted compared to IOS XE and Junos.
The only IOS XR boxes in our network are a handful of ASR9001's that are
promptly being replaced the the Juniper MX204.
I sent all available paths to those ASR9001's, equipped with 8GB of RAM,
and they ran out of ideas at 4,000,000 routes RIB.
Weird design choices all over the place IMO. But that might be just
me. Is it a Cisco-thing to always advertise everything to everyone by
default?!
Not in IOS XE.
But the IOS XR setup is a lot more convoluted, that's for sure. But it
appears to be possible to constrain this with some clever RPL work.
So what if you want to advertise add-paths only to specific peers? I
know, I know, very rare use case.
The BCP makes this a requirement, even if it may be rare.
You can advertise the capability to all neighbors and filter
("destination is-all") in all neighbor-out policies where you don't
want it. But that's a lot of work and changes, it's complex, and error
prone.
Agreed.
The good news is: you can send the capability to specific neighbors
and don't have to change filters for the others. But not how you'd think.
So, if we can enable it globally under the address-family, we might
enable it for just one neighbor? What do you think this does?
router bgp X
neighbor X
address-family ipv6 unicast
additional-paths send
Wrong, this doesn't exist.
Sooo, the next one exists, but what does this do?
router bgp X
neighbor X
capability additional-paths send
Wrong, it doesn't do anything. At least in my lab it did nothing.
Maybe I held it wrong? Maybe it's a bug? Idk.
router bgp
neighbor X
capability additional-paths send disable
Now this one disables advertising the capability for this neighbor and
address-family... idk? All? Probably all.
So you can globally enable "additional-paths send" and disable it for
every neighbor (or in a neighbor-group, which might be convenient).
All in all this one-liner in JunOS translates to something like that
on my IOS XR boxes:
router bgp X
address-family ipv4 unicast
additional-paths send
additional-paths selection route-policy add-path-selection
address-family ipv6 unicast
additional-paths send
additional-paths selection route-policy add-path-selection
neighbor-group ibgp
capability additional-paths send disable
neighbor-group ebgp
capability additional-paths send disable
neighbor X
! no "use neighbor-group ibgp" here!
description Looking Glass
route-policy add-path-selection
set path-selection all advertise
end-policy
To find all this info, I needed to read the documentation, which is
pretty much non-existent, open two TAC cases, read a troubleshooting
guide[1] which is actually documentation and not a troubleshooting guide.
IMO several weird design choices combined with a lack of documentation
makes this pretty frustrating. The good news is: nothing of this will
reset neighbor sessions and you can manually clear them to renegotiate
capabilities whenever.
Hope this saves someone a headache.
How did you get on, in the end?
Mark.
___
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] add-path on XR
Very old thread, but I was digging around and found it, so thought I'd answer, in case no one did: On 5/10/22 13:27, Sebastian Neuner wrote: But on IOS XR, I can only find global options to enable the capability and set a general limit for the number of paths, like this: router bgp 65000 address-family ipv6 unicast additional-paths send maximum-paths ibgp 6 "maximum-paths" only applies to Multipath, not Add-Paths. So that looks like it enables the capability for all iBGP peers, and in the process resets all sessions? (yay, fun) Add-Paths is a non-default capability. If it has not been previously enabled, the session is torn down so it can be negotiated with the neighbor. But there is no configuration per neighbor. Am I missing something? To support the capability, that is at the AF level. After IOS XR 7.3.1, per-neighbor level is supported for path selection. But the capability remains at the AF level, AFAICT. There are some knobs in the route-policy. I found stuff like route-policy monitoring-out set path-selection all advertise But how does that work? Do all iBGP neighbors without "path-selection" options in their out-policy continue to only receive the best path? And the monitoring-servers receive "maximum-paths"-many paths? It means that for any neighbors who are advertising their ability to receive additional paths, you will be able to send them all available paths with that policy. Would configuring that still reset all iBGP neighbors? Or just the ones where the policy sets "path-selection" parameters? No. Path selection policies and their advertisement does not reset BGP sessions. BGP sessions are only reset when you need to negotiate Add-Path capability exchange. And why is the documentation for those products of the last few years in general such a catastrophe? I'm guessing most people don't use Add-Paths as much as they do Multipath, so perhaps why. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] BGP Routes
On 3/12/23 20:21, Mohammad Khalil via cisco-nsp wrote: Greetings I have two ASR9K connected to different providers (Uplinks). I am receiving around 90K routes from each provider , as well , I have iBGP between the ASR9K. What am noticing is that ASR9K1 is advertising around 87K to ASR9K2 where ASR9Ks is advertising around 7K routes. Any hints? A case of active routes being announced to neighbors, where active routes = best routes/paths as seen from each router's point of view. ASR9K1 has more routes with better paths toward destinations via its upstream than ASR9K2 does. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] NCS IOS-XR rant (was:Re: Internet border router recommendations and experiences)
On 3/1/23 10:04, Saku Ytti wrote: There are two paths that consumers would accept a) immutable NOS, you give it image, it boots up and converges in <5min b) mutable NOS, process restarts keep state, if upgrade is hitful, forwarding stoppage should be measured in low seconds I think a) is far easier to achieve. I prefer a), which is why I was never in favour of ISSU, despite all the marketing that was as "promising" as IPoDWDM... The fancier NOS management tries to get, the more we realize we got it right the first time, way back then. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] NCS IOS-XR rant (was:Re: Internet border router recommendations and experiences)
On 2/26/23 16:44, Tarko Tikan via cisco-nsp wrote: Well, not so in practice. You can't issue install from http:// or any other remote URL. You have to sit around and issue "install apply" after "install replace" is finished. Replace is async so you have to sit around and poll the process. After reboot you have to reconnect to device and issue "install commit". In some cases direct upgrades from version X to Y fail so you have to go through this whole process twice (X to Z to Y) that takes around 2 hours on NCS540. In some other X to Y cases there is not sufficient diskspace to complete "install replace". We personally have automated the whole install process via netconf and can workaround the quirks relevant for our platforms and versions. Many people can't do that or can't justify the expense (when they have small number of devices). Some other issues have been solved by Cisco in latest releases, I belive install replace can now be sync operation, maybe not on NCS540 but on larger platforms (IOS-XR consistency between platforms is an issue itself). So I totally get what Mark and Gert are saying. IOS-XR is currently worst NOS operational experience from all large NOSes out there. Oh gosh - it's such a shame that it's 2023 and we still have to put up with shoddy software maintenance processes, just because a vendor insists that their next generation OS core is worth the daily-use pain. I could be okay with doing for this for about 10 - 20 nodes in the core. But even with some level of automation (because you have to baby-sit the automation, especially when the vendor changes things in a bid to "improve" life with their OS), trying to manage this on 100's - 1,000's of nodes in the Metro (or anywhere, really) is just too much of a nightmare. So you either end up with network gear running very old code because operators can't be asked to spend 2hrs on upgrading a single device, or simply tying up too many engineer hours at the expense of other projects. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/26/23 16:29, Phil Bedard wrote: SMUs were a good idea, but not really great in practice. Most customers I work with do not want to manage application level patches, just entire images, even in cases where they are just a process restart. XR for a number of years now has had the concept of a “golden ISO”. It’s a single image either built by Cisco or customers can build their own that include the base software and the SMUs in a single image. You just issue a single “install replace myiso.iso” and that’s it. I did not know that. But then again, we haven't used IOS XR platforms in a while, because we got put off. Basically, Cisco got this wrong the first time, took advice on what operators wanted to make it better, but fumbled still. We moved on. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/26/23 16:21, Phil Bedard wrote: Ok well there are a number those as well. The 55A2 and newer 57C3 also support a number of 100G ports. I quite don’t fully understand the “verbose architecture” comment. I’ve used a lot of router operating systems, Junos since 1999, SROS, XR, XE, you name it, and there isn’t a whole lot of difference between them in terms of configuration complexity and operations. Obviously some just don’t have the feature set others do, but if you aren’t using the features then it doesn’t really matter. There are at this point tens of thousands of NCS 540s deployed in that types of role, so I’m a bit curious if there was something specific in the config or other operations that was a show stopper issue? It's two things specifically for us - RPL construction in IOS XR can be done in Junos for half the number of lines to achieve the same outcome, without losing sophistication. Secondly, maintaining IOS XR (upgrades and SMU's) is too tedious. They may seem like trivial points, but for us, they mean a lot. It's why we still prefer IOS XE (by way of the CSR1000v) as a route reflector vs. Junos or IOS XR. IOS XE is far less verbose than the other two, in that role. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/24/23 19:51, Lukas Tribus via cisco-nsp wrote: Hello, for the unititiated, how does the licensing on a mx204 look like for different or combined use-cases like pure IP edge, mpls layer3 and layer2 VPNs, BNG functionality? IIRC, BNG deployments support up to 1,000 concurrent subscribers by default. Anything more requires a license that should be purchased and activated on the router. For all other non-BNG features, the license is honour-based, and may get enforced during a TAC call. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/24/23 11:01, Gert Doering wrote:
I really do like XR, but the update hassles... so having an "image based"
XR ("scp $new_xr.bin router:", "boot system flash $new_xr.bin", "reload")
would have been really nice.
Now, SMUs and "restart only the affected service" is a great promise, but
in all our time with the ASR9001, all we've seen is "reboot required"
or "the SMU is not compatible with using service packs". So, "just upload
a new image, and then reload" would have had the same effect, with less
argueing with the box.
This.
Which I don't mind in the data centre, because it's a few boxes looking
after tons of traffic.
But in the Metro, where you have 100's - 1000's of boxes, this gets very
painful, very quickly. That and RPL, despite its flexibility, can get
rather rowdy in high-touch scenarios like the Metro.
Copy, save, reboot, is very attractive.
This is why we rejected the NCS540.
Not sure XR64 is better in that regard, no experience - we lost trust in
Cisco before the question of "successor to the 9001? something with XR64?"
arose.
We stopped keeping track.
Mark.
___
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/23/23 21:45, Shawn L via cisco-nsp wrote: That's one of the major reasons we're sticking with the ASR920 in metro deployments for all it's faults. They do silly license stuff on the 12SZ (no bulk, make all the 10G ports work license) but once you figure out their quirks they do work quite well. We did just receive a 9901 (purchased 6 months ago). It seems nice but again, licensing. Want to put more than 120G worth of optics, add a license. And reboot. Really, reboot? That just seems silly in this day and age. Exactly - the Metro will usually see 100's - 1000's of devices. IOS XE is nice and simple for such applications. In fact, Junos too. For IOS XR, it's just too heavy for that sort of thing. Okay in the data centre where we are aggregating a ton of customers and/or Metro-E rings, but not out in the Metro. The Metro calls for a more agile OS. There are simply way too many devices to be dealing with the issue you mention, updating SMU's, rebooting, e.t.c., just to get a functionality and/or a bug fix from IOS XR. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/23/23 21:34, Phil Bedard wrote: The original question was around an Internet border router with 10G support. We have devices like the 55A2-MOD-SE which is similar to some other vendor devices (somewhat of a reference Broadcom design) which we’ve seen be very popular in border router deployments where you do not need a ton of bandwidth. I think the OP came back to clarify that they need a 100Gbps-based router. XRd runs in a container with very little memory, it doesn’t always have to be “fat”. In fact some of the smaller 540 systems have very little RP memory. Not so much the memory footprint of the OS, but really, it's rather verbose architecture for high-touch areas like the Metro, for which the NCS540 was to replace the ASR920. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/23/23 19:20, Brian Turnbow wrote: They also seem to want to follow the same route in metro with the NCS540s and this global bandwidth licensing bucket. You want to turn up 2x100 and 24*10 on a box? Buy 44 "essential right to use v1 for 10g" and all the shabangs that come with it that renew every 3 years... Not so low cost anymore. They sold/sell warehouses full of MEs/asr920s to providers yet seem to want to alienate the market ... A shame Apart from IOS XR being such a fat OS for us in the Metro, it's one of the many reasons we rejected their offer to swap out the ASR920 with the NCS540. Cisco have lost the plot, IMHO. Every solution at every level of the network is now a bulldozer searching for a tiny nail to hammer. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/23/23 14:12, Alexandr Gurbo wrote: For 10g speeds the best solution is a linux box and a contract with an anti ddos partner. Or even a server with a hypervisor running, say, CSR1000v or vMX or vSR will do nicely. A little pricier than Linux, but likely worth it if you have a decent server and are realistic about your traffic-handling capabilities. All announced Juniper MX series, Cisco ASR1k or IOS XR 9k series are very expensive for the initial request. Not to mention about price on licensing, spare parts and RMA contracts from the vendor. I'd throw Nokia and Arista in there, and maybe even Arrcus, as well as consider some of their Broadcom boxes too, but only if your needs are mainly hauling traffic, and not advanced packet manipulation. If you want known vendor, try review they old unsupported models from the second hands. Plenty options there, but only for old gear. MX204's, ASR9000's, even modern ASR1000's, are not readily available on the open market. And if they are, as we have found, they are similarly priced as buying from the OEM directly. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/23/23 13:47, Gert Doering wrote: Basically they have "fixed" that by making the ASR9901/9902/9903 even more expensive. And hence, why we consider other vendors. I mean, the general rule for networking today, is Ethernet. Even in some of the most far-flung regions of the world, one would be hard-pressed to find TDM/PDH/SDH/SONET in any meaningful degree of presence. So if Cisco price themselves out of the market with their flagship Ethernet box - the ASR9000 - that just makes it easier for customers to consider Juniper, Arista, Nokia, e.t.c. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/23/23 08:22, Hank Nussbacher via cisco-nsp wrote: For an ASR9906 to add 4x port 100G here is the GPL pricing: Part Number Description Unit List Price A99-4HG-FLEX-TR= ASR 9900 400GE Packet Transport Combo Line Card - 5th Gen 271,493.78 CON-SNT-A994HGFT SNTC-8X5XNBD ASR 9900 400GE Packet Transport Combo Li 87,210.25 QSFP-100G-LR4-S 100GBASE LR4 QSFP Transceiver, LC, 10km over SMF 35,388.85 $400K GPL with 8x5xNBD support. Price for LR4 is $35K - so the $400K pricing is just for 1x LR4. Very pricey. Which is why we just focus on Juniper and Arista right now. Cisco are still living in the pre-Covid era. Those good ol' days are gone, and unless you have the clout to command proper discounts from Cisco, you are losing out on better efficiencies with other vendors. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/23/23 08:15, Hank Nussbacher via cisco-nsp wrote: A fully licensed asr1001-hx (all 8 10G ports operational) w/ 5 years Cisco Smartnet support - GPL is around $220K. Add your discount here. Cheap is relative. The ASR1000 platforms are pretty sexy, but Cisco have out-priced themselves from that market. The issue they face is Ethernet-centric platforms are much more optimized for today's Internet, and platforms like the ASR1000 simply don't make sense anymore. Why pay all that to get some Ethernet on an ASR1000 when an MX240 or an ASR9000 is around? Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/23/23 01:06, Thomas Scott wrote: Yes - 400 Gbps throughput total If I recall correctly. That's right - it's basically an MPC7E line card with a-third of the capacity, i.e., 1x 3rd generation Trio chip (Eagle). Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/23/23 00:19, Eric Louie wrote: Oh geez, I just realized I left a zero off the interface - we need 100G interfaces both upstream (x1) and downstream (x2) That probably changes the product choices a little bit. Anyone with 100G Internet feeds want to let me know what you're using for a border router? I saw one reply for Arista already. Does the MX204 have 100GE interfaces and throughput? For 100Gbps peering and transit, we have moved way from the MX480 to the MX204. This makes sense for us because we separate peering and transit, and you don't need a massive chassis to handle all of this if you peer or pick up transit in 2 or more locations. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/22/23 20:29, Eric Louie wrote: Mark, thanks. We were quoted a MX304 for the Internet edge from Juniper. How has your experience been with it? are you 10G upstream and downstream? Any IPS on the 10G connection? The MX304 is not worth the money, for as long as the MX204 exists. We tried an NCS-5501 and it was a disaster, in a word. The 10G interface, uRPF, source-based blackholing, and routing table depth with Cisco is a limiting factor in their product line. Broadcom-based systems should always be looked at with one eye open, i.e., test test test before you commit. This applies to any vendor, not just Cisco. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Internet border router recommendations and experiences
On 2/22/23 05:31, Eric Louie via cisco-nsp wrote: Hi folks Recommendations and your experiences with an Internet border router for a 10G Internet connection, with DDoS service and unicast reverse path forwarding. Brand and model requested, if you have it, and bad experiences are ok, too. Likely to be blasphemous, but we are focusing on the Juniper MX204 for this type of function, going forward. On the Cisco side, I think the ASR9902 might be the closest competitor... but unless things have "improved", Cisco's latest licensing structure is rather bitter. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] How can one escalate within Cisco TAC?
On 2/13/23 01:13, Sander Steffann wrote: It makes me sad when I notice that all of the specialists on certain topics are even older than me :( A lot of us learned on the job when the internet was less critical infrastructure and mistakes were part of the learning process. These days a lot of experience is getting lost, and the industry hasn’t found a way to transfer that knowledge to new generations. The focus on "automation" and turning the network into software has created the perfect condition where the next generation of kids that need to take over from us are skipping 10 - 15 years of experience gains. And we have all seen automation fail to pieces, on a global scale, more often than we would like, and a lack of knowledge on the bare basics delaying restoration. It's worse in the training space, because a lot of the instructors who have the protein the kids need are old and/or retiring. Who is going to take over from them? These are the concerns that panel is looking to draw out. I have nothing against automation, or whatever an operator uses to define the easing of repetitive tasks. But it should be complimentary to the basics, and not a replacement of them. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] How can one escalate within Cisco TAC?
On 2/9/23 09:01, Joe Maimon wrote: Effective human capability redundancy does not persist as a stable status inside of any discreet organization. Tell that to HR departments that think "institutionalizing" skilled labour is a practical thing beyond the paper the policy is written on. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] How can one escalate within Cisco TAC?
For those going to Manila for this year's APRICOT meeting, I will be part of a panel that is discussing this very issue - about the dwindling talent pool as it pertains to those with the hard skills, that were able to train-up the the next generation of network engineers: https://2023.apricot.net/program/schedule/#/day/11/panel-discussion-internet-operations-talent-drain While it is focused on the ongoing battle between the ISP and content communities, it has not spared the talent situation in the vendor community either, as this thread is clearly exposing. Mark. On 2/8/23 19:22, Mario Ruiz via cisco-nsp wrote: Yes miss the old days On Wed, Feb 8, 2023 at 12:21 PM Hank Nussbacher via cisco-nsp < cisco-nsp@puck.nether.net> wrote: On 08/02/2023 15:27, Mark Tinka via cisco-nsp wrote: On 2/8/23 10:23, Saku Ytti via cisco-nsp wrote: Working would be much more pleasurable if half the world's white collar workers wouldn't be unemployed plat card holders and cruising without output, while looking down on people doing 3 jobs and not qualifying for a mortgage. Sadly, as folk move up in career, title, status and income, they tend to become less useful on a real, practical, rubber-meets-the-road level. Which, in all fairness, I would be okay with if they had a team that made them look good. But in most cases, they don't even have that, or if they do, find a proper way to muck that up as well. It's a general issue - not to pick only on Cisco. Ah the days when a post on cisco-nsp or nanog would get Tony Li answering with a detailed solution. I'm getting old :-) Thanks to all for your recommendations. -Hank ___ cisco-nsp mailing listcisco-...@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive athttp://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] How can one escalate within Cisco TAC?
On 2/8/23 16:45, Aaron wrote: i think the problem is they let the good ones go. That is a trend currently affecting our industry - mostly because our group has converged on the basics of a well-built platform, and "automation" is causing exec's to think they don't need the hard skills anymore. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] How can one escalate within Cisco TAC?
On 2/8/23 09:48, Hank Nussbacher via cisco-nsp wrote: We opened a case on Jan 22 (Case #694936467). Since then we have exchanged countless email, countless logs and countless command output captures. On Jan 31 we requested transfer to a more senior IOS-XR team. The case was transferred to Mexico TAC on Jan 31 and was assigned an engineer, yet after 9 days we have not heard from anyone inside Cisco TAC. The case is listed as moderate - we requested that the case be moved to Amsterdam on Feb 5 and as of today no Cisco engineer is assigned to the case, no engineer manager is listed and it would appear that after 9 days in TAC limbo, no one wants to touch this TAC case since they have run out of ideas of how to solve it. So how does one escalate such an issue within TAC? Is there some secret email like escalati...@cisco.com or vp-...@cisco.com that one can contact? Your account team at Cisco are your best bet. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] How can one escalate within Cisco TAC?
On 2/8/23 10:23, Saku Ytti via cisco-nsp wrote: Working would be much more pleasurable if half the world's white collar workers wouldn't be unemployed plat card holders and cruising without output, while looking down on people doing 3 jobs and not qualifying for a mortgage. Sadly, as folk move up in career, title, status and income, they tend to become less useful on a real, practical, rubber-meets-the-road level. Which, in all fairness, I would be okay with if they had a team that made them look good. But in most cases, they don't even have that, or if they do, find a proper way to muck that up as well. It's a general issue - not to pick only on Cisco. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Best Practices for Transporting Layer-2 Services
On 1/14/23 04:40, Tom Hill via cisco-nsp wrote: The normal answer in Cisco land, even today, is to use Martini-draft P2P pseudowires (either tag or port-based MPLS interconnects) which will use tLDP for establishment, and should serve you very well (especially at a port-based level) for a P2P service. In theory tLDP could run in concert with SR-MPLS, but you might need to think carefully about label allocation, or... [read on] ... use BGP EVPN, and pay very careful attention to the port security options (e.g. bpduguard, BUM rate-limits) as well as the ARP/ND sponging/proxy facilities therein. For multipoint L2VPN, this should be replacing VPLS now. Realistically though, protection from storms is hardware dependent, and making sure that the config is correct is only half of the battle. I would consider not building L2VPNs for third parties where you don't control the CE, realistically. Do they really need L2? Tend to agree. We use Martini pw's in our network too. We have stayed away from VPLS and EVPN, as we find out the most customers can accomplish complex p2mp or mp2mp via IP instead of Ethernet. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] v6 vrrp
On 7/15/22 21:16, Charles Sprickman wrote: If you’re not looking for any new features from IOS and simply want to have a secure/patched version, is there any option at all to park in XE and stay there? There’s a handful of these that have become pretty dumb big routers w/very simple BGP and not much else due to the world moving to metro-e as the go-to access option. From what I can see, 3.16.9S(MD) is the latest release from that era. It is from March of 2019, though. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] v6 vrrp
On 7/15/22 07:56, Saku Ytti wrote: Juniper is coming up with licensing but have strategically decided not to do technical enforcement. I am not against licensing wholesale, but I want it to be a commercial problem, not a technical one. I'm fine calling home and reporting non-compliance. Agreed, I also like the Juniper model. The licensing is more commercial and support, than technical (except for PPPoE session scaling in BNG applications, IIRC). But like Charles, we are also doing less and less work on Cisco. The last time we did an IOS XE upgrade on the ASR1002-X and ASR1006, it was anywhere between 21 - 42 steps to move from 3.x to 17.x :-(. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] v6 vrrp
On 7/10/22 11:56, Randy Bush wrote: The standard states that the first address in VRRP v3 IPv6 needs to be an IPv6 link-local address. https://datatracker.ietf.org/doc/html/rfc5798 yup. but as saku says, both xr and junos create the link local automagically. and that is what i had in a differen pop. so i mistakenly assumed xe would do the dirty. my error. As we've seen many times before, IOS XE and IOS XR may, just as well, be from two very different companies. We've hit a few issues with VRRP for IPv6, in the past, where it would simply stop working for no apparent reason, while VRRP for IPv4 is working just fine, on the same interface. In the end, a reboot of the router fixed the issue. Seems to be a hardware programming issue, that is very intermittent. Look out for this, on the ASR1000 family. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] v6 vrrp
On 7/10/22 11:56, Randy Bush wrote: The standard states that the first address in VRRP v3 IPv6 needs to be an IPv6 link-local address. https://datatracker.ietf.org/doc/html/rfc5798 yup. but as saku says, both xr and junos create the link local automagically. and that is what i had in a differen pop. so i mistakenly assumed xe would do the dirty. my error. As we've seen many times before, IOS XE and IOS XR may, just as well, be from two very different companies. We've hit a few issues with VRRP for IPv6, in the past, where it would simply stop working for no apparent reason, while VRRP for IPv4 is working just fine, on the same interface. In the end, a reboot of the router fixed the issue. Seems to be a hardware programming issue, that is very intermittent. Look out for this, on the ASR1000 family. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Link down affecting BGP peer
On 5/6/22 07:04, Ted Pelas Johansson wrote: Try configuring MicroBFD/BoB rather than using Logic bundle/BLB, see more details at https://community.cisco.com/t5/service-providers-blogs/bfd-over-logical-bundle-blb-implementation-on-ncs5500-platforms/ba-p/3309345 Is this a p2p or LAN? Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Link down affecting BGP peer
On 5/5/22 18:04, Hank Nussbacher wrote: What could be causing the bgp peer to flap even though the LAG stays up? Could be that the session flows are hashed to the 1 member link that you fail. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] BFD not working on ASR920
On 4/5/22 20:00, Bryan Holloway wrote: Curious to hear what other options you are considering? We're also looking for alternatives ... ACX7k... Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] BFD not working on ASR920
On 4/3/22 21:25, Shawn L wrote: That's a big no on NAT. We're still deploying them on the edge, feeding customer facing FTTH and DSL customers. At least they support EIGRP (yes we still use that) and MPLS. I really wish they would do some more development for the platform, or at least fix all the 'wierd-ness' on them. They won't. If you want a better ASR920, Cisco will force you on to the NCS540 train. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] BFD not working on ASR920
On 4/3/22 20:30, Gert Doering wrote: Netflow is sort of semi-supported, if I remember right - by using the SPAN feature of the chip to siphon traffic off to the CPU, and do netflow there, capped to 1GE of traffic. Or something like that. Did not try NAT or PPPoE on that box... but I'm sure there is excitement. (Speaking of SPAN - trying to debug BGP using a local SPAM is a real adventure, as packets sent by the local CPU are not seen on a RX/TX mirror session... only the RSTs coming in from the other end gave a clue what happened) We are finding the ASR920 has reached its usefulness for us. We are looking at other options now; and yes, from me, that also includes boxes shipping with Broadcom :-\... Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] BFD not working on ASR920
On 4/3/22 12:01, Gert Doering wrote: ... all the "down and we cannot find a reason" BFD sessions came up the moment I removed the "ip flow ingress" from the interface config. IIRC, isn't Netflow support on the ASR920 somewhere between suspect to non-existent? Or was that NAT? Or was that PPPoE? Thanks for the tip. I'm sure it will be helpful for many! Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR9902 experiences?
On 3/10/22 16:01, Drew Weaver wrote: Has anyone that has operated an ASR9010/9001, etc ever used an NCS system before? Our Cisco rep is sort of steering us towards NCS 55A1-24H as an alternative to the ASR9902 and it runs IOS XR but is this one of those circumstances where the OS operates totally differently depending on which hardware it's running on? (Like the entire Nexus line does with NXOS unfortunately) They are completely different platforms, NPU-wise. The ASR9902 is Cisco's in-house silicon, while the NCS 55A1 is based on Jericho+. So the OS is the same, but behaviour may differ due to this, depending on your use-case. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] C8200/Spitfire/Pacific
On 3/6/22 09:41, Saku Ytti wrote: Yellow, The box has been out for quite some time now, but I've not heard much from the community. I don't even know anyone else but 1299 who operate it. I'd very much like to hear from anyone who is running the device in production about their experience with it, even if the experience is just 'i configured it, we run features xyz, seems to work'. Or if you specifically decided not to run it, why not? I'm equally curious that no one has really spoken about it in the wild. Might it go the way of the NCS6000? Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] LSR platforms
On 12/10/20 20:28, aar...@gvtc.com wrote: BTW, on mpls agg edge we had Cisco ME3600's for years too, pretty decent, but out grew them...likewise, went with Juniper ACX5048 there. I'd probably consider the ACX5448 or even more so the MX204 for that smaller edge at this point, as I don't think the ACX5048 integrates routing into MPLS L2 services... where as I was able to accomplish that on ME3600's and I think MX204 can too. (I think even the ACX5448 can)... but we are able to do lots of L3VPN on the ACX5048 for residential bb, and cell back haul pw's ... tons. There are "better things" coming from Juniper in the Broadcom space for Metro, in the coming months. They even have my attention... Stay tuned :-)... Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] LSR platforms
On 12/14/20 15:28, adamv0...@netconsultings.com wrote: But another view point might be, With merchant silicon it's all about who has the biggest buying power (bigger lobby) right? For example: cisco says we need the capability to implement feature X in a single pass through the new chip, while juniper and arista says we need it in two passes via recirc. to spare chip resources for feature Z. -well in this example it's tough luck for cisco. And similarly I imagine this applies to all levels of the stack from actual chip architecture all the way to SDKs. If Cisco says screw this I'm going to design my own chip and always going o prioritize my own needs then consider needs of others (to an extent obviously), then I could see how Cisco box for role A powered by Cisco's own merchant silicon chip will either perform better or have richer feature set than a Cisco box for the same role A but powered by a comparable Broadcom chip. For Cisco, Juniper and Nokia, Broadcom is a "me too" move, so they can put a foot in the door (and hopefully, sell you long enough to the point where you buy their own silicon). For Arista, Arrcus, e.t.c., it's their bread & butter. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] LSR platforms
On 12/10/20 20:02, James Mitchell wrote: What hardware platforms are operators running as P routers for smaller MPLS networks? I’m not interested in large CRS type platforms, but simply an LSR thats main function is MPLS switching at 10/40/100G speeds. Preferably Cisco. Anyone have a recommendation based on experience? Old thread, but somehow, missed it. We dumped our CRS-X's (8-slot) for the Juniper PTX1000 (2U box). We realized that we didn't really need all of those fan trays, power supplies, control planes, fabrics, line cards, forwarding cards, and half a rack, just to push a serious amount of traffic. It helps that one doesn't also need a chassis, nowadays, to carry a good number of 100Gbps ports. Sure, Cisco probably has something of similar size and stature as the PTX1000, but given their new path, we couldn't be asked. That said, as others also did, there are plenty of boxes carrying Broadcom's J2 that will do the job, especially if you relegate it to just a P function, and don't try to get funky with edge features. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] NCS-5501 - EVPN L2VPN BVI mac-address weirdness
On 12/23/21 13:41, Tarko Tikan wrote: Which feature? Egress policing. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] NCS-5501 - EVPN L2VPN BVI mac-address weirdness
On 12/21/21 09:26, James Bensley wrote: Hi Drikus, Did you ever resolve this? We saw issues with MAC addresses on NCS55K too, but not related to EVPN. For example, one can use the commands 'interface foo; mac .." to set a custom MAC on a physical interface, XR commits the config but on these Broadcom chips it doesn't actually do anything. CLI output shows the custom MAC but a packet capture shows the BIA. We had another issue with the MAC address of CDP/LLDP frames addresses on bundle interface (I can't remember the exact details right now, but I think the source MAC address of these frames sometimes had the logical bundle MAC and sometimes had the member link MAC, and it was inconsistent). So it seems these chips have issues with MAC address consistency. I'm wondering if there is some relation to what you are seeing. The joys of merchant silicon - completely unpredictable; especially on platforms that you are used to being so. Given the rising(?) cost of hardware, and the declining revenue from customers, merchant silicon is only going to become more and more relevant. We can't still be playing these games at this stage in the game. I have lowered my guard, a little, and am now willing to test some boxes from traditional vendors (Juniper + Nokia) that are shipping on the back of Broadcom J2 chips. But I've already had to dump Nokia because they say they won't support a chip-restricted feature which Juniper claim they will. I don't know who's lying, or telling the half-truth. So much confusion, in this space. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/22/21 14:10, Robert Hass wrote: We will keep our ASR 9001 until support will expire, but for small Edge nodes. Well it is hard to trust Cisco currently. I can recall our CSR 1000V story (permament licenses). CSR 1000V permament licenses are EoL/EoS. But subscription CSR 1000V are not, so you need to pay again for something you already paid for. Next move was Catalyst 8000V which is CSR 1000V but with just new name, How to deceive a customer who bought licenses? 1. Release a newer version under a newer name (Catalyst 8000V). 2. Retiring the previous product/version - EoS/EoL - CSR 1000V. 3. By that you simply stop discussion regarding perpetual -> subscrption model change 4. Done. Dear customers - pay again for same piece of software! Oh dear, sorry to hear that. But that's exactly the kind of nonsense we are running away from Cisco for. We love the CSR1000v, but we are not running the latest & greatest. We use them primarily for iBGP route reflection, and have settled on 15.5(3)S9 - a.k.a. 3.16(09)S - for some time now, even though it shipped in March of 2019. It has all the relevant BGP features we need now, and into the future, so we aren't worried about running old code. We did attempt upgrading them to Amsterdam - 17.3(4a) - but that was as cluster-f* of note. First, due to some issue between later versions of CSR1000v (16.x and higher) and how VMware identifies interface drivers, you lose the ability to set Jumbo frames, even if the host supports it. This totally broke iBGP sessions because IS-IS was confused, without looking confused. Secondly, the license drama went to hell, same way you describe. But what I didn't know is that Cisco make you pay again for it; what horror! I just assumed they'd honour your previous permanent license, and port you over. Thanks for that! Anyway, we downgraded back to what we are running today, and are happy. Our perpetual licenses are still valid, for the code we run. If you need the newer code, I'd recommend taking a hard look at vMX or vSR, just for a better account management situation, if nothing else. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] RPKI-Based Policy Without Route Refresh
Randy will be presenting draft-ymbk-sidrops-rov-no-rr during RIPE-83, at around 1530hrs UTC: https://datatracker.ietf.org/doc/html/draft-ymbk-sidrops-rov-no-rr-02 Most grateful if you can join, and provide some initial feedback. Thanks. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/14/21 08:58, Saku Ytti wrote: I don't think IETF is making standards for specific implementation. The implementation agnostic solution is to keep all routes which we rejected due to consulting validation database, regardless of validation state. Agreed, and covered in the draft: https://datatracker.ietf.org/doc/html/draft-ymbk-sidrops-rov-no-rr-02 Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/13/21 17:20, Saku Ytti wrote: I chose my words carefully when I said 'RPKI rejects', instead of 'invalid'. Well, this only really happens on IOS XE since Cisco apply policy by default. On IOS XR, you'll need 'bgp bestpath origin-as allow invalid' for Invalids not to be automatically dropped. The problem only cursorily relates to a specific RPKI validation state. We may reject RPKI 'unknown', we may even imagine policies which reject based on some criteria AND RPKI 'valid' (maybe I have my own motivations for how I use VRP and want to capitalise on all three states arbitrarily, maybe I'm rejecting valids, because I'm collecting invalids to some separate RIB for research purposes). And that is all fine, provided YOU, as the operator, are deciding policy. The problem is that Cisco seem to want to automatically apply policy, particularly on IOS XE. We've hounded them about this since 2015, and nothing has changed. IOS XR is a little better in this specific regard, but not by much when compared against Junos. soft-reconfiguration inbound rpki ## default, keep if policy rejected route while using validation database state (may have used something else, but as long as reject policy used validation state, regardless of state, we need to keep it). This is what we are trying to write the RFC for - to decouple the historical need to keep or drop Adj-RIB-In from the operational requirements of RTR dynamics, i.e., leverage the value of Route Refresh to its fullest extent. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 16:49, Gert Doering wrote: Not exactly displaying best of software quality, but more of a minor nuisance (for us). Fundamentally, we just need to provide a lot more feedback to router vendors as well as validator coders, about real world experiences. Especially now since, really, there are effectively just 3 - 4 validators in the wild: https://docs.google.com/spreadsheets/d/1uuDlO6g1DLATV5OVCa20kU9OOiX9XWBFoZT2OkOezi8/edit#gid=0 Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 16:33, Lukas Tribus wrote: I believe with the amount of RAM we have on those boxes nowadays, keeping a copy of everything should be a non-issue. On the other hand, leveraging route-refresh changes your EBGP behaviour, which can trigger remote and local bugs, or, as in your case, trigger humans with most likely a little over-dramatic monitoring. I won't trust other peoples BGP routers and implementations more than I absolutely have to and I don't think my time is well spent arguing with other people about their underdimensioned control plane CPU, oversensitive CPU load monitoring or troubleshooting corner cases in their BGP implementation that trigger bugs in route refresh code. And then the need to explain in a RFO why your network heavily uses route-refresh which triggered that remote bug in the first place, while your competitor didn't change anything in their BGP configuration in the last decade, so "they didn't have any issue with this, only your network has issues''. Those are all rabbit holes that I will gladly trade for a little bit of RAM usage in a heartbeat. So some friends and I are working on an RFC draft to fix this: https://datatracker.ietf.org/doc/html/draft-ymbk-sidrops-rov-no-rr Comments and contributions are most welcome. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 16:26, Lukas Tribus wrote: Still, the monitoring script makes sense. Yep - lots of monitoring is needed. I'm monitoring some odd behaviour where Fort will, after a few days, grow its delta in the number of VRP's vs. rpki-client. Probably one of the best reasons to run different validators. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 16:02, Lukas Tribus wrote: When I tested RTR on IOS-XR I hit some strange bugs in the RTR client, specifically the RTR session would hang in certain scenarios (router restart, RTR server reboot, etc), requiring manual intervention with a "clear bgp rpki-server 1.2.3.4". It was *a lot* better in more recent releases, but 6.5.3 was not part of those better releases. I still did not have a lot of trust in XR for this reason, so I wrote a NETCONF script that would make some sanity checks on the RTR session state on XR boxes (nagios friendly): https://gist.github.com/lukastribus/695c9e780d118755271519d4d3cb54f3 (minimum number of absolute v4 and v6 VRPs on each RTR server, maximum v4/v6 VRP deviation between the RTR servers, RTR servers not in "connected" state). And now, with the code you are currently running for IOS XR? Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 15:43, Lukas Tribus wrote: For ROV to work reliably it needs to be able to reconsider previously rejected invalids, so I would not recommend disabling soft-reconfiguartion inbound, instead I'd suggest to set it to always. If my router vendor is not automatically applying BGP policy based on RPKI state, it shouldn't matter what ended up in RIB if I have not set an explicit policy to act on RPKI state. So if a previously-Invalid route now becomes a VRP, and is then good to go for export toward a neighbor based on existing policy that matches, why can we not leverage Route Refresh to only cater for that change? I'm wondering if we can carry a loose relationship between the VRP database, and RIB state. Of course it would be better to store ROV-rejects separately, instead of rejecting them. Better yet: add a new drop statement in RPL, which makes the route not eligible for path selection, but doesn't remove it from memory - this way the operator has full flexibility. I still don't get why we need to worry about Invalid routes, if the operator is doing ROV to drop them. As soon as they become Valid, the RTR update will tell us that and we can allow for incremental changes for what we ask our neighbors to accept. I can't believe I'm saying this, but a famous CPE vendor from Latvia actually supports this (routing filter action: reject vs discard) [1], maybe the XR BGP team could steal some ideas from them ;) :-). I don't like to depend on optional or not commonly used BGP features and code paths in other people's networks for changes of any kind on my end. In Juniper's case, their default to keep a copy of Adj-RIB-In had the unintended side-effect of making ROV deployment less exciting. However, I still feel not being able to leverage Route Refresh and avoid this clumsiness with ROV is below par for what we can design. After all, that was the whole point of Route Refresh... Memory consumption was negligible for my use-case, iirc it was 200 - 300 MB for 30 - 40 peers, one of which was transit (so full table - this was about a year ago). I believe we had this conversation before, and you mentioned that this could be a DoS vector, which is true but all the solutions that we can possibly suggest simply implement a "selective" soft-reconfig inbound always" anyway, so the DoS vector needs to be addressed in a different way, in my opinion. Well, we had several peers disable sessions with us because their CPU was being impacted by all the refresh messages we were sending. So we have seen it become a DoS vector toward others, and then by extension, toward us when we have to lose shorter paths to those peers due to the session tear-down. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 11:22, Saku Ytti wrote: I think it should just be a config error. You're not just cucking yourself, but your peers and customers. So it shouldn't be a choice you can make. I don't disagree, especially as there are likely several other operators working this way, and not knowing it because the neighbor either hasn't complained, or isn't detecting for Route Refresh noise. However, the documentation should still be updated for folk running old code earlier than the new code which would have this improvement. We can also imagine improvements 1) by default keep all RPKI rejects, and have 'soft-inbound never' optionally to turn that off Similar to how Junos does it, but specifically for RPKI. That would make sense. Of course, if someone already uses 'soft-reconfiguration inbound' for historical reasons, then keeping it as they enable ROV works out for them anyway. 2) have 1 bit per neighbor indicating policy had rpki rejects and 2 bits for validation database update iindicating database become less/more permissive IFF database became more permissive and neighbor has rpki rejects and we have soft-inbound never, then refresh Reasonable. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 10:26, Gert Doering wrote: I've been extremely underwhelmed with the quality of IOS XR documentation in many respects, ROV being one of them. (And ROV on IOS XE is full of different eels... as if a totally different company has implemented it, without ever speaking to someone who has done this before, or understand what it's supposed to do) I and the usual friends you know of have been fighting with Cisco to fix ROV's default behaviour in IOS XE since 2015. They refuse to listen. Personally, I gave up. We don't run RPKI on our CSR1000v route reflectors, so I'm not bothered. We don't run RPKI on the few ASR1000's that are in the network, and those are also being replaced by the MX204. We are also working on replacing our ASR920's with something better. We ran RPKI on those for all of 3 days until we turned it off. The box has bigger problems, like insufficient FIB, that we are already dealing with. We're moving towards Arista 7280R, which is not without its own set of surprises... We use this box for customer Layer 2 aggregation in the data centre. Very happy with it. As far as its maturity goes as an IP/MPLS box goes, I cannot tell you. But, knowing you, I'll keep my eyes on arista-nsp open wide open :-). It's an awfully quiet list - haven't got anything new on there since February of this year. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 10:10, Saku Ytti wrote: Correct. Add 'keep none' to junos, and you'll have the same issue. Since we started running ROV in IOS XE in 2014, we would have hit this issue then and allowed for it if the BGP best path evaluation process in IOS XE did not do strange things by default, which I believe have not yet been fixed. So we turned it off then. We always used Juniper (MX80 included) for peering back then, so didn't run into this given Junos' default policy. We ran the ASR9001 for peering/transit for a long time before coming up against this, but it makes sense that the complaints only picked up in the last 2 years when ROV was on the rise, globally. Thanks for the clue, Saku. Hopefully someone here has the energy to ask Cisco to update their documentation, to make this a recommendation. I can't be asked :-). Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 09:51, Gert Doering wrote: I seem to remember it's related to RPKI ROV (every time the RPKI server sends new data the BGP implementation asks for a refresh to re-validate the incoming feed). Or at least there is something in the back of my head that says "I have heard someone talk about this unintended side-effect of RPKI ROV, together with a BGP setup without 'soft in always'". Might there be a correlation in your environment? Yes, this makes sense. We, generally, did not run "soft-reconfiguration inbound" since the IOS days, to save on RAM. Also, Route Refresh was the gold standard. I'm not sure the RAM issue is a big deal anymore, considering how large it is in routers these days... But I can see how this creates an undesired side effect with ROV, which then puts pressure on Route Refresh. I have to say, we did not consider it; which hardly surprises me since TAC didn't either. I have read a lot of Cisco documentation on configuring ROV on IOS XE and IOS XR, and unless something has been recently updated, this is not one of the explicit recommendations in that documentation. I can see how easy it is to overlook (or if you do have 'soft-reconfiguration inbound' configured, how that significance can also be overlooked). All that notwithstanding, the move to 100Gbps peering/transit + faster CPU's on the MX204 is still a worthy reason to move away from the ASR9001, for us anyway :-). Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 09:51, Saku Ytti wrote: When you receive an RTR update, you change your ingress policy, and you don't know what is the correct result without re-evaluating. I'm with you now - makes sense. And Junos defaults to always maintaining a copy of Adj-RIB-In, which is why we don't have that issue there, non? Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 09:42, Saku Ytti wrote: Did you run RPKI? Yes. Did you have softinbound disabled? Yes. This would cause a refresh on every RTR update. Basically a misconfiguration, if you run RPKI you must keep policy rejected routes. What's the thought-process here? Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/11/21 09:18, Gert Doering wrote:
"newer code" might be the key issue here - what version are you on?
Our 9001 have been extremely well-behaved in regards to BGP performance
and robustness. Not as fast as the RSP440, but still well up to the
job.
We're on 6.5.3 ("nothing interesting in newer versions, and still
supported").
We are currently running 6.7.1, mainly to fix some LDPv6 issues.
However, we started seeing this "too many BGP refresh messages" issue as
far back as 6.4.2.
Mark.
___
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/10/21 20:47, Shawn L wrote: I'll second Tom on this one. I actually have 2 9001S routers (the S is the port / memory limited license). They have 2 full feeds (V4 and V6) and some EIGRP, and that's about it. Currently handling things fine. Though only doing ~ 5gig in/out at this point. We have nothing against the forwarding performance of the ASR9001. It's the control/management plane that seems to be slowing down (at least for us, anyway) with newer code and a growing Internet DFZ. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/10/21 20:00, Tom Hill wrote: I may live to regret asking this, but... I've run a lot more than that on a 9001, and it handled it all with aplomb. They're not as fast as an RSP440 (or RSP880) but in no way did I find them to be liabilities when running alongside measurably faster routers in the same ASN. They were extremely competent, in fact. So how are you measuring this, and/or how is the issue manifesting? At various peering locations where the ASR9001 had been running for some years, it was starting to struggle to manage several hundred sessions (none of which were a full table). The router kept sending too many Refresh notices to neighbors, across a number of versions of IOS XR code. Cisco couldn't figure it out, and we kept losing sessions due to the nuisance we'd become across the exchange fabric. What was clear was that the issue kept growing as peers increased routes, or as we added neighbors. In applications where we had just 2x full BGP sessions, IS-IS would hang and blackhole traffic. The issue was hard to reproduce, but every time it happened, we knew that rebooting the router was the only solution. We still see this issue in the few nodes we have running. The good news it does not happen often, but also, it shouldn't. Is the ASR9001 CPU as slow as the MX80? I would say no, but considering it's the only non-x86 box we have, the performance delta of the control and management plane is visible. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/10/21 08:48, Saku Ytti wrote: Ranting and raving follows. All (smart) executives claim the upside is because of their leadership and downside is because of the market. While no data supports that replacing executive A with executive B improved or reduced company performance, that is, we don't know what qualities make companies fail or succeed. But I admire the beauty of something like this: https://exainfra.net/about-us/ 'Under Xs’ leadership, GTT bought over 40 companies and grew annual revenues from $65 million to over $1.6 billion during his tenure' I call this the "bottom line factor". People do not care about details anymore, both on the consumer side, and on the provider side. They just bottom line their experience based on who was in charge. Odd, then, that Steve Jobs had success, failure and success, at Apple, with the last bout totally blowing the whole game out the water. After all, if Apple nearly collapsed under him, then the metrics suggest he'd have no chance of reviving it. But... Having said that, 5y performance: SP500: 110% CSCO: 90% NOK: 20% JNPR: 10% PANW: 300% ANET: 450% So Cisco is losing to the wide market only very little, and is outperforming other SP vendors (Huawei excluded). So the market doesn't entirely agree with your assertion of user base attrition. ... I think the question was about loyal customers. Cisco know how to drum up new business, and they know how to increase cash output from existing business. But none of this suggests that this business is loyal. There does come a time where you can't keep adding new business, to keep showing growth. Unless, of course, you buy everything up and grow that way, which we know Cisco love to do. But even that has a shelf life. Dodge it as you may, but eventually, you need to provide value to the paying customer base, and that always catches up with you. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/9/21 19:04, Gert Doering wrote:
Looking at the prices for 9901/02/03/04, and the licensing models for
NCS5700, I can assure you, "it won't be $80,000".
More like "$100k for the Chassis, and then another $300k for all required
licenses, to be renewed every 3 years" (subject to bazaar style discount
negotiations that make any sort of budget planning impossible)...
Cisco is a textbook example how to drive away a truly loyal user base,
and then blaim it on stock market analysts ("they said that any company
without a recurring revenue software model will be dead soon").
Gert nailed it.
This is one of the (many) reasons we aren't interested in Cisco anymore.
We are phasing out all the kit we have with them. We will, however, keep
the CSR1000v, as we like that, and can support ourselves on it.
Mark.
___
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/9/21 18:49, Drew Weaver wrote: We have a couple of these in production also are you planning on going with the 9902 or 9903 or are you switching altogether? We moved to the MX204. Not really that interested in Cisco anymore. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On 11/4/21 21:40, Tom Hill wrote: It's Typhoon (2nd Gen) so 4M v4, or 2M v6. Assuming you don't use any of it for anything else, e.g. labels. You're likely to run into other limits first (PPS, RAM, cXR discontinued...) As Tom said. We are retiring ours, because CPU performance for just 2x IPv4 + 2x IPv6 full sessions is too much for the Freescale PPC CPU. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR1001-X Password Recovery Failure
On 10/21/21 17:44, Tim Densmore wrote: It turns out that the issue was related to "controller mode" vs "autonomous mode". Somehow deleting the config caused the the asr1001s to choose "controller mode" looking for an SDWAN controller. Oh dear Lord... Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR1001-X Password Recovery Failure
On 10/17/21 20:16, Tim Densmore wrote: Hi Mark, Unfortunately I don't have the exact code and it's possible that I only upgraded one of them so far. Things got a little confused once I could no longer access the CLI on the devices. But basically 16-something to the latest GA 17, though if 17 is problematic we may just stay at the latest 16. When I was moving from 3.x to 17.x (via 16.x and lots of ROMMON, FPGA and CPLD drama), I recall some licensing agreements that were required before the box could load the saved configuration. Did any of that pop up on your console? I'm talking about the ASR1006 here. It's possible that the ASR1001-X may have something different going on, as I did not hit a similar issue on our ASR1002-X's. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR1001-X Password Recovery Failure
On 10/17/21 04:12, Tim Densmore wrote: Hi Folks, I recently upgraded and erased the startup config on a couple of asr1001-x routers that are being repurposed. Standard procedure, though this is the first time I have used the 1001-x specifically. Upon reboot I was expecting a "Router>" prompt, but instead I'm presented with a "Username:" prompt that does not allow me to log in with any of the local users or passwords that would have been in the config on this router. So, I boot into rommon, run "confreg 0x2142" and reboot the router. Upon reboot, the same "Username:" prompt shows up. I figure my settings in rommon didn't stick so I check them and verify that confreg really is set to 0x2142. This exact same issue is present on two different asr1001-x routers which leads me to believe that there must be something in the 1001-x docs about password recovery that I'm missing. Has anyone ever seen this, and if so could you point me to documentation that shows how to fix it? I have never seen anything like it and I'd rather not have to go through TAC if this is simple to fix. From what code did you upgrade, and to what? Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] CSR1000v + ASR1000 Code Upgrade Pleasure...
Hi all. I thought I'd share our recent experiences, per subject, just in case others run into the same problems. So... we finally decided to try 17.3(4a)MD for the CSR1000v, after years of happy operation. Good Lord, what a drama! At first, we couldn't figure out why iBGP sessions to all Cisco boxes could not stand up. Then we realized it's because IS-IS to them could not stand up. Then we realized it's because BFD sessions could not stand up. But even after removing BFD, IS-IS remained down. After 3 days of searching, we finally landed on CSCuz58508. In case you don't have CCO access, it is the same issue as described here: https://community.cisco.com/t5/cisco-cloud-service-router-csr/b00ocg4q4e-csr-1000v-16-3-1a-can-t-set-mtu-on-gig-interface/td-p/3054853 This was even more confusing for us, because our interface driver on VMware ESXi is vmxnet3. The bug ID suggests the problem is fixed in 16.3(2) and 16.4(1). So to be safe, we tested 16.12(5)MD, which allowed us to enable jumbo frames, but that only appeared to be a cosmetic thing. In the background, the box was simply dropping packets, silently. We found this out when we tried to copy other files to the node, and it would just hang without any feedback. Removing the jumbo frame support allowed the files to come through. We noticed that nodes still running 3.17(0)S did not have any issues with IS-IS or BFD, or MTU. However, this code was only ever released as an ED train (and to be fair, we've been having dodgy issues with it in recent years), so we decided to downgrade to 3.16(9)S (which is actually an upgrade from 3.17(00)S, since the 3.16 train is an MD release, with the latest release being March 2019, vs. July 2017 for 3.17(4)SED). With that, no more MTU issues, BFD and IS-IS are happy, iBGP is happy. We definitely won't be wasting any more time trying to make Denali, Gibraltar, Fuji, Everest or Amsterdam work on our CSR1000v complement. Needless to say, moving the ASR1000 platform to 17.3 has also come with its own avenue of pleasure, what with all the ROMMON, CPLD and FPGA upgrade mess that is. What the documentation says and what happens in real life are two very different things. It has taken us a week to come up with our own working procedure to upgrade just one box, worse if it's a dual-RP system. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] policer on ASR1001X
Thanks, Lukasz. For clarity, my requests didn't fall on deaf ears. The local AM team escalated all the way to the TME's and BU heads in San Jose, and they gave the answer to move to the NCS540 because the ASR920 is on its way out. Very possible that things changed since then, which would be great for customers. Glad to hear that you, indeed. But we are way down our path to migrate away now. That said, we still do like the CSR1000v very much as our out-of-path RR. Considering the current code is full of great features, many of which we use and many that we don't, we'd be keeping that anyway even if IOS XE were to ever disappear :-). Trying to run Junos or IOS XR as an RR just hurts my eyes. So I'll give IOS XE that :-). Mark. On 9/10/21 18:25, Łukasz Bromirski wrote: Mark, I’m from a different BU, but overall - yes, I remember some of the discussions we’ve had in the past. I’m very sorry it turned out this way. Unfortunately, some of the decisions are not made on single-platform level, and I do get you’re frustrated because either there’s no one to talk to, answers are not satisfactory or there’s simply black hole - no answers. Just for everyone to know - SP team changed the way they process feature requests from Customers and Account Teams. Tools by themselves won’t solve any problem, but right now it should be much easier to track all of the requests received across the board. If you, like Mike, are frustrated with the suggestions or requests falling to „deaf ears”, please try once again. And hey - solid competition is what can move us all forward :) ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] policer on ASR1001X
On 9/10/21 18:04, Lukasz Bromirski wrote: You may be talking about ASR 903/907, which indeed changed into NCS 540 (XR based) and NCS 560 (also XR based). No, I was asking for the ASR920. NCS 520 is IOS-XE based though given the positioning of the platform (access). Feature-wise, sounds like the ASR920 would still be miles ahead, non? NX OS teams is virtualizing everything right now. There are no plans to introduce IOS-XE to NX-OS switches that I am aware of. I mean the other way around... Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] policer on ASR1001X
On 9/10/21 17:50, Lukasz Bromirski wrote: IOS-XE is here to stay :) Indeed, there’s “dumbed down” version of it for SD-WAN, and they’re being slowly unified with normal IOS-XE being adopted to work in “centralized” (vs “autonomous”) mode. That’s not “autonomous” like with the Autonomic Networking feature from some years back, it’s “normal” IOS-XE. From hardware perspective, yes, UADP (Catalyst/switches) and QFP (Catalyst/ASR/routers) can handle a lot of fancy QoS duties, and doing pps/bps at the same time would be just enhancement. PPS limit for normal traffic seems to be less popular as Customers usually care more about bandwidth/throughput than PPS, while PPS is *very* important and more applicable for Control-Plane protection duties, as all processing is PPS-bound obviously. @James - please reach out to your account team to request such feature. Thanks, Lukasz. But with respect, this is one of the reasons I am changing all our gear over to Juniper. The messaging from Cisco depends on who you speak to, and when. Last year with our AM team was horrible, trying to get features into the ASR920, and being told that the NCS540 is where all focus is going; so, sorry! This may or may not have been true last year. This may or may not be true this year. But I can't build a business on this uncertainty. I'm not moaning at you, just to be clear :-). Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] policer on ASR1001X
On 9/10/21 14:38, Saku Ytti wrote: Didn't they just release next-gen catalyst switches and isr cpes (rebranded as catalyst?) with IOS-XE? It wouldn't be the first time Cisco had different camps competing for direction, internally. Some of the features I have asked for on IOS XE in the past have been declined on the basis that many IOS XE-based platforms will be moving to IOS XR. Maybe this only affects routers, and not switches. Not sure how the NX OS team feel about IOS XE switches :-). Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] policer on ASR1001X
On 9/9/21 20:04, Saku Ytti wrote: Definitely no reason why ASR1k could not support it if you have leverage towards vendor. With Cisco putting a lot more effort into IOS XR, I really wonder if the ASR1000 and other platforms based on IOS XE will be around in the medium-to-long term. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] RPKI extended-community RFC8097
On 12/19/20 11:13, Robert Raszuk wrote: Jakob, It has been a while, but IIRC the original idea for the validation was that regardless if this is done by configuration enabling pre-best path eligibility or in route map no path will be dropped. At no point in the BGP design discussions there was a plan to automatically do any of this. So your REFRESH story or soft-in alternative sound like the original plan has somehow changed. See even if you validate in route map you may just mark it not-eligible or set higher local pref for VALID etc I am not sure how anyone could come with the idea to just drop there. So IMHO there is nothing wrong with specification. It is suboptimal implementation or configuration which needs to be fixed. It beats me why it is taking so long ... Absolutely! The spec. is clear on nodes only performing validation at the behest of the operator, and never automatically or inherently. This is a Cisco-specific issue, and either a mis-interpretation of the RFC, or a workaround to the impact of the spec. on their implementation. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] RPKI extended-community RFC8097
On 12/19/20 10:45, Saku Ytti wrote: I think the community largely got blindsided by this, I suspect marketability of the whole solution would have been a lot poorer if this argument was thrown around at standardisation. However, that ship has sailed, we can implement new cheaper methods, but the damage is done and it will be there long after we've retired. I know I got blindsided, and it was so obvious, but not a problem I was aware until a customer complained about excessive refresh. It would be funny to analyse how much more wattage is drawn because of this globally. how many early control-plane upgrades. Is it immaterial or material? I don't know. But it does seem to put some customers control-planes over the edge. We suffered with this a great deal when we used ASR9001's for peering. A bunch of peers complained about it, to the extent that they had to drop sessions. Problem was fixed by moving to the MX204. Not elegant, and an unnecessary spend. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] RPKI extended-community RFC8097
On 12/18/20 10:25, Ben Maddison wrote: Getting from where you are today, to what I'm advocating obviously means changing existing behaviors, which can be an unwelcome surprise. Earlier this year, I did ask Jeff (and Jakob) to help fix the IOS XE RPKI brokenness with a myriad of suggestions. There hasn't been much progress (specifically, feedback) on that since then. I was promised some test code with the recommended fixes, but the year is almost out, and I can understand that with the pandemic, priorities may have shifted. At any rate, these are the suggestions I made to Jeff and Jakob back in March. Not sure if they are still relevant anymore, as we turned off RPKI on all our IOS XE boxes, and haven't bothered to keep up with how it has developed on that platform since: * * When an RTR session is established with the validator, the router should be able to accept and store all validation information. However, it SHOULD NOT automatically use this information to make routing decisions. So this would be the first thing to fix so we have a good working base. * Have a single command that is configurable within BGP to automatically drop Invalid routes. This is useful if an operator wants to use RPKI mainly to drop Invalid routes, and has no need to evaluate other RPKI states. With this command, it negates the need to build a route-map to achieve the same thing. A sample configuration for this would be: router bgp 1234 address-family ipv4 bgp bestpath prefix drop-invalids address-family ipv6 bgp bestpath prefix drop-invalids * Remove the "bgp bestpath prefix-validate allow-invalid" command, as with this fix, the operator can decide to do this via a route-map if they want to. It also takes the gun - that they could use to easily shoot their foot - out of their hands. * Remove the "bgp bestpath prefix-validate disable" command, as with this fix, validation does not occur by default unless the operator enables this via a route-map, or via the "bgp bestpath prefix drop-invalids" command. * All other methods to enable validation should be done via a route-map only. When a route-map with RPKI match and set conditions is applied to a BGP session, the router will consult the validation database as populated by the RTR session, and then apply the policy on the router accordingly, as dictated by the route-map configuration. The route-map should be able to apply this policy both on the inbound and outbound directions of a BGP session. * An example of what a route-map should be able to do is as per below: route-map BGP-POLICY permit|deny 10 match rpki valid|not-found|invalid set rpki valid|not-found|invalid * So as per the route-map example above, matching an RPKI state ONLY tells the router to look for any routes in the validation database that correspond to the match condition defined in the route-map. This DOES NOT tell the router to do anything with those routes from a routing perspective. This is CRITICAL. It does, however, allow the router to export or drop those routes toward an eBGP or iBGP neighbor if applied in the outbound direction. * ONLY when the operator uses a "set" condition in the route-map does the router then apply the corresponding policy when used in the inbound direction from an eBGP or iBGP neighbor. If the operator does not use the "set" condition in the route-map, but applies it on an eBGP or iBGP session in the inbound direction, the router WILL NOT apply any policy related to RPKI. * It's VERY CRITICAL to note that the router SHOULD NOT make any INHERENT DECISIONS on routing policy based on the "match" or "set" conditions in the route-map. The decision on what to do SHOULD ONLY be based on whether the route-map is a "permit" or "deny" route-map. In other words, the router is unaware about the significance of RPKI state, i.e., Valid, Invalid, NotFound. To the router, those RPKI states are just arbitrary values with no real meaning. The router SHOULD ONLY make the decision under one of two circumstances: - The route-map's "permit" or "deny" action. - The "bgp bestpath prefix drop-invalids" command. * If an operator tries to configure both the "bgp bestpath prefix drop-invalids" command and the route-map version of dropping Invalids with a "deny" action, the router SHOULD prefer the route-map version over the global "bgp bestpath prefix drop-invalids" command. In fact, it would be helpful if the router can throw back an error if it recognizes that the operator has configured (or has tried to configure) both of these options at the same time. That way, the operator knows that one of the two has already been configured, and that the router SHOULD tell the operator which one will be used (and
Re: [c-nsp] RPKI extended-community RFC8097
On 12/18/20 08:58, Jakob Heitz (jheitz) wrote: Hi Lukas, Mark, Ben, The default bestpath prefix-validate behavior treats invalid routes as unfeasible and prefers valid routes over not-found. The default bestpath prefix-validate behavior cannot be used unless all paths of a net have the correct RPKI validity. That can only happen if all EBGP sessions into an AS validate their incoming routes and apply the RFC8097 extended community. If these conditions are not satisfied, then you cannot use the bestpath prefix-validate behavior and you must use route-maps to process the RPKI validity, like this: router bgp ... bgp rpki server tcp [...] address-family ipv4 bgp bestpath prefix-validate disable [...] route-map RM_EBGP_IN deny 10 match rpki invalid [...] I have a proposal to improve the bestpath prefix-validate behavior to better match how most operators use it. By a new configuration, I would treat valid and not-found with the same preference. Invalid would continue to be unfeasible. Then, a received IBGP route without the RFC8097 community will be fine. Thoughts? What I've been asking Cisco to do since 2014 is to prevent IOS XE from applying policy by default. This is broken and is in direct violation of the RFC. All RPKI policy must only be applied by the operator. The router has no business using RPKI state as part of its best path calculation process, unless specifically told to do so by the operator. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
