Re: Router Suggestions
I bought three MX204 a year ago and paid maybe 50% more than the quoted 11K for hardware and standard license. On top of that I paid a significant amount for BNG features and scale licenses, but not everyone needs that. The third MX204 was considerably cheaper (half price) because its purpose in life is to be a cold spare and a lab router. Why pay someone else for having a cold spare ready for next day replacement when you can have it yourself? Having a lab router to test config before rollout has really been a life saver. Averaged by the three routers I may have hit close to 11K, not counting the BNG licenses. Regards, Baldur On Mon, Jun 15, 2020 at 10:57 PM Forrest Christian (List Account) < li...@packetflux.com> wrote: > We just got a MX204 quote and it was close to 2.5x the price you're > quoting, with apparently the minimum license needed for full tables, and > Next Day replacement. So if it's really $11K, please shoot me an email > off list. Or if someone has a better place to get a decent quote for a > MX204, or can clarify where this quote might have went wrong, that would be > useful too. > > We're also looking at going the virtual router route where we put 2-3 > servers in a HA cluster loaded up with 10Gb interfaces and running some > sort of routing software. In case you didn't catch on, I'm fairly early in > running this idea through the paces, although it seems like this is a > pretty common thing nowadays. > > On Mon, Jun 15, 2020 at 6:02 AM Colton Conor > wrote: > >> For around $11,000 right now, you can get a brand new Juniper MX204 >> router. Alternatively, you can get a used MX240 / MX480 with quad power >> supplies, redundant quad core RE's, and 2 16X10G MIC cards for around >> $12,000. >> >> My question, is there anything else worth looking at in this price range >> / port configuration? Open to both new and used options. Looking to take >> full BGP routes. >> >> >> > > -- > - Forrest >
Re: Hurricane Electric has reached 0 RPKI INVALIDs in our routing table
On Tue, 16 Jun 2020 at 07:51, Mike Leber via NANOG wrote: Hey, > These prefix filters are updated automatically both through a system of > daily updates and real time updates to prevent RPKI INVALID routes from > being carried in our routing table. What does real time mean in this context? Does it mean exactly 0s leak of INVALID, or 99% less than 30s? Or how do you define it? I'm trying to think of an ideal way to do this in Junos which does a few second ephemeral config commits. I could have an always-on SSH session to each device to amortise login time, but even then if I can do this cycle in 5s, I'd have to wait for BGP propagation delay in DFZ, which is measured in minutes not seconds. So my definition of real time here would be 99% <5min. -- ++ytti
Re: Hurricane Electric has reached 0 RPKI INVALIDs in our routing table
congratulations HE team!. On Mon, Jun 15, 2020 at 9:56 PM TJ Trout wrote: > absolutely awesome Mike! > > Can you put on the roadmap to enable irr based filters for customers with > bgp communities? > > On Mon, Jun 15, 2020 at 9:48 PM Mike Leber via NANOG > wrote: > >> I'm pleased to announce Hurricane Electric has completed our RPKI >> INVALID filtering project and we now have 0 RPKI INVALIDs in our routing >> table. >> >> Hurricane Electric has 29021 BGP sessions with 22109 prefix filters with >> 7191 networks directly and 8239 networks including Internet exchanges. >> >> We filter all BGP sessions using prefix filters based on IRR and RPKI. >> >> These prefix filters are updated automatically both through a system of >> daily updates and real time updates to prevent RPKI INVALID routes from >> being carried in our routing table. >> >>
Re: Hurricane Electric has reached 0 RPKI INVALIDs in our routing table
absolutely awesome Mike! Can you put on the roadmap to enable irr based filters for customers with bgp communities? On Mon, Jun 15, 2020 at 9:48 PM Mike Leber via NANOG wrote: > I'm pleased to announce Hurricane Electric has completed our RPKI > INVALID filtering project and we now have 0 RPKI INVALIDs in our routing > table. > > Hurricane Electric has 29021 BGP sessions with 22109 prefix filters with > 7191 networks directly and 8239 networks including Internet exchanges. > > We filter all BGP sessions using prefix filters based on IRR and RPKI. > > These prefix filters are updated automatically both through a system of > daily updates and real time updates to prevent RPKI INVALID routes from > being carried in our routing table. > >
Hurricane Electric has reached 0 RPKI INVALIDs in our routing table
I'm pleased to announce Hurricane Electric has completed our RPKI INVALID filtering project and we now have 0 RPKI INVALIDs in our routing table. Hurricane Electric has 29021 BGP sessions with 22109 prefix filters with 7191 networks directly and 8239 networks including Internet exchanges. We filter all BGP sessions using prefix filters based on IRR and RPKI. These prefix filters are updated automatically both through a system of daily updates and real time updates to prevent RPKI INVALID routes from being carried in our routing table.
Re: ... you kicked out the patch cable (or, major global internet outages)
On Mon, Jun 15, 2020 at 7:38 PM wrote: > > Ben Cannon wrote on 6/15/2020 4:11 PM: > > > https://downdetector.com/ > > ...you kicked out a patch cable. (Nods to BOFH) > > > > In all seriousness, looks major... Long-haul cut? Did we lose a pie > > or COs? > > > > -Ben > T-mobile implies routing issue: > https://twitter.com/TMobileHelp/status/1272645683263094784 > > Outage seemed to have started with them earlier today but has spread to > other carriers. 'spread to other carriers' ... someones are having cellular backhaul issues maybe? oh, the twitter/etc/outages says: "tmo/sprint merger software go cray-cray"
Re: ... you kicked out the patch cable (or, major global internet outages)
Ben Cannon wrote on 6/15/2020 4:11 PM: https://downdetector.com/ ...you kicked out a patch cable. (Nods to BOFH) In all seriousness, looks major... Long-haul cut? Did we lose a pie or COs? -Ben T-mobile implies routing issue: https://twitter.com/TMobileHelp/status/1272645683263094784 Outage seemed to have started with them earlier today but has spread to other carriers.
... you kicked out the patch cable (or, major global internet outages)
https://downdetector.com/ ...you kicked out a patch cable. (Nods to BOFH) In all seriousness, looks major... Long-haul cut? Did we lose a pie or COs? -Ben
Re: Router Suggestions
Yes I too looked into that. And it was not near that price. Please send me and email off list. I would like to know where I might find that. On Mon, Jun 15, 2020 at 2:58 PM Forrest Christian (List Account) < li...@packetflux.com> wrote: > We just got a MX204 quote and it was close to 2.5x the price you're > quoting, with apparently the minimum license needed for full tables, and > Next Day replacement. So if it's really $11K, please shoot me an email > off list. Or if someone has a better place to get a decent quote for a > MX204, or can clarify where this quote might have went wrong, that would be > useful too. > > We're also looking at going the virtual router route where we put 2-3 > servers in a HA cluster loaded up with 10Gb interfaces and running some > sort of routing software. In case you didn't catch on, I'm fairly early in > running this idea through the paces, although it seems like this is a > pretty common thing nowadays. > > On Mon, Jun 15, 2020 at 6:02 AM Colton Conor > wrote: > >> For around $11,000 right now, you can get a brand new Juniper MX204 >> router. Alternatively, you can get a used MX240 / MX480 with quad power >> supplies, redundant quad core RE's, and 2 16X10G MIC cards for around >> $12,000. >> >> My question, is there anything else worth looking at in this price range >> / port configuration? Open to both new and used options. Looking to take >> full BGP routes. >> >> >> > > -- > - Forrest >
RE: Router Suggestions
As someone who has used VSR (Nokia) and VMX (Juniper) I’d suggest, good luck on your plan to use servers for this sort of routing. If you want a cheap router to handle full tables and a couple of 10G interfaces worth of throughput I’d suggest you would be a lot better off with Mikrotik’s latest hardware offering - https://mikrotik.com/product/ccr2004_1g_12s_2xs Just my 2c >We're also looking at going the virtual router route where we put 2-3 servers >in a HA cluster loaded up with 10Gb interfaces and running some sort of >routing software. In case you didn't catch on, I'm fairly early in running >this idea through the paces, although it seems like >this is a pretty common >thing nowadays.
Re: Router Suggestions
We just got a MX204 quote and it was close to 2.5x the price you're quoting, with apparently the minimum license needed for full tables, and Next Day replacement. So if it's really $11K, please shoot me an email off list. Or if someone has a better place to get a decent quote for a MX204, or can clarify where this quote might have went wrong, that would be useful too. We're also looking at going the virtual router route where we put 2-3 servers in a HA cluster loaded up with 10Gb interfaces and running some sort of routing software. In case you didn't catch on, I'm fairly early in running this idea through the paces, although it seems like this is a pretty common thing nowadays. On Mon, Jun 15, 2020 at 6:02 AM Colton Conor wrote: > For around $11,000 right now, you can get a brand new Juniper MX204 > router. Alternatively, you can get a used MX240 / MX480 with quad power > supplies, redundant quad core RE's, and 2 16X10G MIC cards for around > $12,000. > > My question, is there anything else worth looking at in this price range / > port configuration? Open to both new and used options. Looking to take full > BGP routes. > > > -- - Forrest
academic paper on Peerlock BGP protection mechanism
Dear colleagues, An interesting paper has been made available as pre-print: "Peerlock: Flexsealing BGP" by Tyler McDaniel, Jared M. Smith, and Max Schuchard from the University of Tennessee. The paper probably is the most formal description of Peerlock so far. They even conducted active measurements to reverse-engineer what the state of Peerlock deployment is in the global Intenet routing system. Abstract: https://arxiv.org/abs/2006.06576 PDF: https://arxiv.org/pdf/2006.06576.pdf Recommended reading! Kind regards, Job
Re: [c-nsp] LDPv6 Census Check
On 15/Jun/20 12:13, adamv0...@netconsultings.com wrote: > Not to mention this whole thread is focused solely on next-hop > identification -which is just the lowest of the layers of abstraction > in the vertical stack. We haven’t talked about other "entities" that > need identification like: VPNs, applications, policies (yes I'm > looking at you VXLAN!) etc... - all of which are way better identified > by a simple label rather than IPinIPinIP The problem is if you want to have MPLS services signaled over IPv6, you first need an IPv6 control plane that will signal the labels that will support those services, i.e., LDPv6 and RSVPv6. Right now, all vendors that support LDPv6 do so only for pure MPLS-IPv6 forwarding. If you want l2vpnv6, l3vpnv6, EVPNv6, 4PE, 4VPE, TE-FRRv6, e.t.c., you can't get that today. You'd have to signal those services over LDPv4 or RSVPv4. The guys did a great gap analysis back in 2015, when LDPv6 began to appear in IOS XR and Junos, that showed the challenges toward an IPv6-only MPLS network. I believe much of this is still relevant in 2020: https://tools.ietf.org/html/rfc7439 I have to commend Vishwas, together with both Rajiv's, for kicking this off way back in 2008. The road has been long and hard. Mark.
Fwd: [lacnog] LACNOG 2020 - Call for Presentations
Hi all, LACNOG (the Latin American and Caribbean Network Operators Group) will be a virtual meeting this year. Looking forward to great talks from our big brother NANOG members :-) /Carlos LACNOG PC Forwarded message: From: Jorge Villa To: Latin America and Caribbean Region Network Operators Group Subject: [lacnog] LACNOG 2020 - Call for Presentations Date: Mon, 08 Jun 2020 11:49:02 -0400 LACNOG 2020 - Call for Presentations https://www.lacnog.org/eventos/ LACNOG, the Latin American and Caribbean Network Operators Group, will hold its LACNOG 2020 conference in the city of Santa Cruz de la Sierra, Bolivia, from 5 to 9 October 2020, which will be co-located with the LACNIC 34 event. The LACNOG 2020 Program Committee invites the Internet community to submit their proposals for the event. In line with the spirit of LACNOG, proposed topics must be geared towards Internet development in the region. The following is a non-exhaustive list of some of the topics of interest for the LACNOG 2020 meeting: Network operation and professional experiences, success stories Internet of Things MANRS Community networks IPv6 integration and deployment Experiences involving botnets, malware, spam, viruses, denial of service attacks and exploit techniques IP network architecture, sizing, configuration and administration Routing and switching protocols, including unicast, multicast, anycast, SDN, etc. End-user applications (e.g. e-mail, HTTP, DNS, NFVs, etc.) Value-added services such as VPNs, distributed systems, cloud computing, etc. Peering, Internet traffic exchange, IXPs Network data security and management, attack mitigation Network monitoring, performance, measurements and telemetry Network automation, evolution and convergence Infrastructure and physical transport, including optical and wireless networks Legislation, regulations and Internet governance issues Research and education Possible presentation formats include: Lightning talk: brief, 8-minute presentation plus 4 additional minutes for questions. Presentation: 20-minute presentation plus 5 additional minutes for questions. The deadlines for the 2020 call for proposals will be as follows: Reception of proposals: 8 June - 7 July 2020 Proposals will be accepted until: 7 July 2020 at 23:59 UTC-3 (Uruguay time) Evaluation by the Program Committee: 8-19 July 2020 Announcement of results: 20 July 2020 Deadline for submitting the final presentation: 1 August - 18 September 2020 Final presentations will be accepted until: 21 September 2020 at 23:59 UTC-3 (Uruguay time) Event date: 5-9 October 2020 Applicants must submit a summary and a draft of the slides of their proposed presentation along with a brief biography and photograph using the form available at https://vulcano.lacnog.org/e/lacnog2020 If your work is selected, you authorize LACNOG and LACNIC to publish your name, photograph, biography, and final work in the event program. Regardless of the chosen format, all works must presented in person at the event venue. Given the current pandemic caused by the coronavirus (covid-19) outbreak, there is the possibility that the event may be held in virtual format, in which case speakers will be notified in advance and the necessary adjustments will be coordinated based on the schedule determined by the Program Committee. Applicants whose proposals are accepted will be exempted from paying their in-person event registration fee but will not automatically receive financial assistance for their travel and/or accommodation expenses. However, LACNIC offers a fellowship program for attending the LACNIC 34 event which will be held jointly with LACNOG 2020 to which speakers can apply independently. When submitting your draft to the LACNOG Program Committee, please specify whether you are applying (or planning to apply) for a LACNIC fellowship. Speakers presenting their work at the LACNOG 2020 conference will receive a certificate acknowledging their participation. Guidelines for Submitting a Presentation for LACNOG have been prepared containing a description of the criteria that will be considered when evaluating each proposal, presentation format details and other data. These Guidelines are available at https://www.lacnog.org/guiapresentaciones/. Communications with the Program Committee will be handled through p...@lacnog.org. We thank you in advance for your attention and look forward to your proposals for LACNOG 2020. Program Committee ___ LACNOG mailing list lac...@lacnic.net https://mail.lacnic.net/mailman/listinfo/lacnog Cancelar suscripcion: https://mail.lacnic.net/mailman/options/lacnog
Re: Router Suggestions
Patrick Cole wrote on 15/06/2020 14:16: MX204's may have gotten chaper in the last year I don't know. But YMMV. OP needs to check the licensing package for the MX204, and work out the N-year TCO. Nick
Re: [c-nsp] LDPv6 Census Check
On 15/Jun/20 12:13, adamv0...@netconsultings.com wrote: > Not to mention this whole thread is focused solely on next-hop identification > -which is just the lowest of the layers of abstraction in the vertical stack. > We haven’t talked about other "entities" that need identification like: VPNs, > applications, policies (yes I'm looking at you VXLAN!) etc... - all of which > are way better identified by a simple label rather than IPinIPinIP The problem is if you want to have MPLS services signaled over IPv6, you first need an IPv6 control plane that will signal the labels that will support those services, i.e., LDPv6 and RSVPv6. Right now, all vendors that support LDPv6 do so only for pure MPLS-IPv6 forwarding. If you want l2vpnv6, l3vpnv6, EVPNv6, 4PE, 4VPE, TE-FRRv6, e.t.c., you can't get that today. You'd have to signal those services over LDPv4 or RSVPv4. The guys did a great gap analysis back in 2015, when LDPv6 began to appear in IOS XR and Junos, that showed the challenges toward an IPv6-only MPLS network. I believe much of this is still relevant in 2020: https://tools.ietf.org/html/rfc7439 I have to commend Vishwas, together with both Rajiv's, for kicking this off way back in 2008. The road has been long and hard. Mark.
Re: Router Suggestions
Drew, A 6 Tbps router is a little more expensive than a 2 Tbps router, yes. I was referring to the 7280SR range not the 7280CR. I ended up getting our SR2k's around the same price as MX204's with the help of our friendly Arista rep. MX204's may have gotten chaper in the last year I don't know. But YMMV. -PC > We've been setting up some Arista DCS-7280CR2K-30-F lately and they have been > just OK. The pricing is not at all close to $12,000 though. > > -Drew > > > -Original Message- > From: NANOG On Behalf Of Patrick Cole > Sent: Monday, June 15, 2020 8:42 AM > To: Colton Conor > Cc: NANOG > Subject: Re: Router Suggestions > > Colton, > > We recently opted for the Arista 7280R2K for peering edge. They come in at > similar price points (maybe a little more?) to the MX204 and are a bit higher > capacity. > > Worth a look in. > > Cheers, > > Patrick > > Mon, Jun 15, 2020 at 07:00:55AM -0500, Colton Conor wrote: > > > >For around $11,000 right now, you can get a brand new Juniper MX204 > >router. Alternatively, you can get a used MX240 / MX480 with quad power > >supplies, redundant quad core RE's, and 2 16X10G MIC cards for around > >$12,000. > >My question, is there anything else worth looking at in this price range > > / > >port configuration? Open to both new and used options. Looking to take > >full BGP routes. > > -- > Patrick Cole > Principal Engineer > Spirit Telecom Ltd > 19-25 Raglan St, South Melbourne VIC 3205 > Desk:0385541391 > Mobile: 0410626630 > -- Patrick Cole Principal Engineer Spirit Telecom Ltd 19-25 Raglan St, South Melbourne VIC 3205 Desk:0385541391 Mobile: 0410626630
RE: Partial vs Full tables
Yeah, as I mentioned this was a few years ago. =) -Original Message- From: Saku Ytti Sent: Monday, June 15, 2020 8:54 AM To: Drew Weaver Cc: William Herrin ; brad dreisbach ; nanog@nanog.org Subject: Re: Partial vs Full tables Hey Drew, > The only time we have ever noticed any sort of operational downside of using > uRPF loose was when NTTs router in NYC thought a full table was only 500,000 > routes a few years back. If NTT is 2914 this can no longer happen and it is difficult to see 2914 would ever go back to uRPF. In typical implementation today ACL is much cheaper than uRPF, so we've migrated to ACL. uRPF value proposition is mostly on CLI Jockey networks, if configuration are generated for most use-cases ACL is superior solution anyhow. In your particular defect, it doesn't seem to matter if uRPF was or was not enabled, was it dropped by uRPF/loose failure or lookup failure seems uninteresting (We do not default route). -- ++ytti
Re: Partial vs Full tables
Hey Drew, > The only time we have ever noticed any sort of operational downside of using > uRPF loose was when NTTs router in NYC thought a full table was only 500,000 > routes a few years back. If NTT is 2914 this can no longer happen and it is difficult to see 2914 would ever go back to uRPF. In typical implementation today ACL is much cheaper than uRPF, so we've migrated to ACL. uRPF value proposition is mostly on CLI Jockey networks, if configuration are generated for most use-cases ACL is superior solution anyhow. In your particular defect, it doesn't seem to matter if uRPF was or was not enabled, was it dropped by uRPF/loose failure or lookup failure seems uninteresting (We do not default route). -- ++ytti
RE: Router Suggestions
We've been setting up some Arista DCS-7280CR2K-30-F lately and they have been just OK. The pricing is not at all close to $12,000 though. -Drew -Original Message- From: NANOG On Behalf Of Patrick Cole Sent: Monday, June 15, 2020 8:42 AM To: Colton Conor Cc: NANOG Subject: Re: Router Suggestions Colton, We recently opted for the Arista 7280R2K for peering edge. They come in at similar price points (maybe a little more?) to the MX204 and are a bit higher capacity. Worth a look in. Cheers, Patrick Mon, Jun 15, 2020 at 07:00:55AM -0500, Colton Conor wrote: >For around $11,000 right now, you can get a brand new Juniper MX204 >router. Alternatively, you can get a used MX240 / MX480 with quad power >supplies, redundant quad core RE's, and 2 16X10G MIC cards for around >$12,000. >My question, is there anything else worth looking at in this price range / >port configuration? Open to both new and used options. Looking to take >full BGP routes. -- Patrick Cole Principal Engineer Spirit Telecom Ltd 19-25 Raglan St, South Melbourne VIC 3205 Desk:0385541391 Mobile: 0410626630
Re: Router Suggestions
Colton, We recently opted for the Arista 7280R2K for peering edge. They come in at similar price points (maybe a little more?) to the MX204 and are a bit higher capacity. Worth a look in. Cheers, Patrick Mon, Jun 15, 2020 at 07:00:55AM -0500, Colton Conor wrote: >For around $11,000 right now, you can get a brand new Juniper MX204 >router. Alternatively, you can get a used MX240 / MX480 with quad power >supplies, redundant quad core RE's, and 2 16X10G MIC cards for around >$12,000. >My question, is there anything else worth looking at in this price range / >port configuration? Open to both new and used options. Looking to take >full BGP routes. -- Patrick Cole Principal Engineer Spirit Telecom Ltd 19-25 Raglan St, South Melbourne VIC 3205 Desk:0385541391 Mobile: 0410626630
Re: Router Suggestions
On 6/15/20 8:00 AM, Colton Conor wrote: For around $11,000 right now, you can get a brand new Juniper MX204 router. Alternatively, you can get a used MX240 / MX480 with quad power supplies, redundant quad core RE's, and 2 16X10G MIC cards for around $12,000. My question, is there anything else worth looking at in this price range / port configuration? Open to both new and used options. Looking to take full BGP routes. Do you want high-touch or a packet pusher? The MX204 is somewhere in the middle. Extreme SLX9540 and Arista 7280SR will "take full tables" with some FIB compression and route caching. YMMV if they'll actually work in your application, but my experience with the 9540 has been positive in a typical leaf edge application. Street price is in the ballpark of what you're talking, and you get a few 100GbE ports to go with your 10GbE ports. The SLX9640 or 7280R will apparently actually fit full routes in hardware, but the pricing seems to be a fair bit higher than you're talking. All of these are pretty much packet pushers with minimal "touch". In particular, traffic control capabilities are somewhat limited aside from applying them to the port itself, and they definitely won't do "BNG" type functionality with PPPoE or tag-per-customer with shared L2 appearance at least not at any real scale. -- Brandon Martin
RE: Partial vs Full tables
This is just my experience so do whatever you want with that. The only time we have ever noticed any sort of operational downside of using uRPF loose was when NTTs router in NYC thought a full table was only 500,000 routes a few years back. That is a fairly real consideration though. =) -Original Message- From: NANOG On Behalf Of William Herrin Sent: Thursday, June 11, 2020 12:18 PM To: brad dreisbach Cc: nanog@nanog.org Subject: Re: Partial vs Full tables On Thu, Jun 11, 2020 at 9:08 AM brad dreisbach wrote: > uRPF absolutely kills the pps performance or your hardware due to the > packet having to be recirculated to do the check(at least this is the > case on every platform that ive ever tested it on). use acl's to protect your > edge. Hi Brad, Don't the ACLs generally live in a partition of the TCAM too? So you're going from two constant-time TCAM lookups per packet (route, acls) to three (route, urpf, acls)? Not rhetorical; getting close to the edge of my knowledge here. Regards, Bill Herrin -- William Herrin b...@herrin.us https://bill.herrin.us/
Router Suggestions
For around $11,000 right now, you can get a brand new Juniper MX204 router. Alternatively, you can get a used MX240 / MX480 with quad power supplies, redundant quad core RE's, and 2 16X10G MIC cards for around $12,000. My question, is there anything else worth looking at in this price range / port configuration? Open to both new and used options. Looking to take full BGP routes.
RE: [c-nsp] LDPv6 Census Check
> From: Saku Ytti > Sent: Monday, June 15, 2020 11:02 AM > > On Mon, 15 Jun 2020 at 12:46, wrote: > > > Yes it can indeed, and that's moving towards the centre between the > extreme cases that David laid out. > > It's about how granular one wants to be in identifying an end-to-end path > between a pair of edge nodes. > > I agree with you that MPLS is still better than IP, and I tried to > > illustrate that even enumerating every possible paths using deep label > stack is not a problem (and even that can be alleviated using hierarchy of > LSPs). > > The entirety of my point is, if we were rational, we'd move towards > increasingly efficient solutions. And technically everything we do in MPLS > tunnels, we can do in IP tunnels and converse. Should we imagine a future > where all features and functions are supported in both, it's clear we should > want to do MPLS tunnels. Just the [IGP][BGP-LU] 8B overhead, compared to > IP 40B overhead should drive the point home, and ultimately, that's the only > difference, rest is implementation. > > And I'm saddened we've been marketed snake-oil like SRv6 with fake > promises of inherent advantages or simplicity 'just IP'. > > We can do better than MPLS, absolutely. But IP is worse. > Yes I absolutely agree, Not to mention this whole thread is focused solely on next-hop identification -which is just the lowest of the layers of abstraction in the vertical stack. We haven’t talked about other "entities" that need identification like: VPNs, applications, policies (yes I'm looking at you VXLAN!) etc... - all of which are way better identified by a simple label rather than IPinIPinIP adam
Re: [c-nsp] LDPv6 Census Check
On Mon, 15 Jun 2020 at 12:46, wrote: > Yes it can indeed, and that's moving towards the centre between the extreme > cases that David laid out. > It's about how granular one wants to be in identifying an end-to-end path > between a pair of edge nodes. > I agree with you that MPLS is still better than IP, > and I tried to illustrate that even enumerating every possible paths using > deep label stack is not a problem (and even that can be alleviated using > hierarchy of LSPs). The entirety of my point is, if we were rational, we'd move towards increasingly efficient solutions. And technically everything we do in MPLS tunnels, we can do in IP tunnels and converse. Should we imagine a future where all features and functions are supported in both, it's clear we should want to do MPLS tunnels. Just the [IGP][BGP-LU] 8B overhead, compared to IP 40B overhead should drive the point home, and ultimately, that's the only difference, rest is implementation. And I'm saddened we've been marketed snake-oil like SRv6 with fake promises of inherent advantages or simplicity 'just IP'. We can do better than MPLS, absolutely. But IP is worse. -- ++ytti
RE: [c-nsp] LDPv6 Census Check
> From: Saku Ytti > Sent: Monday, June 15, 2020 10:31 AM > > On Mon, 15 Jun 2020 at 12:24, wrote: > > > > Yes this is where each node needs to have a label uniquely identifying > > every LSP passing through it. > > Saku, > > With IP header you don't need this, > > Consider this: > > PE1 to PE2 via 3 P-core nodes > > With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2, > > which will be load-shared across all 3 paths. > > Using MPLS If you need to uniquely identify each path you need 3 FECs > > (3 LSPs one via each P core node), now imagine you have 100K possible > > paths across the fabric -that's a lot of FECs on PE1 or any node in > > the fabric where each has to have a unique label for every possible > > unique path via the core that the particular node is part of. > > Are we talking about specific implementations of fundamentals? It sounds > like we are talking about a specific case where IP next-hop is unilist of N > next- > hops, and MPLS next-hop is a single item without indirection? This is not a > fundamental difference, this is implementation detail. > There is no particular reason MPLS next-hop couldn't be unilist of N > destinations. > Yes it can indeed, and that's moving towards the centre between the extreme cases that David laid out. It's about how granular one wants to be in identifying an end-to-end path between a pair of edge nodes. I agree with you that MPLS is still better than IP, and I tried to illustrate that even enumerating every possible paths using deep label stack is not a problem (and even that can be alleviated using hierarchy of LSPs). adam
Re: [c-nsp] LDPv6 Census Check
On Mon, 15 Jun 2020 at 12:24, wrote: > Yes this is where each node needs to have a label uniquely identifying every > LSP passing through it. > Saku, > With IP header you don't need this, > Consider this: > PE1 to PE2 via 3 P-core nodes > With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2, which > will be load-shared across all 3 paths. > Using MPLS If you need to uniquely identify each path you need 3 FECs (3 > LSPs one via each P core node), now imagine you have 100K possible paths > across the fabric > -that's a lot of FECs on PE1 or any node in the fabric where each has to > have a unique label for every possible unique path via the core that the > particular node is part of. Are we talking about specific implementations of fundamentals? It sounds like we are talking about a specific case where IP next-hop is unilist of N next-hops, and MPLS next-hop is a single item without indirection? This is not a fundamental difference, this is implementation detail. There is no particular reason MPLS next-hop couldn't be unilist of N destinations. I think people are too focused on thinking IP and MPLS have some inherent magical property differences, they don't. We only care about lookup cost (again IP can be made cheap with IPinIPinIP tunnels and telling LSR devices all lookups are LEM host lookups) and we care about key width. Rest is implementation detail. Yes on typical case there are some biases in IP and MPLS, but these can be rendered away leaving the fundamental differences. Briding the gap from MPLS to IP is far smaller than bridging the gap from IP to MPLS, there is so much added value which depends on MPLS tunnels. -- ++ytti
RE: [c-nsp] LDPv6 Census Check
> David Sinn > Sent: Friday, June 12, 2020 4:42 PM > > > On Jun 12, 2020, at 8:26 AM, Saku Ytti wrote: > > > > On Fri, 12 Jun 2020 at 18:16, David Sinn wrote: > > > The label stack question is about the comparisons between the two > extremes of SR that you can be in. You either label your packet just for it's > ultimate destination or you apply the stack of the points you want to pass > through. > > In the former case you are, at the forwarding plane, equal to what you see > with traditional MPLS today, with every node along the path needing to > know how to reach the end-point. Yes, you have lowered label space from > traditional MPLS, but that can be done with site-cast labels already. And, > while the nodes don't have to actually swap labels, when you look at > commodity implementations (across the last three generations since you > want to do this with what is deployed, not wholesale replace the network) a > null swap still ends up eating a unique egress next-hop entry. So from a > hardware perspective, you haven't improved anything. Your ECMP group > count is high. > Yes this is where each node needs to have a label uniquely identifying every LSP passing through it. Saku, With IP header you don't need this, Consider this: PE1 to PE2 via 3 P-core nodes With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2, which will be load-shared across all 3 paths. Using MPLS If you need to uniquely identify each path you need 3 FECs (3 LSPs one via each P core node), now imagine you have 100K possible paths across the fabric -that's a lot of FECs on PE1 or any node in the fabric where each has to have a unique label for every possible unique path via the core that the particular node is part of. > In the extreme latter case, you have to, on ingress, place the full stack of > every "site" you want to pass through. That has the benefits of "sites" only > need labels for their directly connected sites, so you have optimized the > implications on commodity hardware. However, you now have a label stack > that can be quite tall. At least if you want to walk the long-way around the > world, say due to failure. On top, that depth of label stack means devices in > the middle can't look at the original payload to make ECMP decisions. So you > can turn to entropy labels, but that sort of makes matters worse. > David, You can use hierarchy of tunnels to help with the deep label-stack imposition problem. adam
RE: [c-nsp] LDPv6 Census Check
> From: David Sinn > Sent: Friday, June 12, 2020 4:19 PM > > > On Jun 11, 2020, at 2:02 PM, Mark Tinka wrote: > > > > On 11/Jun/20 17:32, David Sinn wrote: > > > >> Respectfully, that is deployment dependent. In a traditional SP topology > that focuses on large do everything boxes, where the topology is fairly point- > to-point and you only have a small handful of nodes at a PoP, labels can be > fast, cheap and easy. Given the lack of ECMP/WECMP, they remain fairly > efficient within the hardware. > >> > >> However if you move away from large multi-chip systems, which hide > internal links which can only be debugged and monitored if you know the the > obscure, often different ways in which they are partially exposed to the > operator, and to a system of fixed form-factor, single chip systems, labels fall > apart at scale with high ECMP. > > > > I'm curious about this statement - have you hit practical ECMP issues > > with label switching at scale? > > Yes. Path enumeration when you use mult-tier Clos topologies within a PoP > causes you many, many problem. > Hi David, Can you be more specific please? Maybe some examples with numbers. I can see how you might run out of L2 rewrite/adjacency table space on a particular node if you enumerate every possible path downstream of it (especially on leaf nodes), cause that number is has a dependency on the size of the fabric in terms of the total number of links in the fabric (which balloons quickly). Let's focus on the alternate case then, (the deep label-stack one) At each node in the multi-tier Clos (which I assume is the Russ White's butterfly model? But any Clos or Benes fabric needs the same) you need to program a label to uniquely identify each egress interface ,so now there's this nice one-to-one relationship between label and egress interface. Now the depth of the label-stack depends on the number of hops the packet needs to traverse across the fabric. But how deep could the fabric realistically be? Even in the "butterfly" model with separate pods instead of leaf nodes I counted 9 hops (that's not ultra-deep is it)? It's the VM doing label imposition as programmed by the fabric controller (all in SW so can go as deep as you want) and all fabric nodes are just popping top label so no big deal. adam
