The redundant links to the customer site that traverse independent underlay carriers, and in some cases, equal-cost paths that we want to load-balance across, are the hard part. I’m not going to trust STP for that, and we aim for <3sec failover where we do have redundant paths. ERPS can handle the failover, but not the load-balancing. Any L2-over-L3 encapsulation protocol can handle the failover + ECMP features, but I need to do it at ~10G (~20G if ECMP) wire speed.
We provide IaaS services to our customers, which is why we’re stretching VLANs to them in the first place. Viewed from the IaaS perspective, this is a bunch of DC-DC connections… but relative to the overall network, the customer-prem devices fall into the traditional “CPE” category. (Most customers either just plug in bare fiber, or they connect to an intermediate carrier’s CPE.) Adam Thompson Consultant, Infrastructure Services [[MERLIN logo]] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D99ECB.C1CEDE50]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca> From: Joe Freeman <j...@netbyjoe.com> Sent: Wednesday, June 14, 2023 2:16 PM To: Adam Thompson <athomp...@merlin.mb.ca>; nanog <nanog@nanog.org> Subject: Re: 10G CPE w/VXLAN - vendors? I think you’re probably overthinking this a bit. Why do you need to extend your vxlan/evpn to the customer premise? There are a number of 1G/10G even 100G CPE demarc devices out there that push/pop tags, even q-in-q, or 802.1ad. Assuming you have some type of aggregation node you bring these back to, tie those tags to the appropriate EVPN instance at the aggregation point. Don’t extend anything but a management tag and an S-tag essentially to the device at the customer premise. You can even put that management tagged vlan in it’s own L3 segment, or a larger L3 network and impose security. This way you’re not exposing your whole service infrastructure to a bad actor that might unplug your cpe device and plug into your network directly. From: NANOG <nanog-bounces+joe=netbyjoe....@nanog.org<mailto:nanog-bounces+joe=netbyjoe....@nanog.org>> on behalf of Adam Thompson <athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>> Date: Wednesday, June 14, 2023 at 2:52 PM To: nanog <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: 10G CPE w/VXLAN - vendors? Hello, all. I’m having difficulty finding vendors, never mind products, that fit my need. We have a small but growing number of L2 (bridged) customers that have diverse fiber paths available, and, naturally, want to make use of them. We have a solution for this: we extend the edge of our EVPN VXLAN fabric right to the customer premise. The customer-prem device needs 4x10G SFP+ cages (2 redundant paths, plus LAG to customer), and the switches we currently use, Arista 7020Rs, are quite expensive if I’m deploying one one per customer. (Nice switches, but overkill here – I don’t need 40/100G, and I don’t need 24 SFP+ ports. And they still take forever to ship.) We use RFC7438 §6.3 “vlan-aware-bundle” mode, not §6.1 “vlan-based” mode, which limits our choices somewhat. I might be willing to entertain spinning up a separate VXLAN mesh using RFC7438 §6.1 (“vlan-based”) and static VTEPs if it saves me a lot of pain. However, I’m having trouble finding small & cheaper 1U (or even desktop/wallmount) devices that have 4 SFP+ cages, and can do VXLAN, in the first place. Who even makes CPE gear with SFP+ ports? (Other than Mikrotik CRS309-1G-8S+IN / CRS317-1G-16S+RM, which are nice, but our policy requires vendor support contracts, so… no-go.) Vendors? Model#s, if you happen to know any? Reply here or privately, whatever floats your boat – any pointers appreciated! Adam Thompson Consultant, Infrastructure Services [[MERLIN logo]] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D99EC2.B891B0A0]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>
