On Tue, 2 Apr 2024 at 18:25, Michael Hare via juniper-nsp <juniper-nsp@puck.nether.net> wrote:
> We're a US research and education ISP and we've been tasked for coming up > with an architecture to allow on premise DDoS scrubbing with an appliance. > As a first pass I've created an cleanL3VPN routing-instance to function as a > clean VRF that uses rib-groups to mirror the relevant parts of inet.0. It > is in production and is working great for customer learned BGP routes. It > falls apart when I try to protect a directly attached destination that has a > mac address in inet.0. I think I understand why and the purpose of this > message is to see if anyone has been in a similar situation and has > thoughts/advice/warnings about alternative designs. > > To explain what I see, I noticed that mac address based nexthops don't seem > to be copied from inet.0 into cleanL3VPN.inet.0. I assume this means that > mac-address based forwarding must be referencing inet.0 [see far below]. > This obviously creates a loop once the best path in inet.0 becomes a BGP /32. > For example when I'm announcing a /32 for 1.2.3.4 out of a locally attached > 1.2.3.0/26, traceroute implies the packet enters inet.0, is sent to 5.6.7.8 > as the nexthop correctly, arrives in cleanL3VPN which decides to forward to > 5.6.7.8 in a loop, even though the BGP /32 isn't part of cleanL3VPN [see > below], cleanL3VPN Is dependent on inet.0 for resolution. Even if I could > copy inet.0 mac addresses into cleanL3VPN, eventually the mac address would > age out of inet.0 because the /32 would no longer be directly connected. If > I want to be able to protect locally attached destinations so I think my > design is unworkable, I think my solutions are If I understand you correctly, the problem is not that you can't copy direct into CleanVRF, the problem is that ScrubberPE that does clean lookup in in CleanVRF, has label stack of [EgressPE TableLabel], instead of [EgressPE EgressCE], this causes the EgressPE to do IP lookup, which will then see the Direct/32 advertised by the scrubber, causing loop. While what you want is end-to-end MPLS lookup, so that egressPE MPLS lookup has egressMAC. I believe in BGP-LU you could fix this, without actually paying for duplicate RIB/FIB and without opportunistically copying routes to CleanVRF, every prefix would be scrubbable by default. You'd have per-ce for rest, but per-prefix for connected routes, I believe then you would have [EgressPE EgressMAC_CE] label for connected routes, so each host route would have their own label, allowing mac rewrite without additional local IP lookup. I'm not sure if this is the only way, I'm not sure if there would be a way in CleanVRF to force each direct/32 to have a label as well, avoiding the egress IP lookup loops. One doesn't immediately spring to mind, but technically implementation could certainly allow such mode. -- ++ytti _______________________________________________ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
