Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi Daniel, On Mon, Nov 20, 2023, 03:05 Daniel Gröber wrote: > Hi Erin, Juliusz, > > On Sat, Nov 18, 2023 at 11:21:57AM +0100, Erin Shepherd wrote: > > On Sat, 18 Nov 2023, at 03:19, Daniel Gröber wrote: > > > That would be a problem as I specifically want to tie the source > address > > > filtering to this too. I'll have a look at the internals (if and) when > I > > > get around to starting work on this. > > > > Is tying source address filtering to the routing table the right thing to > > do here? It seems to me that it would cause issues similar to those we > > see more generally with Unicast Reverse Path Filtering > > IMO not providing a way to do source address filtering at the routing level > was the original sin :) > > There is certianly the multihoming challange to be overcome as traditional > BCP38 style filtering doesn't cut it in the general case. I have some ideas > on how to deal with this. > > I've done some experiments and found that in Linux multi-nexthop routes > actually match reverse path lookups (using nftables "rt") for _any_ of the > source interfaces involved. I think this can be used to build RFC 3704 > style Feasible Path Reverse Path Forwarding when the routing daemon > involved supports ECMP. > > This experiment is what got me interested in having via-wgpeer in the > routing table in the first place, once we have that we can apply the above > idea not just at the interface level but at the wg peer level. Neat. > > Can you think of a use-case where fpRPF isn't enough? > Yes. IMHO, the problem with RPF is that routing table doesn't reflect the network topology, but only a subset of it. I mean in topologies where multiple pathes are possible, you can choose to use or even learn only a subset of those pathes. And that does not mean that there are no other legitimate pathes exist, that other actors may choose to reach you. In that sense might be yes, the original sin is that the routing table doesn't reflect all the topology, not only the pathes we choose for egress. Not sure though if it is a sin, in that case routing table would be too overcomplicated. If I understand correctly, such fpRPF approach works only if you both learn all possible pathes and use all of them in a multi-nexthop route. But for example in the Internet with its advanced BGP announcement policies it is not true at all. So from my point of view it is good to split the topology definition (ingress decapsulation) and the chosen pathes (egress routing). Because it is related, but still different processes. So the system can be more flexible. Although we need to repeat common things and keep ingress and egress consistent/synced. At the same time we can use single protocol/configuration as a source of information to setup both of those processes in cases when it is ok to sacrifice flexibility for simplicity. Or for example the ingress part can be configured to use routing table as a source of topology information. Actually, when we turn on the RPF, we do something like that. But my point is that RPF (with its variations too) has its bounds and cannot be a universal solution, there is no silver bullet here. > It's also noteworthy that once we have this support for via-wgpeer it'd be > possible to apply ip-rule policy to the filtering decision. Perhaps that > gives some additional power for more fun use-cases :) > > On Sat, Nov 18, 2023 at 01:22:03PM +0100, Juliusz Chroboczek wrote: > > Issues are caused by the kernel performing filtering that the routing > > protocol is not aware of: it causes the routing daemon's routing table to > > no longer match the effective forwarding table (the kernel's routing > > table). That's the reason why uRPF breaks most routing protocols, that's > > the reason why we have trouble making Wireguard work with Babel, and also > > the reason behind https://github.com/jech/babeld/issues/111. > > Right on the money as always. This idea has been on my mind too. > > > Contrariwise, we can teach Babel to explicitly take into account the > > kernel features that we're interested in using. Thus, Babel could be > > aware of the restrictions placed on a wireguard interface, and > collaborate > > with Wireguard so that the routing table and the forwarding table remain > > congruent. I haven't looked at the issue in detail, but I believe that > > would be an interesting (short-term) research project, one that I would > be > > glad to collaborate with (but not necessarily lead, at least not right > now). > > Sounds interesting do you have a funding source in mind? > > > For the specific case of source address filtering, Babel already has an > > (implemented) extension to deal with source addresses, and I encourage > you > > to consider whether it can be used to deal with the issue at hand. > Please > > see https://arxiv.org/pdf/1403.0445.pdf and RFC 9079. > > I don't think I mentioned this to you yet, but I have another one of my > crazy ideas of doing something vaguely similar to BGP flowspec
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
On Sat, 18 Nov 2023, at 03:19, Daniel Gröber wrote: > Hi Alexander, > > On Thu, Nov 09, 2023 at 12:57:26PM +0100, Alexander Zubkov wrote: > > But as I understood the technology, it works only in one way (for > > outgoing packets) and the decapsulation should be processed separately, > > for example in case of VXLAN and MPLS they have their own tables. > > That would be a problem as I specifically want to tie the source address > filtering to this too. I'll have a look at the internals (if and) when I > get around to starting work on this. Is tying source address filtering to the routing table the right thing to do here? It seems to me that it would cause issues similar to those we see more generally with Unicast Reverse Path Filtering___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi Alexander, On Wed, Nov 22, 2023 at 12:17:49AM +0100, Alexander Zubkov wrote: > > Can you think of a use-case where fpRPF isn't enough? > > Yes. IMHO, the problem with RPF is that routing table doesn't reflect the > network topology, but only a subset of it. Right that is the fundamental problem, so my solution to that is: routing should "just represent the full network topology" :) As the routing protocol sees it anyway, since the whole point of RFP is to only allow paths that the routing system chooses. Do note that while I implement the topology information using ECMP routes there's no reason you actually have to use ECMP. You could still have regular routes in your (main) routing table and use a separate table with ECMP routes for RPF and this is very much something I want us to support. > I mean in topologies where multiple pathes are possible, you can choose > to use or even learn only a subset of those pathes. If I undestand correctly you're talking about (local) routing daemon policy here. Yes this is something you can do and my current approach of (abusing) ECMP only works when your routing policy satisfies some symmetry criteria. However as Juliusz pointed out integrating this idea into the routing protocol proper could allow using arbitrary policy without ever breaking RPF, but figuring out the details is (exciting) future work. > In that sense might be yes, the original sin is that the routing table > doesn't reflect all the topology, not only the pathes we choose for egress. > Not sure though if it is a sin, in that case routing table would be too > overcomplicated. Right routing table (modification) performance and clutter is certainly a reason to forgoe this approach but I find that for the kind of (small) networks I want to run and that many people might run using wireguard this is perfectly fine. > If I understand correctly, such fpRPF approach works only if you both learn > all possible pathes and use all of them in a multi-nexthop route. But for > example in the Internet with its advanced BGP announcement policies it is > not true at all. Right to deply fpRPF on a large scale you really need some kind of support from the routing protocol. AFAIK there's nothing like that for BGP yet? I don't think it's completely inapplicable either though, might still work for iBGP with appropriately designed routing policy. My interest lies mostly with doing this using babel though. > So from my point of view it is good to split the topology definition > (ingress decapsulation) and the chosen pathes (egress routing). Because it > is related, but still different processes. So the system can be more > flexible. Although we need to repeat common things and keep ingress and > egress consistent/synced. To me flexibility is only desirable insofar as it doesn't conflict with system security. Source address authenticity is an important property I wouldn't want to give up here. If it's easier to ignore source address filtering than it is to implement it nobody is going to do it (cf. the internet) and I think that's the crux of the problem with "encap". Wireguard gifted us this amazing state of source filtering being the easy default and I want to keep it that way. > my point is that RPF (with its variations too) has its bounds and cannot > be a universal solution, there is no silver bullet here. No, ofc. nothing we do can possibly "fix everything for everyone" but that's no reason not to try a new approach for a particular problem in a particular use-case :) --Daniel signature.asc Description: PGP signature ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi Erin, Juliusz, On Sat, Nov 18, 2023 at 11:21:57AM +0100, Erin Shepherd wrote: > On Sat, 18 Nov 2023, at 03:19, Daniel Gröber wrote: > > That would be a problem as I specifically want to tie the source address > > filtering to this too. I'll have a look at the internals (if and) when I > > get around to starting work on this. > > Is tying source address filtering to the routing table the right thing to > do here? It seems to me that it would cause issues similar to those we > see more generally with Unicast Reverse Path Filtering IMO not providing a way to do source address filtering at the routing level was the original sin :) There is certianly the multihoming challange to be overcome as traditional BCP38 style filtering doesn't cut it in the general case. I have some ideas on how to deal with this. I've done some experiments and found that in Linux multi-nexthop routes actually match reverse path lookups (using nftables "rt") for _any_ of the source interfaces involved. I think this can be used to build RFC 3704 style Feasible Path Reverse Path Forwarding when the routing daemon involved supports ECMP. This experiment is what got me interested in having via-wgpeer in the routing table in the first place, once we have that we can apply the above idea not just at the interface level but at the wg peer level. Neat. Can you think of a use-case where fpRPF isn't enough? It's also noteworthy that once we have this support for via-wgpeer it'd be possible to apply ip-rule policy to the filtering decision. Perhaps that gives some additional power for more fun use-cases :) On Sat, Nov 18, 2023 at 01:22:03PM +0100, Juliusz Chroboczek wrote: > Issues are caused by the kernel performing filtering that the routing > protocol is not aware of: it causes the routing daemon's routing table to > no longer match the effective forwarding table (the kernel's routing > table). That's the reason why uRPF breaks most routing protocols, that's > the reason why we have trouble making Wireguard work with Babel, and also > the reason behind https://github.com/jech/babeld/issues/111. Right on the money as always. This idea has been on my mind too. > Contrariwise, we can teach Babel to explicitly take into account the > kernel features that we're interested in using. Thus, Babel could be > aware of the restrictions placed on a wireguard interface, and collaborate > with Wireguard so that the routing table and the forwarding table remain > congruent. I haven't looked at the issue in detail, but I believe that > would be an interesting (short-term) research project, one that I would be > glad to collaborate with (but not necessarily lead, at least not right now). Sounds interesting do you have a funding source in mind? > For the specific case of source address filtering, Babel already has an > (implemented) extension to deal with source addresses, and I encourage you > to consider whether it can be used to deal with the issue at hand. Please > see https://arxiv.org/pdf/1403.0445.pdf and RFC 9079. I don't think I mentioned this to you yet, but I have another one of my crazy ideas of doing something vaguely similar to BGP flowspec with babel. Restricted to IP source/destination address, so no L4 stuff. I just want to represent firewall policy using ipv6 subtrees and distribute it in realtime using babel :) Unfortunately this is currently stalled due to an apparent nft rt match kernel bug preventing me from representing multiple possible outcomes since I want to support dropping, accepting but also stateful firewalling of matching flows. --Daniel ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
> Is tying source address filtering to the routing table the right thing to do > here? It seems to me that it would cause issues similar to those we see more > generally with Unicast Reverse Path Filtering Issues are caused by the kernel performing filtering that the routing protocol is not aware of: it causes the routing daemon's routing table to no longer match the effective forwarding table (the kernel's routing table). That's the reason why uRPF breaks most routing protocols, that's the reason why we have trouble making Wireguard work with Babel, and also the reason behind https://github.com/jech/babeld/issues/111. Contrariwise, we can teach Babel to explicitly take into account the kernel features that we're interested in using. Thus, Babel could be aware of the restrictions placed on a wireguard interface, and collaborate with Wireguard so that the routing table and the forwarding table remain congruent. I haven't looked at the issue in detail, but I believe that would be an interesting (short-term) research project, one that I would be glad to collaborate with (but not necessarily lead, at least not right now). For the specific case of source address filtering, Babel already has an (implemented) extension to deal with source addresses, and I encourage you to consider whether it can be used to deal with the issue at hand. Please see https://arxiv.org/pdf/1403.0445.pdf and RFC 9079. -- Juliusz ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi Alexander, On Thu, Nov 09, 2023 at 12:57:26PM +0100, Alexander Zubkov wrote: > I heard recently about the lightweight tunnel infrastructure in Linux > kernel (ip route ... encap ...). And I think this might be helpful in > the context of this thread. I hadn't seen that yet, thanks for pointing it out. > Linux kernel allows already to add encapsulation parameters to the route > entry in its table. So you do not need to create tunnel devices for > that. And wireguard encapsulation and destination might be added there > too. Right, I think ultimately it's going to come down to either technical constraints or in the absence of that, maintainer preference whether via-wgpeer or "encap wg" is the way. The idea is very similar anyway. > But as I understood the technology, it works only in one way (for > outgoing packets) and the decapsulation should be processed separately, > for example in case of VXLAN and MPLS they have their own tables. That would be a problem as I specifically want to tie the source address filtering to this too. I'll have a look at the internals (if and) when I get around to starting work on this. Thanks, --Daniel signature.asc Description: PGP signature ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hello all, I heard recently about the lightweight tunnel infrastructure in Linux kernel (ip route ... encap ...). And I think this might be helpful in the context of this thread. Linux kernel allows already to add encapsulation parameters to the route entry in its table. So you do not need to create tunnel devices for that. And wireguard encapsulation and destination might be added there too. But as I understood the technology, it works only in one way (for outgoing packets) and the decapsulation should be processed separately, for example in case of VXLAN and MPLS they have their own tables. Regards, Alexander Zubkov Qrator Labs On Mon, Sep 11, 2023 at 5:46 PM Maria Matejka via Bird-users wrote: > > Hello! > > On 8/29/23 00:13, Daniel Gröber wrote: > > On Mon, Aug 28, 2023 at 07:40:51PM +0200, Juliusz Chroboczek wrote: > > I've read the whole discussion, and I'm still not clear what advantages > the proposed route attribute has over having one interface per peer. Is > it because interfaces are expensive in the Linux kernel? Or is there some > other reason why it is better to run all WG tunnels over a single interface? > > Off the top of my head UDP port exhaustion is a scalability concern here, > > For enterprise setups, this very easily _can_ get a scalability concern > fairly easily. > > One wg-device per-peer means we need one UDP port per-peer and since > currently binding to a specific IP is also not supported by wg (I have a > patch pending for this though) there's no good way to work around this. > > There is a theoretical frankenstein approach, running a virtual machine > (maybe netns is enough) for each of the public IP address, and connect them > by veth. You do not want to do this, but theoretically, it should work. > > Frankly having tons of interfaces is just an operational PITA in all sorts > of ways. Apart from the port exhaustion having more than one wg device also > means I have to _allocate_ a new port for each node in my managment system > somehow instead of just using a static port for the entire network. This > gets dicy fast as I want to move in the direction of dynamic peering as in > tinc. > > Even with my 6 machines running in weird locations, it's a mess. > > All of that could be solved, but I would also like to get my wg+babel VPN > setup deployed more widely at some point and all that friction isn't going > to help with that so I'd rather have this supported properly. > > All in all, I would also like to see this setup deployed worldwide. If we > could somehow help on the BIRD side, please let us know. > > Thank you for bringing this up. > > -- > Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o. ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hello! On 8/29/23 00:13, Daniel Gröber wrote: On Mon, Aug 28, 2023 at 07:40:51PM +0200, Juliusz Chroboczek wrote: I've read the whole discussion, and I'm still not clear what advantages the proposed route attribute has over having one interface per peer. Is it because interfaces are expensive in the Linux kernel? Or is there some other reason why it is better to run all WG tunnels over a single interface? Off the top of my head UDP port exhaustion is a scalability concern here, For enterprise setups, this very easily _can_ get a scalability concern fairly easily. One wg-device per-peer means we need one UDP port per-peer and since currently binding to a specific IP is also not supported by wg (I have a patch pending for this though) there's no good way to work around this. There is a theoretical frankenstein approach, running a virtual machine (maybe netns is enough) for each of the public IP address, and connect them by veth. You do not want to do this, but theoretically, it should work. Frankly having tons of interfaces is just an operational PITA in all sorts of ways. Apart from the port exhaustion having more than one wg device also means I have to _allocate_ a new port for each node in my managment system somehow instead of just using a static port for the entire network. This gets dicy fast as I want to move in the direction of dynamic peering as in tinc. Even with my 6 machines running in weird locations, it's a mess. All of that could be solved, but I would also like to get my wg+babel VPN setup deployed more widely at some point and all that friction isn't going to help with that so I'd rather have this supported properly. All in all, I would also like to see this setup deployed worldwide. If we could somehow help on the BIRD side, please let us know. Thank you for bringing this up. -- Maria Matejka (she/her) | BIRD Team Leader | CZ.NIC, z.s.p.o. ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi, IMO, a good tunnel solution may be if what is now called AllowedIPs, were functionally split into: - AcceptIPS (to be different from AllowedIPs) - RouteIPs Perhaps with a default shorthand of, say, IPs, setting both, as AllowedIPs historically caused confusion wrt. it's meaning. Wireguard API is a bit clunky, but I think one could dynamically manage routes in reasonably efficient ways without extra interfaces and layers. Not sure if it would fullfill all reasonably achievable goals. Don't really have the time to implement anything, and I'm sure it would not be easy, so just a possible tip to consider. Regards, Ivan Labáth On Tue, Aug 29, 2023 at 12:13:12AM +0200, Daniel Gröber wrote: > Hi Juliusz, > > On Mon, Aug 28, 2023 at 07:40:51PM +0200, Juliusz Chroboczek wrote: > > I've read the whole discussion, and I'm still not clear what advantages > > the proposed route attribute has over having one interface per peer. Is > > it because interfaces are expensive in the Linux kernel? Or is there some > > other reason why it is better to run all WG tunnels over a single interface? > > Off the top of my head UDP port exhaustion is a scalability concern here, > just as an example, not that I'd actually ever need that many peers in my > network :) > > One wg-device per-peer means we need one UDP port per-peer and since > currently binding to a specific IP is also not supported by wg (I have a > patch pending for this though) there's no good way to work around this. > > Frankly having tons of interfaces is just an operational PITA in all sorts > of ways. Apart from the port exhaustion having more than one wg device also > means I have to _allocate_ a new port for each node in my managment system > somehow instead of just using a static port for the entire network. This > gets dicy fast as I want to move in the direction of dynamic peering as in > tinc. > > Other than that my `ip -br a` output is getting unmanagably long and having > more than one device means I have to keep ACL lists in sync all over the > system. This is a problem for daemons that don't support automatic reload > (babeld for example :P). I also have to sync the set of interface to > nftables which is easy to get wrong as it's still manual in my setup. > > All of that could be solved, but I would also like to get my wg+babel VPN > setup deployed more widely at some point and all that friction isn't going > to help with that so I'd rather have this supported properly. > > --Daniel ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi Juliusz, On Mon, Aug 28, 2023 at 07:40:51PM +0200, Juliusz Chroboczek wrote: > I've read the whole discussion, and I'm still not clear what advantages > the proposed route attribute has over having one interface per peer. Is > it because interfaces are expensive in the Linux kernel? Or is there some > other reason why it is better to run all WG tunnels over a single interface? Off the top of my head UDP port exhaustion is a scalability concern here, just as an example, not that I'd actually ever need that many peers in my network :) One wg-device per-peer means we need one UDP port per-peer and since currently binding to a specific IP is also not supported by wg (I have a patch pending for this though) there's no good way to work around this. Frankly having tons of interfaces is just an operational PITA in all sorts of ways. Apart from the port exhaustion having more than one wg device also means I have to _allocate_ a new port for each node in my managment system somehow instead of just using a static port for the entire network. This gets dicy fast as I want to move in the direction of dynamic peering as in tinc. Other than that my `ip -br a` output is getting unmanagably long and having more than one device means I have to keep ACL lists in sync all over the system. This is a problem for daemons that don't support automatic reload (babeld for example :P). I also have to sync the set of interface to nftables which is easy to get wrong as it's still manual in my setup. All of that could be solved, but I would also like to get my wg+babel VPN setup deployed more widely at some point and all that friction isn't going to help with that so I'd rather have this supported properly. --Daniel signature.asc Description: PGP signature ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
On Mon, Aug 28, 2023 at 1:41 PM Juliusz Chroboczek wrote: > I've read the whole discussion, and I'm still not clear what advantages > the proposed route attribute has over having one interface per peer. Is > it because interfaces are expensive in the Linux kernel? Or is there some > other reason why it is better to run all WG tunnels over a single interface? Why manage n^2 tunnels and allocate n^2 /30 CIDRs when you can just have one tunnel and a single subnet for a full mesh? IMO, the latter should be a feature differentiating Wireguard from other solutions to creating a mesh VPN. That is, in fact, the whole reason I dropped OpenVPN for it. Kyle ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Daniel, Kyle, I've read the whole discussion, and I'm still not clear what advantages the proposed route attribute has over having one interface per peer. Is it because interfaces are expensive in the Linux kernel? Or is there some other reason why it is better to run all WG tunnels over a single interface? -- Juliusz ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi Kyle, On Mon, Aug 28, 2023 at 11:40:48AM -0400, Kyle Rose wrote: > On Sat, Aug 19, 2023 at 5:25 PM Daniel Gröber wrote: > > Having read Kyle's use-case I'm thinking my original plan to extend the wg > > internal source-address filtering to use a rt lookup with our new attribute > > would not be maximally useful so now my thinking is we should just have a > > boolean toggle to disable it explicitly per device. > > If there is interest among the maintainers in eventually merging a > change with a per-interface knob to turn off the source IP check, I > will go through the trouble of putting together an initial pass at > this. I don't want to spend the time if there is firm opposition to > the idea. I think just a patch to turn off the wg source IP check is not very useful at the moment. It would encourage bad source IP filtering practices when multiple peers are involved as no mechanism for identifying the sending peer is available at the policy routing or netfilter level currently. I think such a patch would have to get merged after some kind of mechanism to identify and filter based on the sending wg peer is available. So if you want to move this along I would suggest working on this first. Since I'm also interested in having this feature I'm happy collaborate. It's just hard to find the motivation for writing more wg patches when my pending ones have (mostly) been lying around for a year without a response, but if you're also keen on this feature I'm sure it's easier to stay motivated together :) If my kernel patches go ignored for too long too I'll probably just resort to getting a forked DKMS wireguard module into Debian with this work. Perhaps that approach (or a package in a different distro) would work for your use-case too? --Daniel ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
On Sat, Aug 19, 2023 at 5:25 PM Daniel Gröber wrote: > Having read Kyle's use-case I'm thinking my original plan to extend the wg > internal source-address filtering to use a rt lookup with our new attribute > would not be maximally useful so now my thinking is we should just have a > boolean toggle to disable it explicitly per device. If there is interest among the maintainers in eventually merging a change with a per-interface knob to turn off the source IP check, I will go through the trouble of putting together an initial pass at this. I don't want to spend the time if there is firm opposition to the idea. Thanks, Kyle ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Daniel,
I attempted several times to send the following message to the mailing
list nearly two months ago, but they all landed in moderation for some
reason and were silently dropped after a few days. You are not the
only one who wants this functionality for mesh VPNs. I note the email
you subsequently replied to (by Bernd) also didn't show up on the
list, suggesting I am not the only one whose messages are being
inexplicably jailed.
Kyle
I really like the straightforward configurability of Wireguard
out-of-the-box. It was astonishingly easy to configure a mesh to
replace my previous hub-and-spoke OpenVPN setup. Thank you for making
this easy.
That said, I'd like the ability to use Linux's policy routing engine
to allow for more complex packet flows across the VPN that are
currently incompatible with Wireguard's internal packet handling. For
example, let's say I have 4 nodes and want to be able to use each of
the nodes as the default gateway for different types of flows.
Modifying the sender side to respect a route's gateway is
straightforward:
--- a/drivers/net/wireguard/allowe
dips.c
+++ b/drivers/net/wireguard/allowe
dips.c
@@ -6,6 +6,8 @@
#include "allowedips.h"
#include "peer.h"
+#include
+
enum { MAX_ALLOWEDIPS_BITS = 128 };
static struct kmem_cache *node_cache;
@@ -356,10 +358,18 @@ int wg_allowedips_read_node(struct
allowedips_node *node, u8 ip[16], u8 *cidr)
struct wg_peer *wg_allowedips_lookup_dst(stru
ct allowedips *table,
struct sk_buff *skb)
{
- if (skb->protocol == htons(ETH_P_IP))
- return lookup(table->root4, 32, _hdr(skb)->daddr);
- else if (skb->protocol == htons(ETH_P_IPV6))
- return lookup(table->root6, 128, _hdr(skb)->daddr);
+ struct rtable *rt = skb_rtable(skb);
+ if (rt->rt_uses_gateway) {
+ if (rt->rt_gw_family == AF_INET)
+ return lookup(table->root4, 32, >rt_gw4);
+ else if (rt->rt_gw_family == AF_INET6)
+ return lookup(table->root6, 128, >rt_gw6);
+ } else {
+ if (skb->protocol == htons(ETH_P_IP))
+ return lookup(table->root4, 32, _hdr(skb)->daddr);
+ else if (skb->protocol == htons(ETH_P_IPV6))
+ return lookup(table->root6, 128, _hdr(skb)->daddr);
+ }
return NULL;
}
The problem is that reply packets will be dropped via the source
address check for all but the peer with the default route listed in
AllowedIPs. The way the trie code works means a highly-invasive change
would be needed to allow for multiple peers to be associated with a
given prefix: I suspect any further complication (not to mention
possible additional data structure bloat) is undesirable, and anyway I
am looking to bypass most of the complexity created by Wireguard's
parallel packet routing infrastructure and instead leverage the far
more flexible Linux policy routing engine.
At this point, what I'd like to do is be able to skip the source
address check by configuration, instead relying on rp_filter and
firewall rules to reject bogus or unwanted packets. With such a config
knob, AllowedIPs would be used only for selecting the destination peer
based on the packet daddr or the route's gateway. For a mesh like I
described above, I would configure only a gateway IP for each peer in
AllowedIPs and use the policy routing engine for all other packet
routing behavior.
I appreciate that Wireguard works the way it does most likely because
routing is configured differently across the wide variety of devices
it supports (and in some cases may be unavailable to users), so I
don't think the AllowedIPs source address check should be removed by
default; but it would be nice if I could turn it off and rely on other
mechanisms in the kernel that would allow for more flexibility.
On Sat, Aug 19, 2023 at 10:05 AM Daniel Gröber wrote:
>
> Hi wireguard, birds, and babelers,
>
> tl;dr I want to add a new Linux route attribute (think "via $wgpeer") to
> supplement wireguard's internal AllowedIPs logic for both routing and
> source address filtering.
>
> I've been pondering how to better integrate wireguard into dynamic routing
> daemons, particularly BIRD and babeld. Essentially we want to be able to
> dynamically add/remove AllowedIPs depending on current reachability and/or
> link quality stats.
>
> Looking at the wg netlink API I see two major efficiency/scalability
> problems: 1) there is no way to be notified of changes in AllowedIPs made
> by other processes meaning we have to do periodic scans and 2) a peer's
> AllowedIPs set can only be replaced wholesale, not modified
> incrementally. This is problematic as "someone" might, in the worst case,
> want to install an entire internet routing table's worth of AllowedIPs and
> the set will likely change frequently. FYI: The IPv4 table has ~1M entries
> at present, yikes.
>
> Assuming external AllowedIPs
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi Steffen, from the questions you ask I fear you've misunderstood my intention to "replace" AllowedIPs. I'm strictly talking of the _implementation_ (allowedips.c) in Linux particular. The netlink interface would naturally stay unchanged. On Sat, Aug 19, 2023 at 10:00:17PM +0200, Steffen Vogel wrote: > Interesting ideas! I am wondering if this complexity is really necessary? I think so, right now wg's behaviour just prevents a number of advanced use-cases which is a pitty. > My biggest concern about the introduction of a route attribute is that > this adds complexity for users. WireGuard's simplicity (and portability) > have been important factors for its success. Nothing would change for users that don't use this feature. > A route attribute would introduce another source for the crypto-routing > peer selection process. What happens if the two mechanisms select > different peers? Which one would have precedence? If you read my previous mail carefully you'll find I specified how this would shake out. The rt approach will necessarily have to override AllowedIPs to be useful. > Similarly also for incoming packets. WireGuard's current principle is > really easy to understand. If the source address in in the peers > AllowedIP list, we will accept the packet. If not its discarded. This is > a central part of WireGuard's crypto-key routing feature which would > become more complex. Having read Kyle's use-case I'm thinking my original plan to extend the wg internal source-address filtering to use a rt lookup with our new attribute would not be maximally useful so now my thinking is we should just have a boolean toggle to disable it explicitly per device. Then users can do whatever fancy rt (reverse-path) lookups they want with nft. I'm particularly happy that nft rt lookups will actually do the right thing with multipath/ECMP routes (any of the involved devices satisfies the lookup) so this should actually work out fine in my case at least. Mmore prototyping is required here though. > A second concern I have with the use of route attributes is limited > portability. Not all platforms support them. How do we handle WireGuard > userspace implementations? No need. The Linux's rt table is Linux specific I have no plans to introduce this on other platforms, that's for other intersted souls to tackle. Again "replace" was referring to implementation concerns. > - Networks which are found in a Peers AllowedIP list will be installed as a > kernel route That's configurable. I always turn this off when dealing with rt daemons. > - Kernel routes with the peers unique IP address as next-hop will be added to > the Peers AllowedIPs list. > > This rather simple feature allows user to pair cunicu with a software > routing daemon like Bird2 while using a single WireGuard interface with > multiple peer-to-peer links. Sounds like you do what I want to do at kernel level in userspace, then at least we can agree this is a useful thing :) > In my setup a periodic synchronization worked fine. But I agree that it > would be nice if we could have a Netlink multicast group for subscribing > to changes like we also have for other parts of the Linux network stack > like routing tables, or link states. This feature was already discussed > on the WireGuard mailing list [7]. But unfortunately the patch was never > accepted. Maybe we can revisit this patch? Sounds like a plan, I'll have a look at it. > [1] Others planned features are: > - IP-autoconfiguration by deriving link-local addresses from peers public keys That's been discussed so many times before on the ML and someone always realises Jason is right and there's no point to this in the end. Key distribution is the crux of the problem. --Daniel ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi Daniel, Interesting ideas! I am wondering if this complexity is really necessary? How many routes do you have per peer? In my personal setup I have maximum of 1-100 routes per peer which I can handle with the current API quite comfortably. My biggest concern about the introduction of a route attribute is that this adds complexity for users. WireGuard's simplicity (and portability) have been important factors for its success. A route attribute would introduce another source for the crypto-routing peer selection process. What happens if the two mechanisms select different peers? Which one would have precedence? Similarly also for incoming packets. WireGuard's current principle is really easy to understand. If the source address in in the peers AllowedIP list, we will accept the packet. If not its discarded. This is a central part of WireGuard's crypto-key routing feature which would become more complex. Also implementation wise I would have doubts: Should WireGuard itself perform route lookups to determine which packets will be accepted? Or does WireGuard needs to synchronize the kernel routing table with its internal data structures itself? A second concern I have with the use of route attributes is limited portability. Not all platforms support them. How do we handle WireGuard userspace implementations? I've tackled this problem in a userspace daemon. The synchronisation of a kernel routing table with a WireGuard AllowedIPs settings can be done by cunicu's route synchronization feature: https://cunicu.li/docs/features/rtsync The route synchronization feature keeps the kernel routing table in sync with WireGuard's AllowedIPs setting. This synchronization is bi-directional: - Networks which are found in a Peers AllowedIP list will be installed as a kernel route - Kernel routes with the peers unique IP address as next-hop will be added to the Peers AllowedIPs list. This rather simple feature allows user to pair cunicu with a software routing daemon like Bird2 while using a single WireGuard interface with multiple peer-to-peer links. I am assigning each WireGuard interface a link-local address which is derived from the peers public key. I am using the peers link-local address as the next-hop in my routing daemon to differentiate to which Peer the AllowedIP entry must be added. I am keeping track of the kernel's routing table and AllowedIPs by regularly polling the kernel. As the route synchronisation is just one of cunicu's features [1], I have a central "watcher" routine in cunicu which observes any modification the the WireGuard interfaces and dispatches events which the individual features then can hook into. These observations are not limited to the AllowedIPs but basically any state of the WireGuard interface. E.g. last handshake time or per/peer traffic counters. In my setup a periodic synchronization worked fine. But I agree that it would be nice if we could have a Netlink multicast group for subscribing to changes like we also have for other parts of the Linux network stack like routing tables, or link states. This feature was already discussed on the WireGuard mailing list [7]. But unfortunately the patch was never accepted. Maybe we can revisit this patch? I would also be a big supported of extending the netlink API for supporting incremental updates the AllowedIP lists. The netlink APIs are already different for each platform. So extending it for one platform wouldn't hurt here. Unfortunately, I have far too many ideas for cunicu and limited time to realize them all. So I've recently moved the whole cunicu project into its own organization at GitHub/Codeberg [6] in attempt to find more contributors. Best regards, Steffen (stv0g) [1] Others planned features are: - Endpoint discovery via ICE/STUN/TURN - Peer discovery - IP-autoconfiguration by deriving link-local addresses from peers public keys I have a lot more ideas here like integrating my - Go babel routing implementation [2] - or Rosenpass PQC key-exchange [3] - or performing proper path-MTU discovery using DPLPMTUD [4] - or using hardware tokens, TPMs, secure enclaves to rotate pre-shared keys backed by a hardware source-of-trust [5] [2] https://github.com/cunicu/go-babel [3] https://github.com/cunicu/go-rosenpass [4] https://github.com/cunicu/go-pmtud [5] https://github.com/cunicu/go-skes [6] https://codeberg.org/cunicu [7] https://lists.zx2c4.com/pipermail/wireguard/2021-January/006318.html On Saturday, August 19, 2023 16:02 CEST, Daniel Gröber wrote: > Hi wireguard, birds, and babelers, > > tl;dr I want to add a new Linux route attribute (think "via $wgpeer") to > supplement wireguard's internal AllowedIPs logic for both routing and > source address filtering. > > I've been pondering how to better integrate wireguard into dynamic routing > daemons, particularly BIRD and babeld. Essentially we want to be able to > dynamically add/remove AllowedIPs depending on current
Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute
Hi Bernd, On Sat, Aug 19, 2023 at 07:50:38PM +0200, Bernd Naumann wrote: > Chances are high I do miss something, but I've just set AllowedIPs to > 0.0.0.0/0 and ::/0 and just used the routing protocol of my choice and > filters to select which routes got exported and imported... :shrug: Right, let me expand a bit. You are absolutely right, right now if you want to use wg with dynamic routing daemons you essentially have to have one wg tunnel per remote node with AllowedIPs=::/0 and that works just fine at small scales. The idea here is that we would like to go back to having just one tunnel for all nodes involved in this particular network instead, due to general operations scalability, mine is a mesh network so the number of tunnels gets rather large quickly :) Lots of tunnels suck for various reasons, monitoring if they're all up and configured properly is one example but my understanding from previous discussions is the performance is probably not ideal either. --Daniel ___ Babel-users mailing list Babel-users@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users
