Hi David, Great stuff (as usual), though I'm thinking it may be simpler to not use iptables, but something like: -- wg set "$INTERFACE" fwmark $table -- which marks the wireguard packets, and use that in "ip rule ..." logic, just examples ...
-- ip $proto route add "$1" dev "$INTERFACE" table $table ip $proto rule add not fwmark $table table $table -- The above snippets are taken from Jason's wg-quick linux bash script ... not easy to read but there are some gems in there: https://git.zx2c4.com/WireGuard/tree/src/tools/wg-quick/linux.bash#n175 Jason uses the UDP port 51820 as the default "fwmark" and "table" value. Lonnie > On Oct 9, 2018, at 3:54 PM, David Kerr <da...@kerr.net> wrote: > > I have been wanting to get access to my PBX over my failover tunnel for some > time now but didn't know how to get it done (when failover was not active -- > works when astlinux is in failover mode). This thread prompted me to try and > get it setup, inspired by Lonnie pointing out fwmark. Unfortunately what I > thought would be a quick exercise took several hours to get going. First a > diagram... > > [RemoteDev] > | public internet > --------------------------------------- > | eth0 | eth0 > [failover] wg0 --------- wg0 [astlinux] > 172.23.x.2 172.23.x.1 | eth1 192.168.x.1 > -------------------------- > | private LAN > [internal system] 192.168.x.y > > > failover could be astlinux or any linux that can act as a router/gateway > failover is connected to public internet through its eth0 interface > failover is connected to astlinux over wireguard. > astlinux is connected to public internet through its eth0 interface > astlinux is connected to failover over wireguard. > astlinux is connected to a private LAN 192.168.x.0/24 > > Desired behavior is to allow [RemoteDev] to access [astlinux] (ssh or https) > or [internal system] connecting through either astlinux eth0 or failover > eth0. For this to work [failover] must NAT inbound ssh or https over to > [astlinux] or [internal system]. That part is easy enough and if failover is > active everything works fine as return traffic is routed back over wireguard. > But I want it to work even if failover is not active. > > First, the wireguard feature to set fwmark on the interface does nothing to > help. I tried. And the documentation (https://www.wireguard.com/netns/) > states that this sets a mark on outbound traffic to the wireguard UDP port. > In other words the already encrypted packets going out through eth0 port > 51820. Actual packets flowing in/out of the tunnel are not marked. But the > principle of using fwmark is sound, we just need another way of marking the > packets. > > This is what worked for me... on the [astlinux] system... > > iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark > iptables -t mangle -A PREROUTING -i wg0 -j MARK --set-xmark 0x4/0x4 > iptables -t mangle -A PREROUTING -i wg0 -j CONNMARK --save-mark > iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark > ip route flush table 300 > ip route add table 300 default dev wg0 via 172.23.x.2 > ip rule add from 192.168.x.0/24 fwmark 0x4/0x4 table 300 priority 1000 > > Explanation... > • We are setting a firewall mark on all inbound traffic from wg0. > fwmark is a 32-bit unsigned and can be treated as a value or a 32-bit field. > I have chosen to treat it as a 32-bit field and using mask only set bit 2, > this means I can use other bits for other purposes in > iptables/netfilter/route if I want. You can pick any bit or simply use a > integer value. Whatever you do needs to be consistent and be aware of any > other uses of fwmark in your firewall. > • We must save this fwmark with the CONNMARK extension, so that we can > reattach the mark on related traffic. fwmarks do not attach to the IP > packet, they exist in the context of this system's kernel only. So > connection tracking is required to keep track of it. > • PREROUTING chain is called for all inbound traffic from all > interfaces. So replies from LAN through eth1 will come here too and this is > where we need to restore the fwmark on returning packets using the CONNMARK > connection tracking. I do that first before setting mark on wg0 traffic. > • That works for traffic passing through [astlinux] to the internal > LAN. But for traffic to astlinux (e.g. the web interface) the replies > originate on the astlinux box itself and therefore do not pass through > PREROUTING. Therefore we must restore the fwmark on the OUTPUT chain so that > locally generated packets are tagged if required before we get to routing. > https://www.frozentux.net/iptables-tutorial/iptables-tutorial.html#TRAVERSINGOFTABLES > The above tags packets that need to be routed back through the wireguard > tunnel. Now have to mess with the routing tables to act on the marks. > • Picking any free ip2route table (I randomly choose 300) delete > anything currently on that table. > • Add a default route for that table to send all traffic through the > wireguard tunnel > • Add an ip rule to select which packets should be routed through this > table based on the fwmark. Critical in this step is to only select packets > originating from the internal LAN -- ie, we are only interested in outbound > packets. if we don't do that then inbound packets from wg0 will also get > selected and we don't want that... because we did not duplicate the content > of the main routing table into this new table so there are no routes to local > destinations. > The above rules can be set on POST_UP in the astlinux wireguard script. And > deleted in the POST_DOWN (or PRE_DOWN, should not matter). Though beware > that if you reload the firewall, without also restarting wireguard, then the > above would get deleted. > > What might this look like in wireguard.script POST_UP section.... > > # Setup routing table for traffic originating on $interface so that > # we can set rules to route replies to that traffic over $interface > # assume /etc/rc.conf read and interface="$2" > if [ -n "$WAN_FAILOVER_SECONDARY_GW" ]; then > echo "WireGuard: set iptables and ip route table $WG0_TUNNEL_ROUTE_TABLE > for $interface to reply to inbound traffic via $WAN_FAILOVER_SECONDARY_GW" > WG0_TUNNEL_ROUTE_TABLE="300" > INTNET=$(netcalc $INTIP $INTNM | sed -n -r -e 's/^Network *: > *([0-9\.\/]+).*$/\1/p') > iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark > iptables -t mangle -A PREROUTING -i $interface -j MARK --set-xmark 0x4/0x4 > iptables -t mangle -A PREROUTING -i $interface -j CONNMARK --save-mark > iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark > ip route flush table $WG0_TUNNEL_ROUTE_TABLE > ip route add table $WG0_TUNNEL_ROUTE_TABLE default dev $interface via > $WAN_FAILOVER_SECONDARY_GW > ip rule add from $INTNET fwmark 0x4/0x4 table $WG0_TUNNEL_ROUTE_TABLE > priority 1000 > fi > > Don't forget to delete on POST_DOWN. > > IPv6 is left as homework exercise for the reader. > > Enjoy !! > > David > > On Sat, Oct 6, 2018 at 7:54 PM Lonnie Abelbeck <li...@lonnie.abelbeck.com> > wrote: > Yes, is all comes down to the routing at PBX2. > > Consider this ... the PC has IP 1.2.3.4, so the NAT forward will have a SRC > address of 1.2.3.4 when received by 172.29.253.2 on PBX2. If the routing on > PBX2 routes 1.2.3.4 back through the wireguard tunnel then it will work as > you want. On the other-hand if PBX2 routes 1.2.3.4 over it's EXT interface > then it will not work as you want. > > Probably the most elegant solution for routing on PBX2 is "policy routing" > using "ip rule ..." where traffic through the wireguard tunnel could have a > "fwmark" and add routing rules based on whether the packet traversed the > wireguard tunnel. I have only played with this ... all the hooks are > currently available using /mnt/kd/wireguard.script > https://doc.astlinux-project.org/userdoc:tt_wireguard_vpn#optional_action_script > > but if you are not familiar with policy-based routing in Linux, this takes > some research to get a handle on. > > Alternatively, if your public PC's are always off a know subnet, you could > add a static destination route on PBX2 to your PC's via the wireguard tunnel. > > Lonnie > > > > > On Oct 6, 2018, at 6:11 PM, Michael Knill > > <michael.kn...@ipcsolutions.com.au> wrote: > > > > Sorry Lonnie I am a little confused. > > The setup is as follows: > > > > PC -- [internet] -- PBX1 -- [WG VPN] -- PBX2 > > > > I can ping the private Wireguard PBX2 address (172.29.253.2) from PBX1 > > (172.29.253.2) > > So I want to NAT PBX1 EXTIF on a particular port to PBX2 WG IP 172.29.253.2. > > I have set up the NAT_FOREIGN_NETWORK for the entire private address space. > > > > Thanks > > > > Regards > > Michael Knill > > > > On 7/10/18, 12:01 am, "Lonnie Abelbeck" <li...@lonnie.abelbeck.com> wrote: > > > > > > > >> On Oct 5, 2018, at 10:29 PM, Michael Knill > >> <michael.kn...@ipcsolutions.com.au> wrote: > >> > >> Hi Group > >> > >> Im wanting to set up a NAT rule from NAT EXT to a Wireguard VPN endpoint. > >> Is this possible? > >> It does not seem to work with NAT EXT -> LAN. > >> If not, is there a custom rule I can try? > >> > >> Basically I want to SSH to the VPN endpoint directly, via the transit DR > >> server. > >> > >> Thanks so much. > > > > Hi Michael, short answer is yes, but depending on the routing. > > > > Start with a diagram ... > > > > public_1 -- pbx1 [ wg_1_ip ] -- wireguard -- [ wg_2_ip ] pbx2 -- public_2 > > > > > > My understanding is you want to SSH to wg_1_ip using public_2 ? Correct > > me if I mis-understood. > > > > Yes, a "NAT EXT -> LAN" on public_2 to wg_1_ip will work *only if* the > > SSH return path at pbx1 goes through the wireguard vpn. > > > > I have personally tried this when pbx1 was on failover using wireguard > > over LTE/4G, as such all pbx1 traffic was routed over wireguard, as such a > > "NAT EXT -> LAN" on public_2 to wg_1_ip worked since the SSH return packets > > passed over wireguard to pbx2. > > > > Tip -> Similar, but if a "NAT EXT -> LAN" on public_2 to a LAN IP on > > pbx1 you would need to set NAT_FOREIGN_NETWORK on pbx2 of the pbx1 LAN so > > it is NAT'ed on pbx2. > > > > Lonnie > > > > > > > > > > _______________________________________________ > > Astlinux-users mailing list > > Astlinux-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > > > Donations to support AstLinux are graciously accepted via PayPal to > > pay...@krisk.org. > > > > > > > > _______________________________________________ > > Astlinux-users mailing list > > Astlinux-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > > > Donations to support AstLinux are graciously accepted via PayPal to > > pay...@krisk.org. > > > > _______________________________________________ > Astlinux-users mailing list > Astlinux-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > Donations to support AstLinux are graciously accepted via PayPal to > pay...@krisk.org. > _______________________________________________ > Astlinux-users mailing list > Astlinux-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > Donations to support AstLinux are graciously accepted via PayPal to > pay...@krisk.org. _______________________________________________ Astlinux-users mailing list Astlinux-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/astlinux-users Donations to support AstLinux are graciously accepted via PayPal to pay...@krisk.org.
