Hi everyone! I like to play with all the cool toys the devs give us,
because, you know, they are there, and it helps me learn. One of my
favorite walls to bang my head against is automatically connecting my
(OpenBSD-stable) laptop to the internet and automatically keeping it
connected as I open and close my laptop, change locations, etc. To make
this more of a challenge, I have wlan, wwan, and ethernet connections to
choose from (ethernet > wlan > wwan), and I like to connect via some
sort of VPN so I can be at the airport or a cafe and not worry too much
about the local folks listening in on my traffic.
I have all this working fine using ifstated, some scripts, and SSH based
VPN, though the CPU usage is a bit high and its not without hiccups.
Anyhow, I thought I'd try out a different way of accomplishing a similar
effect and see what happens. This time around I thought I'd try putting
my connections in separate routing domains so I can test their
connectivity separately and switch between them quickly. Then I layer
some PF config to NAT traffic from rdomain 0 to the rdomain I want to
use for internet access, i.e. rdomain 1 is wwan, rdomain 2 is wlan. That
all works great. As Darth Vader would say, all too easy.
So then I tried connecting to my vpn server (running OpenBSD) using
ikedv2 on rdomain 0 and that works great too. I have another NAT rule in
pf.conf to send traffic over the ipsec flow that ikedv2 negotiated. Ping
works, TCP requests to websites work, but UDP based DNS lookups do not.
I'm using a local unbound instance for DNS lookups, so I can work around
my lack of UDP-ness by configuring it to do TCP based lookups, but I'm
wondering if anyone might have some idea why TCP works with this setup
but UDP does not.
Some IPs and MAC addresses replaced with consistent, unique, obvious
fakes to protect the innocent without, hopefully, interfering with the
usefulness of the logs:
# cat /etc/pf.conf
match in all scrub (no-df random-id max-mss 1440)
pass out on egress to !egress:network nat-to (athn0:0) rtable 2
pass out on enc0 from vether0 nat-to vether0:0
Note: that IS the entire pf.conf, I'm passing everything while I test
this configuration out.
# cat /etc/iked.conf
ikev2 "vpn" active ipcomp \
from egress to 0.0.0.0/0 \
peer 104.xxx.xxx.xxx \
srcid client.ggr.com \
tag IKED
# cat /etc/iked.conf (on vpn server)
ikev2 "vpn" ipcomp \
from 0.0.0.0/0 to 10.0.0.0/8 \
from 0.0.0.0/0 to 172.16.0.0/12 \
from 0.0.0.0/0 to 192.168.0.0/16 \
peer any \
srcid server.ggr.com \
tag IKED
$ ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768
index 4 priority 0 llprio 3
groups: lo
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
inet 127.0.0.1 netmask 0xff000000
em0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:zz:zz:zz:zz:zz
index 1 priority 0 llprio 3
media: Ethernet autoselect (none)
status: no carrier
athn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> rdomain 2 mtu 1500
index 2 priority 4 llprio 3
lladdr 00:yy:yy:yy:yy:yy
index 2 priority 4 llprio 3
groups: wlan
media: IEEE802.11 autoselect (OFDM36 mode 11a)
status: active
ieee80211: nwid MyFakeNetwork chan 153 bssid f0:mm:mm:mm:mm:mm 53dBm wpakey
<not displayed> wpaprotos wpa1,wpa2 wpaakms psk wpaciphers tkip,ccmp
wpagroupcipher tkip
inet 10.0.0.136 netmask 0xffffff00 broadcast 10.0.0.255
enc0: flags=0<>
index 3 priority 0 llprio 3
groups: enc
status: active
vether0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr fe:ii:ii:ii:ii:ii
index 7 priority 0 llprio 3
groups: vether egress
media: Ethernet autoselect
status: active
inet 192.168.211.1 netmask 0xffffff00 broadcast 192.168.211.255
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33144
index 8 priority 0 llprio 3
groups: pflog
ppp0: flags=8010<POINTOPOINT,MULTICAST> rdomain 1 mtu 1500
index 27 priority 0 llprio 3
groups: ppp
$ route -T 0 -n show -inet
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Prio Iface
default 192.168.211.1 UGS 146528 860228 - 8 vether0
127.0.0.1 127.0.0.1 UHl 42 115625 32768 1 lo0
192.168.211/24 192.168.211.1 UC 0 0 - 4 vether0
192.168.211.1 fe:ii:ii:ii:ii:ii UHLl 1 367197 - 1 vether0
192.168.211.255 192.168.211.1 UHb 0 0 - 1 vether0
$ route -T 2 -n show -inet
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Prio Iface
default 10.0.0.1 UGS 238 54110 - 12 athn0
10.0.0/24 10.0.0.136 UC 1 795 - 4 athn0
10.0.0.1 64:kk:kk:kk:kk:kk UHLc 1 275 - 4 athn0
10.0.0.136 00:yy:yy:yy:yy:yy UHLl 0 6 - 1 athn0
10.0.0.255 10.0.0.136 UHb 0 18 - 1 athn0
$ route -T 2 exec host google.com 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:
google.com has address 172.217.9.238
google.com has IPv6 address 2607:f8b0:4006:801::200e
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.
# tcpdump -vvopn -i athn0
07:53:24.518765 10.0.0.136.36354 > 8.8.8.8.53: [udp sum ok] 58253+ A?
google.com. (28) (ttl 64, id 8337, len 56)
07:53:24.545896 8.8.8.8.53 > 10.0.0.136.36354: [udp sum ok] 58253 q: A?
google.com.1/0/0 google.com. A 172.217.9.238 (44) (ttl 45, id 16165, len 72)
07:53:24.546606 10.0.0.136.25323 > 8.8.8.8.53: [udp sum ok] 63849+ AAAA?
google.com. (28) (ttl 64, id 1415, len 56)
07:53:24.578293 8.8.8.8.53 > 10.0.0.136.25323: [udp sum ok] 63849 q: AAAA?
google.com. 1/0/0 google.com. AAAA 2607:f8b0:4006:801::200e (56) (ttl 45, id 4970,
len 84)
07:53:24.578567 10.0.0.136.46295 > 8.8.8.8.53: [udp sum ok] 59450+ MX?
google.com. (28) (ttl 64, id 20803, len 56)
07:53:24.604558 8.8.8.8.53 > 10.0.0.136.46295: 59450 q: MX? google.com. 5/0/0
google.com. MX alt4.aspmx.l.google.com. 50, google.com. MX[|domain] (ttl 45, id
10095, len 164)
$ route -T 0 exec host google.com 8.8.8.8
;; connection timed out; no servers could be reached
# tcpdump -vvopn -i enc0
07:50:36.745230 (authentic,confidential): SPI 0x5b1985a0: 192.168.211.1 >
104.xxx.xxx.xxx: 192.168.211.1.51470 > 8.8.8.8.53: [udp sum ok] 6983+ A? google.com.
(28) (ttl 64, id 42910, len 56) (ttl 64, id 21277, len 76, bad ip cksum 0! -> 7436)
07:50:36.770427 (authentic,confidential): SPI 0xa4554845: 104.xxx.xxx.xxx >
192.168.211.1: 8.8.8.8.53 > 192.168.211.1.51470: [udp sum ok] 6983 q: A?
google.com. 1/0/0 google.com. A 172.217.6.46 (44) (ttl 45, id 15649, len 72) (ttl 55,
id 21022, len 92)
To my untrained eye, it looks like the DNS lookup is going across the
ipsec flow ok, both request and reply, but for some reason I'm not
processing the reply correctly.
