Hi, I did some further investigations on this issue and found out that ist has nothin' to do with isakmp or rdomains. The problem seems just to be related to pf.
I did troubleshooting with a reduced config. em1: 10.0.3.1/24 rdomain 1 em2: 10.0.3.1/24 rdomain 2 Workstation1 (ws1): 10.0.3.10 attached to em1 Workstation2 (ws2): 10.0.3.10 attached to em2 Well, I missed the fact that unlike Check Point OpenBSD seems to have the same order for inspection modules on both incoming and outgoing interface (1. NAT, 2. Packet Filter). Nevertheless there seems to be still an issue with pf (or at least with my comprehension :) I started with the following pf config: block drop in log all match in on em1 from 10.0.3.0/24 to 10.0.5.0/24 rdr-to 10.0.3.0/24 bitmask match out on em1 from 10.0.3.0/24 to 10.0.3.0/24 nat-to 10.0.5.0/24 bitmask match in on em2 from 10.0.3.0/24 to 172.16.1.0/24 rdr-to 10.0.3.0/24 bitmask match out on em2 from 10.0.3.0/24 to 10.0.3.0/24 nat-to 172.16.1.0/24 bitmask pass in on em1 from 10.0.3.0/24 to 10.0.3.0/24 rtable 2 pass in on em2 from 10.0.3.0/24 to 10.0.3.0/24 rtable 1 pass out on em1 from 10.0.3.0/24 to 10.0.3.0/24 pass out on em2 from 10.0.3.0/24 to 10.0.3.0/24 Whe I tried to ping 10.0.5.10 from ws1 (10.0.3.10) or 172.16.1.10 from ws2 (10.0.3.10) the outgoing NAT (nat-to) wasn't done. But the packed was forwarded with the original NAT source. When I thought that NAT is done before filtering I changed the access part of the policy to: pass in on em1 from 10.0.3.0/24 to 10.0.3.0/24 rtable 2 pass in on em2 from 10.0.3.0/24 to 10.0.3.0/24 rtable 1 pass out on em1 from 10.0.5.0/24 to 10.0.3.0/24 pass out on em2 from 172.16.1.0/24 to 10.0.3.0/24 Everything I done fine now. But if NAT is applied before filtering, and the source address had been rewritten before the packet filter match, the packet should be dropped instead of being forwarded in the first case, shouldn't it? Thx in advance! Cheers, Bernd > -----Urspr|ngliche Nachricht----- > Von: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] Im Auftrag von > Bornkessel, Bernd [ICP DE - Schwalbach] > Gesendet: Mittwoch, 2. Mai 2012 15:47 > An: misc@openbsd.org > Betreff: NAT within VPN does not work (pf, isakmpd, rdomains) > > Hi, > > as we have to connect several customers and branches with overlapping > routing/encryption domains, I started creating a box using the rdomain > feature of OpenBSD. > Routing between rdomains using pf is working flawlessly: NAT is applied in > both directions and it works even with fully overlapping routing domains. > Routing between rdomains and a VPN destination (isakmpd running in rdomain > 0) is working as long as no NAT is tried to be applied. > > Network configuration of the OBSD box: > > # ifconfig > > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33152 > priority: 0 > groups: lo > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9 > inet 127.0.0.1 netmask 0xff000000 > em0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> > mtu 1500 > lladdr 00:0c:29:7c:14:1a > priority: 0 > groups: egress > media: Ethernet autoselect (1000baseT full-duplex,master) > status: active > inet 88.77.88.61 netmask 0xfffffff0 broadcast 88.77.88.63 > inet6 fe80::20c:29ff:fe7c:141a%em0 prefixlen 64 scopeid 0x1 > em1: > flags=28b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,NOIN > ET6> rdomain 1 mtu 1500 > lladdr 00:0c:29:7c:14:24 > priority: 0 > media: Ethernet autoselect (1000baseT full-duplex,master) > status: active > inet 10.0.3.2 netmask 0xffffff00 broadcast 10.0.3.255 > em2: > flags=28b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,NOIN > ET6> rdomain 2 mtu 1500 > lladdr 00:0c:29:7c:14:2e > priority: 0 > media: Ethernet autoselect (1000baseT full-duplex,master) > status: active > inet 10.0.3.2 netmask 0xffffff00 broadcast 10.0.3.255 > em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > lladdr 00:0c:29:7c:14:38 > priority: 0 > media: Ethernet autoselect (1000baseT full-duplex,master) > status: active > inet 192.168.230.2 netmask 0xfffffff0 broadcast 192.168.230.15 > inet6 fe80::20c:29ff:fe7c:1438%em3 prefixlen 64 scopeid 0x4 > enc0: flags=41<UP,RUNNING> > priority: 0 > groups: enc > status: active > lo1: flags=28049<UP,LOOPBACK,RUNNING,MULTICAST,NOINET6> rdomain 1 mtu > 33152 > priority: 0 > groups: lo > inet 127.0.0.1 netmask 0xff000000 > lo2: flags=28049<UP,LOOPBACK,RUNNING,MULTICAST,NOINET6> rdomain 2 mtu > 33152 > priority: 0 > groups: lo > inet 127.0.0.1 netmask 0xff000000 > pfsync0: flags=41<UP,RUNNING> mtu 1500 > priority: 0 > pfsync: syncdev: em3 maxupd: 128 defer: off > groups: carp pfsync > pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33152 > priority: 0 > groups: pflog > carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > lladdr 00:00:5e:00:01:0a > priority: 0 > carp: MASTER carpdev em0 vhid 10 advbase 1 advskew 0 > groups: carp > status: master > inet6 fe80::200:5eff:fe00:10a%carp0 prefixlen 64 scopeid 0xc > inet 88.77.88.60 netmask 0xffffff00 broadcast 88.77.88.255 > carp1: flags=28843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,NOINET6> > rdomain 1 mtu 1500 > lladdr 00:00:5e:00:01:14 > priority: 0 > carp: MASTER carpdev em1 vhid 20 advbase 1 advskew 0 > groups: carp > status: master > inet 10.0.3.1 netmask 0xffffff00 broadcast 10.0.3.255 > carp2: flags=28843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,NOINET6> > rdomain 2 mtu 1500 > lladdr 00:00:5e:00:01:1e > priority: 0 > carp: MASTER carpdev em2 vhid 30 advbase 1 advskew 0 > groups: carp > status: master > inet 10.0.3.1 netmask 0xffffff00 broadcast 10.0.3.255 > > em0 is connected to the Internet (default routing domain 0). em2 is one of > the internal networks in rdomain 2. This network is used as the encryption > domain for the vpn. > > The routing tables look like this: > > # route -T 0 -n show -inet > Routing tables > > Internet: > Destination Gateway Flags Refs Use Mtu Prio > Iface > default 88.77.88.49 UGS 0 70 - 8 > em0 > 10.0.3/24 127.0.0.1 UGS 38 76 33152 8 > lo0 > 10.0.4/24 127.0.0.1 UGS 0 0 33152 8 > lo0 > 127/8 127.0.0.1 UGRS 0 0 33152 8 > lo0 > 127.0.0.1 127.0.0.1 UH 3 0 33152 4 > lo0 > 192.168.230.0/28 link#4 UC 0 0 - 4 > em3 > 88.77.88/24 link#12 UC 0 0 - 4 > carp0 > 88.77.88.48/28 link#1 UC 2 0 - 4 > em0 > 88.77.88.49 link#1 UHLc 1 0 - 4 > em0 > 88.77.88.52 00:0c:29:9c:d3:02 UHLc 0 448 - 4 > em0 > 88.77.88.60 88.77.88.60 UH 0 0 - 4 > carp0 > 224/4 127.0.0.1 URS 0 0 33152 8 > lo0 > # > > # route -T 2 -n show -inet > Routing tables > > Internet: > Destination Gateway Flags Refs Use Mtu Prio > Iface > 10.0.3/24 link#3 UC 1 0 - 4 > em2 > 10.0.3.10 00:0c:29:89:77:e2 UHLc 2 461 - 4 > em2 > 10.0.4/24 10.0.3.10 UGS 0 2 - 8 > em2 > 127.0.0.1 127.0.0.1 UH 2 0 33152 4 > lo2 > 192.168.33/24 127.0.0.1 UGS 0 1 33152 8 > lo2 > 88.77.88.48/28 127.0.0.1 UGS 0 0 33152 8 > lo2 > # > > > There is a minimalistic set of pf rules configured: > > # pfctl -s rules > block drop in log all > pass in quick on em2 proto tcp from any to any port = ssh flags S/SA pass > on lo0 all no state pass on lo1 all no state pass on lo2 all no state pass > on em0 proto carp all pass on em1 proto carp all pass on em2 proto carp > all pass on em3 proto pfsync all match in log on enc0 inet from > 192.168.33.0/24 to 10.0.15.0/24 rdr-to > 10.0.3.0/24 bitmask > match out log on enc0 inet from 10.0.3.0/24 to 192.168.33.0/24 nat-to > 10.0.15.0/24 bitmask static-port > pass out on em0 inet proto udp from 88.77.88.60 to 88.77.88.52 port = > isakmp pass out on em0 inet proto udp from 88.77.88.60 to 88.77.88.52 port > = ipsec-nat-t pass out on em0 inet proto esp from 88.77.88.60 to > 88.77.88.52 pass out on em0 all flags S/SA pass out on enc0 inet proto > ipencap from 88.77.88.60 to 88.77.88.52 pass out on em1 all flags S/SA > pass out on em2 all flags S/SA pass out on em3 all flags S/SA pass in on > em0 inet proto udp from 88.77.88.52 to 88.77.88.60 port = isakmp pass in > on em0 inet proto udp from 88.77.88.52 to 88.77.88.60 port = ipsec-nat-t > pass in on em0 inet proto esp from 88.77.88.52 to 88.77.88.60 pass in on > enc0 inet proto ipencap from 88.77.88.52 to 88.77.88.60 pass out log on > em2 inet from 192.168.33.0/24 to 10.0.3.0/24 flags S/SA pass in log on em2 > inet from 10.0.3.0/24 to 192.168.33.0/24 flags S/SA rtable 0 pass in log > on enc0 inet from 192.168.33.0/24 to 10.0.3.0/24 flags S/SA rtable 2 pass > out log on enc0 inet from 10.0.3.0/24 to 192.168.33.0/24 flags S/SA # > > > The vpn configuration is simple and straight forward: > > # cat /etc/ipsec.conf > > ike esp from 10.0.15.0/24 (10.0.3.0/24) to 192.168.33.0/24 \ > local 88.77.88.60 peer 88.77.88.52 \ > main auth hmac-sha1 enc aes-256 group modp1024 \ > quick auth hmac-sha1 enc aes-256 group none \ > psk abctest > > > When I start the VPN everything looks ok: > > # isakmpd -K > # ipsecctl -f /etc/ipsec.conf > # ipsecctl -vs all > FLOWS: > flow esp in from 192.168.33.0/24 to 10.0.3.0/24 peer 88.77.88.52 srcid > 88.77.88.60/32 dstid 88.77.88.52/32 type use flow esp out from 10.0.3.0/24 > to 192.168.33.0/24 peer 88.77.88.52 srcid > 88.77.88.60/32 dstid 88.77.88.52/32 type require > > SAD: > esp tunnel from 88.77.88.60 to 88.77.88.52 spi 0x02234fca auth hmac-sha1 > enc aes-256 > sa: spi 0x02234fca auth hmac-sha1 enc aes > state mature replay 16 flags 4 > lifetime_cur: alloc 0 bytes 1848 add 1335969276 first 1335969277 > lifetime_hard: alloc 0 bytes 0 add 1200 first 0 > lifetime_soft: alloc 0 bytes 0 add 1080 first 0 > address_src: 88.77.88.60 > address_dst: 88.77.88.52 > identity_src: type prefix id 0: 88.77.88.60/32 > identity_dst: type prefix id 0: 88.77.88.52/32 > src_mask: 255.255.255.0 > dst_mask: 255.255.255.0 > protocol: proto 0 flags 0 > flow_type: type use direction out > src_flow: 10.0.3.0 > dst_flow: 192.168.33.0 > lifetime_lastuse: alloc 0 bytes 0 add 0 first 1335969313 esp > tunnel from 88.77.88.52 to 88.77.88.60 spi 0x274763cc auth hmac-sha1 enc > aes-256 > sa: spi 0x274763cc auth hmac-sha1 enc aes > state mature replay 16 flags 4 > lifetime_cur: alloc 0 bytes 256 add 1335969276 first 1335969399 > lifetime_hard: alloc 0 bytes 0 add 1200 first 0 > lifetime_soft: alloc 0 bytes 0 add 1080 first 0 > address_src: 88.77.88.52 > address_dst: 88.77.88.60 > identity_src: type prefix id 0: 88.77.88.52/32 > identity_dst: type prefix id 0: 88.77.88.60/32 > src_mask: 255.255.255.0 > dst_mask: 255.255.255.0 > protocol: proto 0 flags 0 > flow_type: type use direction in > src_flow: 192.168.33.0 > dst_flow: 10.0.3.0 > lifetime_lastuse: alloc 0 bytes 0 add 0 first 1335969415 # > > The VPN is successfully established and the correct (NATed) subnet is uses > for Phase-2 SA negotiation. > > From the logfile of the remote gateway (Check Point Firewall-1 Gaia > R75.40): > > Number: 435 > Date: 2May2012 > Time: 14:23:40 > Interface: daemon > Origin: gaia1 > Type: Log > Action: Key Install > Source: obsd (88.77.88.60) > Destination: gaia1 (88.77.88.52) > Community: cp_obsd > Information: IKE: Quick Mode completion IKE IDs: > subnet: 192.168.33.0 (mask= 255.255.255.0) and subnet: 10.0.15.0 (mask= > 255.255.255.0) > Source Key ID: 0x274763cc > Destination Key ID: 0x02234fca > Encryption Scheme: IKE > Encryption Methods: ESP: AES-256 + SHA1 > IKE Initiator Cookie: 4df1247f162befd5 > IKE Responder Cookie: 3f45060c7024c41e > IKE Phase2 Message ID: 19562ec3 > VPN Peer Gateway: obsd (88.77.88.60) > Subproduct: VPN > VPN Feature: IKE > > Although everything looks fine a ping the remote site from a workstation > on the local site (10.0.3/24) fails. > In a trace on pflog0 I can see, that the packets match the correct rule > but are not translated. > > # tcpdump -nei pflog0 > tcpdump: WARNING: snaplen raised from 116 to 160 > tcpdump: listening on pflog0, link-type PFLOG > > 16:38:18.382729 rule 14/(match) match out on enc0: 10.0.3.10 > > 192.168.33.51: icmp: echo request (DF) > 16:38:19.391099 rule 14/(match) match out on enc0: 10.0.3.10 > > 192.168.33.51: icmp: echo request (DF) > 16:38:20.399146 rule 14/(match) match out on enc0: 10.0.3.10 > > 192.168.33.51: icmp: echo request (DF) > 16:38:21.407203 rule 14/(match) match out on enc0: 10.0.3.10 > > 192.168.33.51: icmp: echo request (DF) > 16:38:22.415234 rule 14/(match) match out on enc0: 10.0.3.10 > > 192.168.33.51: icmp: echo request (DF) > 16:38:23.423168 rule 14/(match) match out on enc0: 10.0.3.10 > > 192.168.33.51: icmp: echo request (DF) > 16:38:24.431330 rule 14/(match) match out on enc0: 10.0.3.10 > > 192.168.33.51: icmp: echo request (DF) > ^C > 7 packets received by filter > 0 packets dropped by kernel > # > > # pfctl -sr -R 14 > match out log on enc0 inet from 10.0.3.0/24 to 192.168.33.0/24 nat-to > 10.0.15.0/24 bitmask static-port > # > > # pfctl -s state | grep 192.168.33 > all icmp 192.168.33.51:8 ((2) 192.168.33.51:8) <- 10.0.3.10:1981 ((2) > 10.0.3.10:1981) 0:0 > # > > Thus the packet is dropped by the remote gateway: > > Number: 441 > Date: 2May2012 > Time: 14:33:12 > Interface: eth0 > Origin: gaia1 > Type: Log > Action: Drop > Source: 10.0.3.10 > Destination: 192.168.33.51 > Protocol: icmp > Information: ICMP: Echo Request ICMP Type: 8 ICMP > Code: 0 encryption failure: According to the policy the packet should not > have been decrypted > Encryption Scheme: IKE > Encryption Methods: ESP: AES-256 + SHA1 > VPN Peer Gateway: obsd (88.77.88.60) > Subproduct: VPN > VPN Feature: VPN > Product: Security Gateway/Management > Log ID: 404821 > Product Family: Network > > > Any hints and help is appreciated. > > Cheers, > Bernd
