On 2020-07-29 21:47, Andrew Hewus Fresh wrote:
> I am helping some folks with some OpenBSD stuff, including at some
> point being more proactive about updating, but they currently have an
> OpenBSD 5.9 machine that has been routing traffic happily for a while.
> Unfortunately, they can't currently update because the proxy arp
> configuration that they are using on 5.9 doesn't work on anything newer.
>
>
> Here is the excellent description from someone there, and I was able to
> reproduce the issue on my laptop, so I don't believe a dmesg will be
> particularly helpful, but here's one for my laptop that's not terribly
> old.
>
> https://dmesgd.nycbug.org/index.cgi?do=view&id=5279
>
> As I haven't ever actually used proxy arp on OpenBSD, so don't actually
> know whether this was a glitch that was fixed or added.
>
> I do see that 60.html says the routing table is now based on ART, so
> that seems a possible culprit.
>
> ---
>
>
> We've found that a proxy-arp configuration we've relied on (which works
> perfectly on OpenBSD 5.9 and earlier) no longer works on OpenBSD 6.0 thru 6.7.
>
> On OpenBSD 5.9, we can add an "arp -s" entry for a public IP on our
> public-facing gateway router's external interface, followed by a host-route
> for that same public IP (pointing to an internal subnet/interface). This
> allows us to route public IPs to clients through our internal private subnets,
> so a client can have a public IP assigned on their router, despite actually
> being several hops deep inside our private network.
>
> With this proxy-arp configuration, from both the client router's perspective
> and from an external perspective, it looks as if the client router is in the
> public IP subnet.
>
> Unfortunately this method doesn't work in OpenBSD 6.0 and later.
>
>
> Here's how it looks when you configure this on OpenBSD 5.9
> (working as expected with no errors):
>
> root@openbsd-59-pub-gw-rtr~ # arp -s 1.2.3.4
> [PUB-GW_em0_EXT-IF_MAC-ADDR] pub
> root@openbsd-59-pub-gw-rtr~ # route add 1.2.3.4/32 10.0.2.47
> add host 1.2.3.4/32: gateway 10.0.2.47
> root@openbsd-59-pub-gw-rtr~ # netstat -rn -finet|grep 1.2.3.4
> 1.2.3.4 10.0.2.47 UGHS 0 0 - 8 em1
> 1.2.3.4/32 [PUB-GW_em0_EXT-IF_MAC-ADDR] ULS2 0 0 -
> 8 em0
> ##### ^ Success: Routing table shows both the host-route and the "arp
> -s" entry.
>
>
> That same setup fails on OpenBSD 6.0 and later
> (tested on OpenBSD 6.0, 6.2, 6.6, and 6.7).
>
>
> If you add an arp -s entry before adding a host route for the same IP, the
> "route add" fails with "File exists" error:
>
> root@openbsd-67-pub-gw-rtr~ # arp -s 1.2.3.4
> [PUB-GW_em0_EXT-IF_MAC-ADDR] pub
> root@openbsd-67-pub-gw-rtr~ # route add 1.2.3.4/32 10.0.2.47
> !-> add host 1.2.3.4/32: gateway 10.0.2.47: File exists
> root@openbsd-67-pub-gw-rtr~ # netstat -rn -finet|grep 1.2.3.4
> 1.2.3.4/32 [PUB-GW_em0_EXT-IF_MAC-ADDR] ULS2 0 0 -
> 8 em0
> ##### ^ Failure: Routing table only shows the "arp -s" entry (failed to
> add host-route)
>
>
> If you add the route before the arp -s entry, "arp -s" fails with error
> "set: proxy entry exists for non 802 device":
>
> root@openbsd-67-pub-gw-rtr~ # route add 1.2.3.4/32 10.0.2.47
> add host 1.2.3.4/32: gateway 10.0.2.47
> root@openbsd-67-pub-gw-rtr~ # arp -s 1.2.3.4
> [PUB-GW_em0_EXT-IF_MAC-ADDR] pub
> !-> set: proxy entry exists for non 802 device
> root@openbsd-67-pub-gw-rtr~ # netstat -rn -finet|grep 1.2.3.4
> 1.2.3.4 10.0.2.47 UGHS 0 0 - 8 em1
> ##### ^ Failure: Routing table only shows the host-route (failed to add
> arp -s entry)
>
>
> For most of our clients who request a dedicated public IP, we're able to
> assign their router a private IP, which we then bi-NAT to a dedicated public
> IP on our gateway router. Unfortunately the bi-NAT method doesn't work for
> certain routers' built-in VPNs, so we're looking for a way to re-create the
> functionality we had with proxy-arp on OpenBSD v5.9. I imagine there's some
> workaround or alternate method to achieve the same thing on OpenBSD 6.x, but I
> haven't found it yet.
You can avoid this problem by first setting the ARP entry, and then
adding the host route with a lower (better) priority. On my system
(which runs OpenBSD-CURRENT), ARP entries have priority 8, so your
host route should have priority below 8. Of course, you need to do
this before interfaces come up, as otherwise packets might be routed
to the wrong destination.
If you do this, you won’t be able to use arp(8) to delete the proxy
ARP entry. This is due to a limitation in arp(8), which is fixed
by the following patch. This patch has been tested on my -CURRENT
system and also allows creating an entry even if a route entry (with
a priority other than 8) already exists.
I would appreciate review of the patch; in particular, I don’t have
any interfaces that would trigger the “goto delete” clause.
Sincerely,
Demi
Index: usr.sbin/arp/arp.c
===================================================================
RCS file: /cvs/src/usr.sbin/arp/arp.c,v
retrieving revision 1.88
diff -u -p -u -p -r1.88 arp.c
--- usr.sbin/arp/arp.c 16 Sep 2019 20:49:28 -0000 1.88
+++ usr.sbin/arp/arp.c 8 Aug 2020 00:40:55 -0000
@@ -343,10 +343,8 @@ tryagain:
printf("set: can only proxy for %s\n", host);
return (1);
}
- if (sin_m.sin_other & SIN_PROXY) {
- printf("set: proxy entry exists for non 802 device\n");
- return (1);
- }
+ if (sin_m.sin_other & SIN_PROXY)
+ goto overwrite;
sin_m.sin_other = SIN_PROXY;
export_only = 1;
goto tryagain;
@@ -415,19 +413,11 @@ tryagain:
return (1);
}
if (sin->sin_addr.s_addr == sin_m.sin_addr.s_addr) {
- if (sdl->sdl_family == AF_LINK && rtm->rtm_flags & RTF_LLINFO) {
+ if (sdl->sdl_family == AF_LINK) {
if (rtm->rtm_flags & RTF_LOCAL)
return (0);
- if (!(rtm->rtm_flags & RTF_GATEWAY))
- switch (sdl->sdl_type) {
- case IFT_ETHER:
- case IFT_FDDI:
- case IFT_ISO88023:
- case IFT_ISO88024:
- case IFT_ISO88025:
- case IFT_CARP:
- goto delete;
- }
+ if (sin_m.sin_other & SIN_PROXY)
+ goto delete;
}
}
