Hi,
I'm building up an OpenBSD router/firewall (migrating away from FreeBSD)
but have been blocked by a behavior of carp in combination with VLANs that
I didn't expect or experience before. I'm hoping somebody could enlighten
me a little bit about why carp floating IPs stop working when the carp
status is master for the physical interface.
Originally, there was a pair of FreeBSD systems (FW1 and FW2) where I had
no issues with carp managed IPs.
At the moment, one system is reinstalled with OpenBSD 5.9 (FW1), the other
remains with FreeBSD (FW2).
The network is setup in such a way that the default vlan (1) is untagged,
and this network is for all the network management. All other traffic goes
over tagged networks. The network switches we have simply work in this way
and so I can't make vlan 1 also a tagged interface to test the impact of
such a configuration.
As long as the OpenBSD system is not the master for the default / untagged
network associated to the physical network interface, the system will
accept packets for its CARP IPs.
When OpenBSD becomes master for the untagged network, it won't forward or
respond (ping) to packets addressed to its floating IP.
Configuration files for the physical interface (sk0) and a couple VLANs (I
run a dozen, but trimmed back to two for the purpose of this mail).
# cat /etc/sysctl.conf
net.inet.carp.allow=1
net.inet.carp.preempt=1
net.inet.ip.forwarding=1
# cat /etc/hostname.sk0
inet
10.1.0.2 255.255.255.0 NONE description "main link"
inet 10.0.0.2 255.255.255.0
# cat /etc/hostname.carp1
vhid 1 pass password carpdev sk0 advskew 150
inet 10.1.0.1 255.255.255.0
inet alias 10.0.0.1 255.255.255.0
# cat /etc/hostname.vlan10
inet 10.10.0.2 255.255.255.0 NONE vlan 10 vlandev sk0 description "Printer
network"
# cat /etc/hostname.carp10
vhid 1 pass
password carpdev vlan10 advskew 150
inet 10.10.0.1 255.255.255.0
# cat /etc/hostname.vlan50
inet 10.50.0.2 255.255.255.0 NONE vlan 50 vlandev sk0 description "Wireless
backbone"
# cat /etc/hostname.carp50
vhid 1 pass password carpdev vlan50 advskew 150
inet 10.50.0.1 255.255.255.0
The other system has a similar configuration with the exception that IPs
ending in .2 are .3 on FW2 and FW2 has advskew 100.
If I make FW1 (OpenBSD) the master for vlan10 and vlan50 (ifconfig carp10
advskew 1; ifconfig carp50 advskew) but not for sk0, then it will forward
packets between those two networks without problem and ping 10.10.0.1 works
fine.
The moment I make it the master for sk0 (ifconfig carp1 advskew 1), it no
longer forwards packets (between vlan10 and vlan50, vlan10 and the untagged
vlan) and it no longer responds to ping for any of the IPs associated to
the carp interfaces from external systems (ping 10.10.0.2 works, ping
10.10.0.1 doesn't work) although from the local box it works (ping
10.10.0.1 from FW1 works). Output from ifconfig shows FW1 is the master for
all interfaces.
Throughout, I am able to keep working with the box remotely as long as I
logged in via the local subnet IP (ie: from a workstation with IP
10.10.0.50, I can ssh to 10.10.0.2).
For testing ... while the FW1 (OpenBSD) is master for all interfaces, I
used tcpdump and could see the packets arriving at the system only if I
took the dump on sk0 or carp1. No packets show up on vlan10 or carp10 for
the box. On vlan10 - I can see all traffic addressed to 10.10.0.2 without
problem. On carp10 - I only see the "CARPv2-advertise" and arp
request/response packets.
To rule things out, I've kept the PF configuration as simple as possible
for testing (simply 1 line: "pass").
I always made sure that the corresponding CARP interfaces were in a backup
state on FW2 (freebsd) and via tcpdump that packets weren't ending up there
by some accident of the switches.
I've tried setting the subnet masks for the floating (carp) IP addresses to
be 255.255.255.255 - didn't change the behavior.
I set net.inet.carp.log=7 - nothing is noted in /var/log/messages beyond
the transitions (carp1: state transition: BACKUP -> MASTER; MASTER ->
BACKUP).
Since then, I'm out of ideas what to try and am turning to the mailing list
for help.
I'm rather new to OpenBSD, but I reviewed the FAQ and searched on google,
read man pages for carp, ifconfig, hostname.if, etc but didn't get any new
ideas.
Any ideas or suggestions what else I might look at?
Is this expected behavior or have I overlooked some configuration option?
Thanks in advance,
Andrew
