Re: Carp and VLANs

[Andrew Seguin]

Thank you,

This (having unique VHID) was the solution.

I had considered originally that since each carp device is on its own VLAN,
that would represent a unique broadcast domain and it wouldn't be violating
anything - but without your suggestion I'm not sure I would have gone back
to review that decision.

I'm still a bit curious how it came to that. I did snoop if carp
announcements were leaking from a tagged vlan onto the default network, but
didn't see any sign of that. So maybe it was because the VLANs were riding
on top of the same physical interface... but a lot less important now.

Regards,
Andrew




On Tue, Aug 23, 2016 at 8:34 PM, John Jasen 
wrote:

> All your carp devices have the same VHID. As two share the same network,
> that could cause problems.
>
>
>
>
> On 08/23/2016 01:40 PM, Andrew Seguin wrote:
> > Hi,
> >
> > I'm building up an OpenBSD router/firewall (migrating away from FreeBSD)
> > but have been blocked by a behavior of carp in combination with VLANs
> that
> > I didn't expect or experience before. I'm hoping somebody could enlighten
> > me a little bit about why carp floating IPs stop working when the carp
> > status is master for the physical interface.
> >
> >
> > Originally, there was a pair of FreeBSD systems (FW1 and FW2) where I had
> > no issues with carp managed IPs.
> >
> > At the moment, one system is reinstalled with OpenBSD 5.9 (FW1), the
> other
> > remains with FreeBSD (FW2).
> >
> > The network is setup in such a way that the default vlan (1) is untagged,
> > and this network is for all the network management. All other traffic
> goes
> > over tagged networks. The network switches we have simply work in this
> way
> > and so I can't make vlan 1 also a tagged interface to test the impact of
> > such a configuration.
> >
> > As long as the OpenBSD system is not the master for the default /
> untagged
> > network associated to the physical network interface, the system will
> > accept packets for its CARP IPs.
> >
> > When OpenBSD becomes master for the untagged network, it won't forward or
> > respond (ping) to packets addressed to its floating IP.
> >
> > Configuration files for the physical interface (sk0) and a couple VLANs
> (I
> > run a dozen, but trimmed back to two for the purpose of this mail).
> >
> > # cat /etc/sysctl.conf
> > net.inet.carp.allow=1
> > net.inet.carp.preempt=1
> > net.inet.ip.forwarding=1
> >
> > # cat /etc/hostname.sk0
> >   inet
> > 10.1.0.2 255.255.255.0 NONE description "main link"
> > inet 10.0.0.2 255.255.255.0
> >
> > # cat /etc/hostname.carp1
> > vhid 1 pass password carpdev sk0 advskew 150
> > inet 10.1.0.1 255.255.255.0
> > inet alias 10.0.0.1 255.255.255.0
> >
> > # cat /etc/hostname.vlan10
> > inet 10.10.0.2 255.255.255.0 NONE vlan 10 vlandev sk0 description
> "Printer
> > network"
> >
> > # cat /etc/hostname.carp10
> > vhid 1
> pass
> > password carpdev vlan10 advskew 150
> > inet 10.10.0.1 255.255.255.0
> >
> > # cat /etc/hostname.vlan50
> > inet 10.50.0.2 255.255.255.0 NONE vlan 50 vlandev sk0 description
> "Wireless
> > backbone"
> >
> > # cat /etc/hostname.carp50
> > vhid 1 pass password carpdev vlan50 advskew 150
> > inet 10.50.0.1 255.255.255.0
> >
> >
> > The other system has a similar configuration with the exception that IPs
> > ending in .2 are .3 on FW2 and FW2 has advskew 100.
> >
> >
> > If I make FW1 (OpenBSD) the master for vlan10 and vlan50 (ifconfig carp10
> > advskew 1; ifconfig carp50 advskew) but not for sk0, then it will forward
> > packets between those two networks without problem and ping 10.10.0.1
> works
> > fine.
> >
> > The moment I make it the master for sk0 (ifconfig carp1 advskew 1), it no
> > longer forwards packets (between vlan10 and vlan50, vlan10 and the
> untagged
> > vlan) and it no longer responds to ping for any of the IPs associated to
> > the carp interfaces from external systems (ping 10.10.0.2 works, ping
> > 10.10.0.1 doesn't work) although from the local box it works (ping
> > 10.10.0.1 from FW1 works). Output from ifconfig shows FW1 is the master
> for
> > all interfaces.
> >
> > Throughout, I am able to keep working with the box remotely as long as I
> > logged in via the local subnet IP (ie: from a workstation with IP
> > 10.10.0.50, I can ssh to 10.10.0.2).
> >
> > For testing ... while the FW1 (OpenBSD) is master for all interfaces, I
> > used tcpdump and could see the packets arriving at the system only if I
> > took the dump on sk0 or carp1. No packets show up on vlan10 or carp10 for
> > the box. On vlan10 - I can see all traffic addressed to 10.10.0.2 without
> > problem. On carp10 - I only see the "CARPv2-advertise" and arp
> > request/response packets.
> >
> > To rule things out, I've kept the PF configuration as simple as possible
> > for testing (simply 1 line: "pass").
> >
> > I always made sure that the 

Re: Carp and VLANs

[John Jasen]

All your carp devices have the same VHID. As two share the same network,
that could cause problems.




On 08/23/2016 01:40 PM, Andrew Seguin wrote:
> Hi,
>
> I'm building up an OpenBSD router/firewall (migrating away from FreeBSD)
> but have been blocked by a behavior of carp in combination with VLANs that
> I didn't expect or experience before. I'm hoping somebody could enlighten
> me a little bit about why carp floating IPs stop working when the carp
> status is master for the physical interface.
>
>
> Originally, there was a pair of FreeBSD systems (FW1 and FW2) where I had
> no issues with carp managed IPs.
>
> At the moment, one system is reinstalled with OpenBSD 5.9 (FW1), the other
> remains with FreeBSD (FW2).
>
> The network is setup in such a way that the default vlan (1) is untagged,
> and this network is for all the network management. All other traffic goes
> over tagged networks. The network switches we have simply work in this way
> and so I can't make vlan 1 also a tagged interface to test the impact of
> such a configuration.
>
> As long as the OpenBSD system is not the master for the default / untagged
> network associated to the physical network interface, the system will
> accept packets for its CARP IPs.
>
> When OpenBSD becomes master for the untagged network, it won't forward or
> respond (ping) to packets addressed to its floating IP.
>
> Configuration files for the physical interface (sk0) and a couple VLANs (I
> run a dozen, but trimmed back to two for the purpose of this mail).
>
> # cat /etc/sysctl.conf
> net.inet.carp.allow=1
> net.inet.carp.preempt=1
> net.inet.ip.forwarding=1
>
> # cat /etc/hostname.sk0
>   inet
> 10.1.0.2 255.255.255.0 NONE description "main link"
> inet 10.0.0.2 255.255.255.0
>
> # cat /etc/hostname.carp1
> vhid 1 pass password carpdev sk0 advskew 150
> inet 10.1.0.1 255.255.255.0
> inet alias 10.0.0.1 255.255.255.0
>
> # cat /etc/hostname.vlan10
> inet 10.10.0.2 255.255.255.0 NONE vlan 10 vlandev sk0 description "Printer
> network"
>
> # cat /etc/hostname.carp10
> vhid 1 pass
> password carpdev vlan10 advskew 150
> inet 10.10.0.1 255.255.255.0
>
> # cat /etc/hostname.vlan50
> inet 10.50.0.2 255.255.255.0 NONE vlan 50 vlandev sk0 description "Wireless
> backbone"
>
> # cat /etc/hostname.carp50
> vhid 1 pass password carpdev vlan50 advskew 150
> inet 10.50.0.1 255.255.255.0
>
>
> The other system has a similar configuration with the exception that IPs
> ending in .2 are .3 on FW2 and FW2 has advskew 100.
>
>
> If I make FW1 (OpenBSD) the master for vlan10 and vlan50 (ifconfig carp10
> advskew 1; ifconfig carp50 advskew) but not for sk0, then it will forward
> packets between those two networks without problem and ping 10.10.0.1 works
> fine.
>
> The moment I make it the master for sk0 (ifconfig carp1 advskew 1), it no
> longer forwards packets (between vlan10 and vlan50, vlan10 and the untagged
> vlan) and it no longer responds to ping for any of the IPs associated to
> the carp interfaces from external systems (ping 10.10.0.2 works, ping
> 10.10.0.1 doesn't work) although from the local box it works (ping
> 10.10.0.1 from FW1 works). Output from ifconfig shows FW1 is the master for
> all interfaces.
>
> Throughout, I am able to keep working with the box remotely as long as I
> logged in via the local subnet IP (ie: from a workstation with IP
> 10.10.0.50, I can ssh to 10.10.0.2).
>
> For testing ... while the FW1 (OpenBSD) is master for all interfaces, I
> used tcpdump and could see the packets arriving at the system only if I
> took the dump on sk0 or carp1. No packets show up on vlan10 or carp10 for
> the box. On vlan10 - I can see all traffic addressed to 10.10.0.2 without
> problem. On carp10 - I only see the "CARPv2-advertise" and arp
> request/response packets.
>
> To rule things out, I've kept the PF configuration as simple as possible
> for testing (simply 1 line: "pass").
>
> I always made sure that the corresponding CARP interfaces were in a backup
> state on FW2 (freebsd) and via tcpdump that packets weren't ending up there
> by some accident of the switches.
>
> I've tried setting the subnet masks for the floating (carp) IP addresses to
> be 255.255.255.255 - didn't change the behavior.
>
> I set net.inet.carp.log=7 - nothing is noted in /var/log/messages beyond
> the transitions (carp1: state transition: BACKUP -> MASTER; MASTER ->
> BACKUP).
>
> Since then, I'm out of ideas what to try and am turning to the mailing list
> for help.
>
> I'm rather new to OpenBSD, but I reviewed the FAQ and searched on google,
> read man pages for carp, ifconfig, hostname.if, etc but didn't get any new
> ideas.
>
> Any ideas or suggestions what else I might look at?
>
> Is this expected behavior or have I overlooked some configuration option?
>
> Thanks in advance,
> Andrew