On 21 Oct 10:35, Carlos Lopez wrote: > Hi all, > > Before upgrade from OpenBSD 6.7 to OpenBSD 6.8, my pair firewalls was using > carp in IP balance mode without problems from several months. These firewalls > are installed in a RHEL 8.2 (fully patched) KVM host. > > After upgrading to OpenBSD 6.8, carp ip balance mode doesn’t works. I have > tested reconfiguring balance mode for ip-stealth and ip-unicast also and the > result is always the same: network packets are not processed by firewalls. > But if I configure CARP using “the simple configuration” and one node is > master and the other is backup all it is working without problems. > > All CARP interfaces are configured as this one: > > carpdev vio0 balancing ip pass 7254e4bc3024e35490e4b9942f919e9b > inet 172.22.55.30 0xffffffe0 172.22.55.31 > carpnodes 10:0,11:100 > description "Production Network" > > sysctl.conf file: > > net.inet.carp.preempt=1 > net.inet.carp.log=2 > net.inet.ip.forwarding=1 > net.inet.tcp.mssdflt=1440 > net.inet.ip.redirect=0 > net.inet.ip.mtudisc=0 > net.inet.tcp.rfc3390=1 > net.inet.ip.arptimeout=60 > kern.bufcachepercent=70 > net.inet.icmp.tstamprepl=0 > net.inet.udp.sendspace=262144 > net.inet.udp.recvspace=262144 > > > OpenBSD kvm guest config: > > <domain type='kvm' id='12'> > <name>obsdfw01</name> > <description>OpenBSD Security Gateway Cluster</description> > <memory unit='KiB'>786432</memory> > <currentMemory unit='KiB'>786432</currentMemory> > <vcpu placement='static'>1</vcpu> > <resource> > <partition>/machine</partition> > </resource> > <os> > <type arch='x86_64' machine='pc-q35-rhel7.6.0'>hvm</type> > <boot dev='hd'/> > </os> > <features> > <acpi/> > <apic/> > </features> > <cpu mode='custom' match='exact' check='full'> > <model fallback='forbid'>Broadwell</model> > <feature policy='require' name='vme'/> > <feature policy='require' name='f16c'/> > <feature policy='require' name='rdrand'/> > <feature policy='require' name='hypervisor'/> > <feature policy='require' name='arat'/> > <feature policy='require' name='xsaveopt'/> > <feature policy='require' name='abm'/> > </cpu> > <clock offset='utc'> > <timer name='rtc' tickpolicy='catchup'/> > <timer name='pit' tickpolicy='delay'/> > <timer name='hpet' present='yes'/> > </clock> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>destroy</on_crash> > <pm> > <suspend-to-mem enabled='no'/> > <suspend-to-disk enabled='no'/> > </pm> > <devices> > <emulator>/usr/libexec/qemu-kvm</emulator> > <disk type='file' device='disk'> > <driver name='qemu' type='qcow2' cache='none'/> > <source file='/data/vmvol0/vmachines/obsdfw01vol.img'/> > <backingStore/> > <target dev='vda' bus='virtio'/> > <alias name='virtio-disk0'/> > <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' > function='0x0'/> > </disk> > <controller type='usb' index='0' model='none'> > <alias name='usb'/> > </controller> > <controller type='virtio-serial' index='0'> > <alias name='virtio-serial0'/> > <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' > function='0x0'/> > </controller> > <controller type='pci' index='0' model='pcie-root'> > <alias name='pcie.0'/> > </controller> > <controller type='pci' index='1' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='1' port='0x10'/> > <alias name='pci.1'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x0' multifunction='on'/> > </controller> > <controller type='pci' index='2' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='2' port='0x11'/> > <alias name='pci.2'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x1'/> > </controller> > <controller type='pci' index='3' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='3' port='0x12'/> > <alias name='pci.3'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x2'/> > </controller> > <controller type='pci' index='4' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='4' port='0x13'/> > <alias name='pci.4'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x3'/> > </controller> > <controller type='pci' index='5' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='5' port='0x14'/> > <alias name='pci.5'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x4'/> > </controller> > <controller type='pci' index='6' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='6' port='0x15'/> > <alias name='pci.6'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x5'/> > </controller> > <controller type='pci' index='7' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='7' port='0x16'/> > <alias name='pci.7'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x6'/> > </controller> > <controller type='pci' index='8' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='8' port='0x17'/> > <alias name='pci.8'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x7'/> > </controller> > <controller type='pci' index='9' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='9' port='0x18'/> > <alias name='pci.9'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x0' multifunction='on'/> > </controller> > <controller type='pci' index='10' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='10' port='0x19'/> > <alias name='pci.10'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x1'/> > </controller> > <controller type='pci' index='11' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='11' port='0x1a'/> > <alias name='pci.11'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x2'/> > </controller> > <controller type='pci' index='12' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='12' port='0x1b'/> > <alias name='pci.12'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x3'/> > </controller> > <controller type='pci' index='13' model='pcie-root-port'> > <model name='pcie-root-port'/> > <target chassis='13' port='0x1c'/> > <alias name='pci.13'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x4'/> > </controller> > <controller type='sata' index='0'> > <alias name='ide'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' > function='0x2'/> > </controller> > <interface type='bridge'> > <mac address='00:50:56:6f:64:aa'/> > <source bridge='prodif'/> > <target dev='obsdprod0'/> > <model type='virtio'/> > <alias name='net0'/> > <address type='pci' domain='0x0000' bus='0x01' slot='0x00' > function='0x0'/> > </interface> > <interface type='bridge'> > <mac address='00:50:56:ab:44:05'/> > <source bridge='pubif'/> > <target dev='obsdpub0'/> > <model type='virtio'/> > <alias name='net1'/> > <address type='pci' domain='0x0000' bus='0x02' slot='0x00' > function='0x0'/> > </interface> > <interface type='bridge'> > <mac address='00:50:56:3c:e5:61'/> > <source bridge='mgmtif'/> > <target dev='obsdmgmt0'/> > <model type='virtio'/> > <alias name='net2'/> > <address type='pci' domain='0x0000' bus='0x03' slot='0x00' > function='0x0'/> > </interface> > <interface type='bridge'> > <mac address='00:50:56:4c:d6:34'/> > <source bridge='dmzif'/> > <target dev='obsddmz0'/> > <model type='virtio'/> > <alias name='net3'/> > <address type='pci' domain='0x0000' bus='0x04' slot='0x00' > function='0x0'/> > </interface> > <interface type='bridge'> > <mac address='00:50:56:73:a4:ff'/> > <source bridge='vpnif'/> > <target dev='obsdvpn0'/> > <model type='virtio'/> > <alias name='net4'/> > <address type='pci' domain='0x0000' bus='0x05' slot='0x00' > function='0x0'/> > </interface> > <interface type='bridge'> > <mac address='00:50:56:29:0d:b5'/> > <source bridge='encif'/> > <target dev='obsdenc0'/> > <model type='virtio'/> > <alias name='net5'/> > <address type='pci' domain='0x0000' bus='0x06' slot='0x00' > function='0x0'/> > </interface> > <interface type='bridge'> > <mac address='00:50:56:d1:ba:cc'/> > <source bridge='idpmif'/> > <target dev='obsdidp0'/> > <model type='virtio'/> > <alias name='net6'/> > <address type='pci' domain='0x0000' bus='0x07' slot='0x00' > function='0x0'/> > </interface> > <interface type='bridge'> > <mac address='00:50:56:49:21:d0'/> > <source bridge='syncif'/> > <target dev='obsdsync0'/> > <model type='virtio'/> > <alias name='net7'/> > <address type='pci' domain='0x0000' bus='0x08' slot='0x00' > function='0x0'/> > </interface> > <interface type='bridge'> > <mac address='00:50:56:a6:72:ff'/> > <source bridge='winif'/> > <target dev='obsdwin0'/> > <model type='virtio'/> > <alias name='net8'/> > <address type='pci' domain='0x0000' bus='0x09' slot='0x00' > function='0x0'/> > </interface> > <serial type='pty'> > <source path='/dev/pts/4'/> > <target type='isa-serial' port='0'> > <model name='isa-serial'/> > </target> > <alias name='serial0'/> > </serial> > <console type='pty' tty='/dev/pts/4'> > <source path='/dev/pts/4'/> > <target type='serial' port='0'/> > <alias name='serial0'/> > </console> > <channel type='spicevmc'> > <target type='virtio' name='com.redhat.spice.0' state='disconnected'/> > <alias name='channel0'/> > <address type='virtio-serial' controller='0' bus='0' port='1'/> > </channel> > <input type='mouse' bus='ps2'> > <alias name='input0'/> > </input> > <input type='keyboard' bus='ps2'> > <alias name='input1'/> > </input> > <graphics type='vnc' port='5903' autoport='yes' listen='127.0.0.1' > keymap='es'> > <listen type='address' address='127.0.0.1'/> > </graphics> > <video> > <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' > primary='yes'/> > <alias name='video0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x01' > function='0x0'/> > </video> > <memballoon model='virtio'> > <alias name='balloon0'/> > <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' > function='0x0'/> > </memballoon> > <rng model='virtio'> > <backend model='random'>/dev/urandom</backend> > <alias name='rng0'/> > <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' > function='0x0'/> > </rng> > </devices> > <seclabel type='dynamic' model='selinux' relabel='yes'> > <label>system_u:system_r:svirt_t:s0:c82,c777</label> > <imagelabel>system_u:object_r:svirt_image_t:s0:c82,c777</imagelabel> > </seclabel> > <seclabel type='dynamic' model='dac' relabel='yes'> > <label>+107:+107</label> > <imagelabel>+107:+107</imagelabel> > </seclabel> > </domain> > > Dmesg output: > > OpenBSD 6.8 (GENERIC) #97: Sun Oct 4 18:00:46 MDT 2020 > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC > real mem = 788389888 (751MB) > avail mem = 749596672 (714MB) > random: good seed from bootblocks > mpath0 at root > scsibus0 at mpath0: 256 targets > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5af0 (9 entries) > bios0: vendor SeaBIOS version "1.11.1-4.module+el8.1.0+4066+0f1aadab" date > 04/01/2014 > bios0: Red Hat KVM > acpi0 at bios0: ACPI 3.0 > acpi0: sleep states S5 > acpi0: tables DSDT FACP APIC MCFG > acpi0: wakeup devices > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: Intel Core Processor (Broadwell), 1900.29 MHz, 06-3d-02 > cpu0: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,RDTSCP,LONG,LAHF,ABM,3DNOWP,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,ARAT,XSAVEOPT,MELTDOWN > cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB > 64b/line 16-way L2 cache > cpu0: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped > cpu0: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped > cpu0: smt 0, core 0, package 0 > mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges > cpu0: apic clock running at 1000MHz > ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins > acpimcfg0 at acpi0 > acpimcfg0: addr 0xb0000000, bus 0-255 > acpiprt0 at acpi0: bus 0 (PCI0) > "ACPI0006" at acpi0 not configured > acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001 > acpicmos0 at acpi0 > "PNP0A06" at acpi0 not configured > "PNP0A06" at acpi0 not configured > "QEMU0002" at acpi0 not configured > "ACPI0010" at acpi0 not configured > acpicpu0 at acpi0: C1(@1 halt!) > cpu0: using Broadwell MDS workaround > pvbus0 at mainbus0: KVM > pvclock0 at pvbus0 > pci0 at mainbus0 bus 0 > pchb0 at pci0 dev 0 function 0 "Intel 82G33 Host" rev 0x00 > vga1 at pci0 dev 1 function 0 "Red Hat QXL Video" rev 0x04 > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) > ppb0 at pci0 dev 2 function 0 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 22 > pci1 at ppb0 bus 1 > virtio0 at pci1 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio0 at virtio0: address 00:50:56:6f:64:aa > virtio0: msix shared > ppb1 at pci0 dev 2 function 1 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 22 > pci2 at ppb1 bus 2 > virtio1 at pci2 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio1 at virtio1: address 00:50:56:ab:44:05 > virtio1: msix shared > ppb2 at pci0 dev 2 function 2 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 22 > pci3 at ppb2 bus 3 > virtio2 at pci3 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio2 at virtio2: address 00:50:56:3c:e5:61 > virtio2: msix shared > ppb3 at pci0 dev 2 function 3 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 22 > pci4 at ppb3 bus 4 > virtio3 at pci4 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio3 at virtio3: address 00:50:56:4c:d6:34 > virtio3: msix shared > ppb4 at pci0 dev 2 function 4 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 22 > pci5 at ppb4 bus 5 > virtio4 at pci5 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio4 at virtio4: address 00:50:56:73:a4:ff > virtio4: msix shared > ppb5 at pci0 dev 2 function 5 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 22 > pci6 at ppb5 bus 6 > virtio5 at pci6 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio5 at virtio5: address 00:50:56:29:0d:b5 > virtio5: msix shared > ppb6 at pci0 dev 2 function 6 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 22 > pci7 at ppb6 bus 7 > virtio6 at pci7 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio6 at virtio6: address 00:50:56:d1:ba:cc > virtio6: msix shared > ppb7 at pci0 dev 2 function 7 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 22 > pci8 at ppb7 bus 8 > virtio7 at pci8 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio7 at virtio7: address 00:50:56:49:21:d0 > virtio7: msix shared > ppb8 at pci0 dev 3 function 0 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci9 at ppb8 bus 9 > virtio8 at pci9 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio8 at virtio8: address 00:50:56:a6:72:ff > virtio8: msix shared > ppb9 at pci0 dev 3 function 1 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci10 at ppb9 bus 10 > virtio9 at pci10 dev 0 function 0 "Qumranet Virtio 1.x Console" rev 0x01 > virtio9: no matching child driver; not configured > ppb10 at pci0 dev 3 function 2 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci11 at ppb10 bus 11 > virtio10 at pci11 dev 0 function 0 "Qumranet Virtio 1.x Storage" rev 0x01 > vioblk0 at virtio10 > scsibus1 at vioblk0: 1 targets > sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > > sd0: 16384MB, 512 bytes/sector, 33554432 sectors > virtio10: msix shared > ppb11 at pci0 dev 3 function 3 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci12 at ppb11 bus 12 > virtio11 at pci12 dev 0 function 0 vendor "Qumranet", unknown product 0x1045 > rev 0x01 > viomb0 at virtio11 > virtio11: apic 0 int 23 > ppb12 at pci0 dev 3 function 4 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci13 at ppb12 bus 13 > virtio12 at pci13 dev 0 function 0 "Qumranet Virtio 1.x RNG" rev 0x01 > viornd0 at virtio12 > virtio12: apic 0 int 23 > virtio7: msix shared > ppb8 at pci0 dev 3 function 0 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci9 at ppb8 bus 9 > virtio8 at pci9 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio8 at virtio8: address 00:50:56:a6:72:ff > virtio8: msix shared > ppb9 at pci0 dev 3 function 1 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci10 at ppb9 bus 10 > virtio9 at pci10 dev 0 function 0 "Qumranet Virtio 1.x Console" rev 0x01 > virtio9: no matching child driver; not configured > ppb10 at pci0 dev 3 function 2 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci11 at ppb10 bus 11 > virtio10 at pci11 dev 0 function 0 "Qumranet Virtio 1.x Storage" rev 0x01 > vioblk0 at virtio10 > scsibus1 at vioblk0: 1 targets > sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > > sd0: 16384MB, 512 bytes/sector, 33554432 sectors > virtio10: msix shared > ppb11 at pci0 dev 3 function 3 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci12 at ppb11 bus 12 > virtio11 at pci12 dev 0 function 0 vendor "Qumranet", unknown product 0x1045 > rev 0x01 > viomb0 at virtio11 > virtio11: apic 0 int 23 > ppb12 at pci0 dev 3 function 4 vendor "Red Hat", unknown product 0x000c rev > 0x00: apic 0 int 23 > pci13 at ppb12 bus 13 > virtio12 at pci13 dev 0 function 0 "Qumranet Virtio 1.x RNG" rev 0x01 > viornd0 at virtio12 > virtio12: apic 0 int 23 > pcib0 at pci0 dev 31 function 0 "Intel 82801IB LPC" rev 0x02 > ahci0 at pci0 dev 31 function 2 "Intel 82801I AHCI" rev 0x02: msi, AHCI 1.0 > scsibus2 at ahci0: 32 targets > ichiic0 at pci0 dev 31 function 3 "Intel 82801I SMBus" rev 0x02: apic 0 int 16 > iic0 at ichiic0 > isa0 at pcib0 > isadma0 at isa0 > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > pckbc0 at isa0 port 0x60/5 irq 1 irq 12 > pckbd0 at pckbc0 (kbd slot) > wskbd0 at pckbd0: console keyboard, using wsdisplay0 > pms0 at pckbc0 (aux slot) > wsmouse0 at pms0 mux 0 > pcppi0 at isa0 port 0x61 > spkr0 at pcppi0 > vscsi0 at root > scsibus3 at vscsi0: 256 targets > softraid0 at root > scsibus4 at softraid0: 256 targets > root on sd0a (dcd0d9bbce80825c.a) swap on sd0b dump on sd0b > carp0: state transition: BACKUP -> MASTER > carp1: state transition: BACKUP -> MASTER > carp2: state transition: BACKUP -> MASTER > carp3: state transition: BACKUP -> MASTER > carp4: state transition: BACKUP -> MASTER > carp5: state transition: BACKUP -> MASTER > carp6: state transition: BACKUP -> MASTER > carp7: state transition: BACKUP -> MASTER > pfsync: failed to receive bulk update > > Regards, > C. L. Martinez >
I experienced exactly the same after upgrade tonight. Doing: "ifconfig -g carp carpdemote" to promote on node to master for both ids immediately makes the carp address respond again - but of course no load balancing then. -- wq: ~uw
