Hi misc@! After deploying a new OpenBSD 5.3 firewall today I ran into a strange problem. The first rule in my ruleset is one NAT-ing ICMP packets from my host to Google's DNS IP (8.8.8.8):
> fw1a-spt # pfctl -sr -R0 > pass out log quick inet proto icmp from 192.168.5.96 to 8.8.8.8 nat-to 195.182.23.4 195.182.23.4 is my public IP address. The problem is that only one in every ~20 packets gets NAT-ed. Other ones get passed as-is. A tcpdump is available here: http://hpaste.org/90099 I managed to increase the number of NAT-ed packets to about one-in-five adding the following line to my PF: > set timeout { icmp.first 0, icmp.error 0 } I would like not to paste my whole pf.conf file, since it's a bit large (~1k lines) and has been recently migrated from a FreeBSD machine. There are no arcane things inside (like limits or any other 'set' directives), just other pass and block rules. The trunk0 interface configuration: > trunkproto lacp > trunkport bge0 > trunkport bge1 > up Disabling one of the bge interfaces didn't help either, which makes me think LACP is unrelated to this. A carp interface with the 195.182.23.4 IP address sits on top of the trunk interface: > vhid 1 advskew 10 carpdev trunk0 pass *snip!* > inet 195.182.23.4 255.255.255.0 NONE > other aliases here... If any other information is needed please ask and I'll happily provide it. Does anybody have an idea what might be causing this? I'll try testing this on CURRENT later, since I'm running out of ideas here... And a mandatory dmesg follows: OpenBSD 5.3 (GENERIC.MP) #62: Tue Mar 12 18:21:20 MDT 2013 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 2145452032 (2046MB) avail mem = 2065903616 (1970MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.31 @ 0xdc010 (57 entries) bios0: vendor HP version "O08" date 08/13/2007 bios0: HP ProLiant DL140 G3 acpi0 at bios0: rev 0 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP SPMI APIC MCFG BOOT SPCR SSDT acpi0: wakeup devices BPD0(S5) BMF3(S5) P0P4(S5) P0P6(S5) PEX0(S5) PEX1(S5) PEX2(S5) PEX3(S5) USB1(S5) USB2(S5) USB3(S5) EUSB(S5) PCIB(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Xeon(R) CPU 5130 @ 2.00GHz, 1995.29 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,TM2,SS SE3,CX16,xTPR,PDCM,DCA,NXE,LONG,LAHF,PERF cpu0: 4MB 64b/line 16-way L2 cache cpu0: apic clock running at 332MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Xeon(R) CPU 5130 @ 2.00GHz, 1995.00 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,TM2,SS SE3,CX16,xTPR,PDCM,DCA,NXE,LONG,LAHF,PERF cpu1: 4MB 64b/line 16-way L2 cache ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins ioapic1 at mainbus0: apid 3 pa 0xfec80000, version 20, 24 pins acpimcfg0 at acpi0 addr 0xe0000000, bus 0-23 acpiprt0 at acpi0: bus 1 (P0P2) acpiprt1 at acpi0: bus 2 (BMD0) acpiprt2 at acpi0: bus 3 (BPD0) acpiprt3 at acpi0: bus -1 (BPD1) acpiprt4 at acpi0: bus -1 (BPD2) acpiprt5 at acpi0: bus 7 (BMF3) acpiprt6 at acpi0: bus 12 (P0P4) acpiprt7 at acpi0: bus 14 (P0P6) acpiprt8 at acpi0: bus 0 (PCI0) acpiprt9 at acpi0: bus 22 (PEX0) acpiprt10 at acpi0: bus 23 (PEX1) acpiprt11 at acpi0: bus -1 (PEX2) acpiprt12 at acpi0: bus -1 (PEX3) acpiprt13 at acpi0: bus 24 (PCIB) acpicpu0 at acpi0 acpicpu1 at acpi0 acpibtn0 at acpi0: PWRB pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 5000X Host" rev 0x31 ppb0 at pci0 dev 2 function 0 "Intel 5000 PCIE x8" rev 0x31 pci1 at ppb0 bus 1 ppb1 at pci1 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01 pci2 at ppb1 bus 2 ppb2 at pci2 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01 pci3 at ppb2 bus 3 ppb3 at pci1 dev 0 function 3 "Intel 6321ESB PCIE-PCIX" rev 0x01 pci4 at ppb3 bus 7 ppb4 at pci0 dev 3 function 0 "Intel 5000 PCIE" rev 0x31 pci5 at ppb4 bus 8 ppb5 at pci0 dev 4 function 0 "Intel 5000 PCIE x16" rev 0x31: msi pci6 at ppb5 bus 12 ppb6 at pci0 dev 5 function 0 "Intel 5000 PCIE" rev 0x31: msi pci7 at ppb6 bus 13 ppb7 at pci0 dev 6 function 0 "Intel 5000 PCIE" rev 0x31: msi pci8 at ppb7 bus 14 ppb8 at pci0 dev 7 function 0 "Intel 5000 PCIE" rev 0x31: msi pci9 at ppb8 bus 15 pchb1 at pci0 dev 16 function 0 "Intel 5000 Error Reporting" rev 0x31 pchb2 at pci0 dev 16 function 1 "Intel 5000 Error Reporting" rev 0x31 pchb3 at pci0 dev 16 function 2 "Intel 5000 Error Reporting" rev 0x31 pchb4 at pci0 dev 17 function 0 "Intel 5000 Reserved" rev 0x31 pchb5 at pci0 dev 19 function 0 "Intel 5000 Reserved" rev 0x31 pchb6 at pci0 dev 21 function 0 "Intel 5000 FBD" rev 0x31 pchb7 at pci0 dev 22 function 0 "Intel 5000 FBD" rev 0x31 ppb9 at pci0 dev 28 function 0 "Intel 6321ESB PCIE" rev 0x09: msi pci10 at ppb9 bus 22 bge0 at pci10 dev 0 function 0 "Broadcom BCM5721" rev 0x11, BCM5750 B1 (0x4101): apic 2 int 16, address 00:1c:c4:74:80:dc brgphy0 at bge0 phy 1: BCM5750 10/100/1000baseT PHY, rev. 0 ppb10 at pci0 dev 28 function 1 "Intel 6321ESB PCIE" rev 0x09: msi pci11 at ppb10 bus 23 bge1 at pci11 dev 0 function 0 "Broadcom BCM5721" rev 0x11, BCM5750 B1 (0x4101): apic 2 int 17, address 00:1c:c4:74:80:dd brgphy1 at bge1 phy 1: BCM5750 10/100/1000baseT PHY, rev. 0 uhci0 at pci0 dev 29 function 0 "Intel 6321ESB USB" rev 0x09: apic 2 int 23 uhci1 at pci0 dev 29 function 1 "Intel 6321ESB USB" rev 0x09: apic 2 int 23 uhci2 at pci0 dev 29 function 2 "Intel 6321ESB USB" rev 0x09: apic 2 int 23 ehci0 at pci0 dev 29 function 7 "Intel 6321ESB USB" rev 0x09: apic 2 int 23 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb11 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xd9 pci12 at ppb11 bus 24 vga1 at pci12 dev 2 function 0 "Matrox MGA G200e (ServerEngines)" rev 0x02 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) pcib0 at pci0 dev 31 function 0 "Intel 6321ESB LPC" rev 0x09 pciide0 at pci0 dev 31 function 2 "Intel 6321ESB SATA" rev 0x09: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility wd0 at pciide0 channel 0 drive 0: <FB080C4080> wd0: 16-sector PIO, LBA48, 76319MB, 156301488 sectors wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5 atapiscsi0 at pciide0 channel 1 drive 0 scsibus0 at atapiscsi0: 2 targets cd0 at scsibus0 targ 0 lun 0: <TEAC, DW-224E-V, C.CA> ATAPI 5/cdrom removable cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2 ichiic0 at pci0 dev 31 function 3 "Intel 6321ESB SMBus" rev 0x09: apic 2 int 19 iic0 at ichiic0 usb1 at uhci0: USB revision 1.0 uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb2 at uhci1: USB revision 1.0 uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb3 at uhci2: USB revision 1.0 uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1 isa0 at pcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 mtrr: Pentium Pro MTRR support uhidev0 at uhub3 port 1 configuration 1 interface 0 "ServerEngines SE USB Device" rev 1.10/0.01 addr 2 uhidev0: iclass 3/1 ukbd0 at uhidev0: 8 variable keys, 6 key codes wskbd1 at ukbd0 mux 1 wskbd1: connecting to wsdisplay0 uhidev1 at uhub3 port 1 configuration 1 interface 1 "ServerEngines SE USB Device" rev 1.10/0.01 addr 2 uhidev1: iclass 3/1 ums0 at uhidev1: 8 buttons, Z dir wsmouse0 at ums0 mux 0 vscsi0 at root scsibus1 at vscsi0: 256 targets softraid0 at root scsibus2 at softraid0: 256 targets root on wd0a (bde2e1931dc01d48.a) swap on wd0b dump on wd0b carp1: state transition: BACKUP -> MASTER arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value arp_rtrequest: bad gateway value carp10: state transition: BACKUP -> MASTER carp13: state transition: BACKUP -> MASTER carp2: state transition: BACKUP -> MASTER arp_rtrequest: bad gateway value carp20: state transition: BACKUP -> MASTER carp3: state transition: BACKUP -> MASTER carp4: state transition: BACKUP -> MASTER carp48: state transition: BACKUP -> MASTER carp49: state transition: BACKUP -> MASTER carp5: state transition: BACKUP -> MASTER carp50: state transition: BACKUP -> MASTER carp2: state transition: MASTER -> BACKUP carp20: state transition: MASTER -> BACKUP carp3: state transition: MASTER -> BACKUP carp4: state transition: MASTER -> BACKUP carp48: state transition: MASTER -> BACKUP carp10: state transition: MASTER -> BACKUP carp1: state transition: MASTER -> BACKUP carp50: state transition: MASTER -> BACKUP carp49: state transition: MASTER -> BACKUP carp5: state transition: MASTER -> BACKUP regards, WiesÅaw Herr
