Hi, Currently, I am regulary (~1 per day) get panic on an amd64 host (OpenBSD 6.9-current (GENERIC.MP) #492: Sat May 1 17:37:28 MDT 2021).
panic: kernel diagnostic assertion "refcnt != ~0" failed: file "/usr/src/sys/kern/kern_synch.c", line 826 Starting stack trace... panic(ffffffff81dfbc8e) at panic+0x11d __assert(ffffffff81e64b54,ffffffff81e0a6ee,33a,ffffffff81e03b7f) at __assert+0x2b refcnt_rele(fffffd810bf02458) at refcnt_rele+0x6f pf_state_key_unref(fffffd810bf023f0) at pf_state_key_unref+0x21 pf_remove_state(fffffd810c0c4578) at pf_remove_state+0x1fa pf_purge_expired_states(2) at pf_purge_expired_states+0x232 pf_purge(ffffffff82236a30) at pf_purge+0x33 taskq_thread(ffff800000032080) at taskq_thread+0x81 end trace frame: 0x0, count: 249 End of stack trace. The machine is running with ddb.panic=0, and hangs before rebooting (I assume it hangs during dump). I just changed the config to keep in ddb, and I should be able to get additionnal informations next time. As it is panicing in pf related code, some informations on the network: the machine has two networks cards (sis0 and em0) bridged, and a vlan (in rdomain). em0 is egress, and sis0 has only a network printer behind it. It panics whereas there is no particular interaction with the host, but time of panic is on state removal so I assume after a timeout. # ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768 index 4 priority 0 llprio 3 groups: lo inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet 127.0.0.1 netmask 0xff000000 em0: flags=248b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,AUTOCONF6TEMP,AUTOCONF6> mtu 1500 lladdr 00:1e:4f:f4:e8:50 index 1 priority 0 llprio 3 groups: egress media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause) status: active inet 192.168.92.23 netmask 0xffffff00 broadcast 192.168.92.255 inet6 fe80::21e:4fff:fef4:e850%em0 prefixlen 64 scopeid 0x1 inet6 2001:41d0:fe39:c05c:56b8:d15b:2e0a:8775 prefixlen 64 autoconf pltime 604614 vltime 2591814 inet6 2001:41d0:fe39:c05c:ed62:96cf:f62:de1a prefixlen 64 autoconf temporary pltime 51835 vltime 172081 sis0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:0f:b5:42:59:c0 index 2 priority 0 llprio 3 media: Ethernet autoselect (100baseTX full-duplex) status: active enc0: flags=0<> index 3 priority 0 llprio 3 groups: enc status: active bridge0: flags=41<UP,RUNNING> index 5 llprio 3 groups: bridge priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp sis0 flags=3<LEARNING,DISCOVER> port 2 ifpriority 0 ifcost 0 em0 flags=3<LEARNING,DISCOVER> port 1 ifpriority 0 ifcost 0 vlan102: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> rdomain 102 mtu 1500 lladdr 00:1e:4f:f4:e8:50 index 6 priority 0 llprio 3 encap: vnetid 102 parent em0 txprio packet rxprio outer groups: vlan media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause) status: active inet 192.168.102.23 netmask 0xffffff00 broadcast 192.168.102.255 lo102: flags=8008<LOOPBACK,MULTICAST> rdomain 102 mtu 32768 index 7 priority 0 llprio 3 groups: lo pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33136 index 8 priority 0 llprio 3 groups: pflog # cat /etc/pf.conf # See pf.conf(5) and /etc/examples/pf.conf # # avoid skip on lo to get tryton redir #set skip on lo block return # block stateless traffic pass # establish keep-state # redir 80 -> 8000 pass in proto tcp to (self) port 80 rdr-to 2001:41d0:fe39:c05c:56b8:d15b:2e0a:8775 port 8000 # By default, do not permit remote connections to X11 block return in on ! lo0 proto tcp to port 6000:6010 # By default, no network access for build users block return out proto {tcp udp} user {build _pbuild} Full dmesg + panic message (captured via serial). >> OpenBSD/amd64 BOOT 3.53 boot> bsd booting hd0a:bsd: 14476584... entry point at 0xffffffff81001000 [ using 3023408 bytes of bsd ELF symbol table ] Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. Copyright (c) 1995-2021 OpenBSD. All rights reserved. https://www.OpenBSD.org OpenBSD 6.9-current (GENERIC.MP) #492: Sat May 1 17:37:28 MDT 2021 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 4141871104 (3949MB) avail mem = 4000907264 (3815MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xf0450 (80 entries) bios0: vendor Dell Inc. version "A22" date 06/11/2012 bios0: Dell Inc. OptiPlex 755 acpi0 at bios0 acpi0: TCPA checksum error: ACPI 3.0 acpi0: sleep states S0 S1 S3 S4 S5 acpi0: tables DSDT FACP SSDT APIC BOOT ASF! MCFG HPET TCPA DMAR SLIC SSDT SSDT SSDT acpi0: wakeup devices VBTN(S4) PCI0(S5) PCI4(S5) PCI2(S5) PCI3(S5) PCI1(S5) PCI5(S5) PCI6(S5) USB0(S3) USB1(S3) USB2(S3) USB3(S3) USB4(S3) USB5(S3) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM)2 Duo CPU E6750 @ 2.66GHz, 2660.30 MHz, 06-0f-0b cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF,MELTDOWN cpu0: 4MB 64b/line 16-way L2 cache mtrr: Pentium Pro MTRR support, 7 var ranges, 88 fixed ranges cpu0: apic clock running at 332MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Core(TM)2 Duo CPU E6750 @ 2.66GHz, 2660.01 MHz, 06-0f-0b cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF,MELTDOWN cpu1: 4MB 64b/line 16-way L2 cache ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins, remapped acpimcfg0 at acpi0 acpimcfg0: addr 0xe0000000, bus 0-255 acpimcfg0: addr 0x0, bus 0-0 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 3 (PCI4) acpiprt2 at acpi0: bus 2 (PCI2) acpiprt3 at acpi0: bus -1 (PCI3) acpiprt4 at acpi0: bus 1 (PCI1) acpiprt5 at acpi0: bus -1 (PCI5) acpiprt6 at acpi0: bus -1 (PCI6) acpibtn0 at acpi0: VBTN acpipci0 at acpi0 PCI0 acpicmos0 at acpi0 acpicpu0 at acpi0: C1(@1 halt!), PSS acpicpu1 at acpi0: C1(@1 halt!), PSS cpu0: Enhanced SpeedStep 2660 MHz: speeds: 2667, 2333, 2000 MHz pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 82Q35 Host" rev 0x02 ppb0 at pci0 dev 1 function 0 "Intel 82Q35 PCIE" rev 0x02: msi pci1 at ppb0 bus 1 radeondrm0 at pci1 dev 0 function 0 "ATI Radeon HD 2400 Pro" rev 0x00 drm0 at radeondrm0 radeondrm0: msi "Intel 82Q35 HECI" rev 0x02 at pci0 dev 3 function 0 not configured pciide0 at pci0 dev 3 function 2 "Intel 82Q35 PT IDER" rev 0x02: DMA (unsupported), channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide0: using apic 8 int 18 for native-PCI interrupt pciide0: channel 0 ignored (not responding; disabled or no drives?) pciide0: channel 1 ignored (not responding; disabled or no drives?) puc0 at pci0 dev 3 function 3 "Intel 82Q35 KT" rev 0x02: ports: 16 com com4 at puc0 port 0 apic 8 int 17: ns16550a, 16 byte fifo com4: probed fifo depth: 0 bytes em0 at pci0 dev 25 function 0 "Intel ICH9 IGP AMT" rev 0x02: msi, address 00:1e:4f:f4:e8:50 uhci0 at pci0 dev 26 function 0 "Intel 82801I USB" rev 0x02: apic 8 int 16 uhci1 at pci0 dev 26 function 1 "Intel 82801I USB" rev 0x02: apic 8 int 17 ehci0 at pci0 dev 26 function 7 "Intel 82801I USB" rev 0x02: apic 8 int 22 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 azalia0 at pci0 dev 27 function 0 "Intel 82801I HD Audio" rev 0x02: msi azalia0: codecs: Analog Devices AD1984 audio0 at azalia0 ppb1 at pci0 dev 28 function 0 "Intel 82801I PCIE" rev 0x02: msi pci2 at ppb1 bus 2 uhci2 at pci0 dev 29 function 0 "Intel 82801I USB" rev 0x02: apic 8 int 23 uhci3 at pci0 dev 29 function 1 "Intel 82801I USB" rev 0x02: apic 8 int 17 uhci4 at pci0 dev 29 function 2 "Intel 82801I USB" rev 0x02: apic 8 int 18 ehci1 at pci0 dev 29 function 7 "Intel 82801I USB" rev 0x02: apic 8 int 23 usb1 at ehci1: USB revision 2.0 uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb2 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x92 pci3 at ppb2 bus 3 sis0 at pci3 dev 2 function 0 "NS DP83815" rev 0x00, DP83816A: apic 8 int 18, address 00:0f:b5:42:59:c0 nsphyter0 at sis0 phy 0: DP83815 10/100 PHY, rev. 1 pcib0 at pci0 dev 31 function 0 "Intel 82801IO LPC" rev 0x02 ahci0 at pci0 dev 31 function 2 "Intel 82801I AHCI" rev 0x02: msi, AHCI 1.2 ahci0: port 0: 3.0Gb/s ahci0: port 1: 1.5Gb/s ahci0: PHY offline on port 2 ahci0: PHY offline on port 3 ahci0: PHY offline on port 5 scsibus1 at ahci0: 32 targets sd0 at scsibus1 targ 0 lun 0: <ATA, ST3160815AS, 3.AD> t10.ATA_ST3160815AS_9RX7BKBD sd0: 152587MB, 512 bytes/sector, 312500000 sectors cd0 at scsibus1 targ 1 lun 0: <HL-DT-ST, DVD+-RW GSA-H73N, B103> removable ichiic0 at pci0 dev 31 function 3 "Intel 82801I SMBus" rev 0x02: apic 8 int 18 iic0 at ichiic0 spdmem0 at iic0 addr 0x50: 1GB DDR2 SDRAM non-parity PC2-5300CL5 spdmem1 at iic0 addr 0x51: 1GB DDR2 SDRAM non-parity PC2-5300CL5 spdmem2 at iic0 addr 0x52: 1GB DDR2 SDRAM non-parity PC2-5300CL5 spdmem3 at iic0 addr 0x53: 1GB DDR2 SDRAM non-parity PC2-5300CL5 usb2 at uhci0: USB revision 1.0 uhub2 at usb2 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb3 at uhci1: USB revision 1.0 uhub3 at usb3 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb4 at uhci2: USB revision 1.0 uhub4 at usb4 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb5 at uhci3: USB revision 1.0 uhub5 at usb5 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb6 at uhci4: USB revision 1.0 uhub6 at usb6 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 isa0 at pcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com0: console pckbc0 at isa0 port 0x60/5 irq 1 irq 12 pckbd0 at pckbc0 (kbd slot) wskbd0 at pckbd0 mux 1 vga0 at isa0 port 0x3b0/48 iomem 0xa0000/131072 wsdisplay at vga0 not configured pcppi0 at isa0 port 0x61 spkr0 at pcppi0 vmm0 at mainbus0: VMX (using slow L1TF mitigation) dt: 443 probes uhidev0 at uhub2 port 1 configuration 1 interface 0 "Dell Dell USB Keyboard" rev 1.10/3.52 addr 2 uhidev0: iclass 3/1 ukbd0 at uhidev0: 8 variable keys, 6 key codes wskbd1 at ukbd0 mux 1 uhidev1 at uhub2 port 2 configuration 1 interface 0 "Dell Dell Premium USB Optical Mouse" rev 2.00/29.00 addr 3 uhidev1: iclass 3/1 ums0 at uhidev1: 5 buttons, Z and W dir wsmouse0 at ums0 mux 0 vscsi0 at root scsibus2 at vscsi0: 256 targets softraid0 at root scsibus3 at softraid0: 256 targets root on sd0a (f1307242b412e6c8.a) swap on sd0b dump on sd0b radeondrm0: RV610 radeondrm0: 1680x1050, 32bpp wsdisplay0 at radeondrm0 mux 1 pckbd_enable: command error wskbd1: connecting to wsdisplay0 wsdisplay0: screen 0-5 added (std, vt100 emulation) Automatic boot in progress: starting file system checks. /dev/sd0a (f1307242b412e6c8.a): file system is clean; not checking /dev/sd0h (f1307242b412e6c8.h): file system is clean; not checking /dev/sd0g (f1307242b412e6c8.g): file system is clean; not checking /dev/sd0d (f1307242b412e6c8.d): file system is clean; not checking /dev/sd0e (f1307242b412e6c8.e): file system is clean; not checking /dev/sd0f (f1307242b412e6c8.f): file system is clean; not checking /dev/sd0j (f1307242b412e6c8.j): file system is clean; not checking /dev/sd0k (f1307242b412e6c8.k): file system is clean; not checking /dev/sd0l (f1307242b412e6c8.l): file system is clean; not checking /dev/sd0m (f1307242b412e6c8.m): file system is clean; not checking kbd: keyboard mapping set to fr pf enabled machdep.allowaperture: 0 -> 2 ddb.panic: 1 -> 0 ddb.log: 1 -> 1 net.inet.ip.forwarding: 0 -> 0 net.inet6.ip6.forwarding: 0 -> 0 kern.seminfo.semmni: 10 -> 60 kern.seminfo.semmns: 60 -> 1024 starting network em0: 192.168.92.23 lease accepted from 192.168.92.2 (c8:be:19:e2:2c:ed) reordering libraries: done. starting early daemons: syslogd pflogd ntpd. starting RPC daemons:. savecore: no core dump acpidump: XSDT entry 7 is corrupt checking quotas: done. clearing /tmp kern.securelevel: 0 -> 1 turning on accounting creating runtime link editor directory cache. preserving editor files. starting network daemons: sshd smtpd sndiod. starting package daemons: messagebus cupsd postgresql trytond trytond_cron. starting local daemons: apmd cron xenodm. Mon May 3 08:43:39 CEST 2021 pckbd_enable: command error pckbd_disable: command error panic: kernel diagnostic assertion "refcnt != ~0" failed: file "/usr/src/sys/kern/kern_synch.c", line 826 Starting stack trace... panic(ffffffff81dfbc8e) at panic+0x11d __assert(ffffffff81e64b54,ffffffff81e0a6ee,33a,ffffffff81e03b7f) at __assert+0x2b refcnt_rele(fffffd810bf02458) at refcnt_rele+0x6f pf_state_key_unref(fffffd810bf023f0) at pf_state_key_unref+0x21 pf_remove_state(fffffd810c0c4578) at pf_remove_state+0x1fa pf_purge_expired_states(2) at pf_purge_expired_states+0x232 pf_purge(ffffffff82236a30) at pf_purge+0x33 taskq_thread(ffff800000032080) at taskq_thread+0x81 end trace frame: 0x0, count: 249 End of stack trace. syncing disks...8 7 done [EOT] -- Sebastien Marie