Seen about a dozen times on a box doing ospfd + ospf6d (for internal
routes), bgpd (for default route) + isakmpd, shortly after startup
after updating:
splassert: sorwakeup: want 5 have 4
Starting stack trace...
splassert_check() at splassert_check+0x78
sorwakeup() at sorwakeup+0x27
timeout_run() at timeout_run+0x48
softclock() at softclock+0x14c
softintr_dispatch() at softintr_dispatch+0x8b
Xsoftclock() at Xsoftclock+0x1f
--- interrupt ---
end of kernel
end trace frame: 0x51, count: 251
0x8:
End of stack trace.
# grep splassert:.sorwakeup /var/log/messages
Sep 26 13:43:13 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:13 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:13 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:14 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:14 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:15 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:15 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:15 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:15 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:15 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:20 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:20 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:20 /bsd: splassert: sorwakeup: want 5 have 4
At this point it has stopped recurring.
More logs in case the timing gives any clues:
Sep 26 13:40:30 reboot: rebooted by sthen
Sep 26 13:40:35 syslogd: exiting on signal 15
Sep 26 13:41:29 syslogd: start
Sep 26 13:41:29 syslogd: sendto: Can't assign requested address
Sep 26 13:41:29 syslogd: sendto: Can't assign requested address
Sep 26 13:41:29 /bsd: OpenBSD 6.0-current (GENERIC.MP) #2492: Sun Sep 25
20:10:21 MDT 2016
Sep 26 13:41:29 /bsd:
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
..
Sep 26 13:41:36 isakmpd[81976]: sendmsg (14, 0x7f7ffffd30f0, 0): No route to
host
Sep 26 13:41:36 isakmpd[81976]: sendmsg (14, 0x7f7ffffd30f0, 0): No route to
host
Sep 26 13:41:43 isakmpd[81976]: message_recv: invalid cookie(s)
aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbb
Sep 26 13:41:43 isakmpd[81976]: dropped message from xx.xxx.xx.210 port 500
due to notification type INVALID_COOKIE
Sep 26 13:41:43 isakmpd[81976]: sendmsg (14, 0x7f7ffffd30f0, 0): No route to
host
Sep 26 13:41:43 last message repeated 2 times
Sep 26 13:41:50 isakmpd[81976]: message_recv: invalid cookie(s)
aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbb
Sep 26 13:41:50 isakmpd[81976]: dropped message from xx.xxx.xx.210 port 500
due to notification type INVALID_COOKIE
Sep 26 13:41:50 isakmpd[81976]: sendmsg (14, 0x7f7ffffd30f0, 0): No route to
host
Sep 26 13:41:52 last message repeated 2 times
Sep 26 13:41:59 isakmpd[81976]: message_recv: invalid cookie(s)
aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbb
Sep 26 13:41:59 isakmpd[81976]: dropped message from xx.xxx.xx.210 port 500
due to notification type INVALID_COOKIE
Sep 26 13:41:59 isakmpd[81976]: sendmsg (14, 0x7f7ffffd30f0, 0): No route to
host
Sep 26 13:42:03 isakmpd[81976]: sendmsg (14, 0x7f7ffffd30f0, 0): No route to
host
Sep 26 13:42:03 isakmpd[81976]: transport_send_messages: giving up on exchange
peer-yy.yy.yyy.140-local-zzz.zz.zzz.64, no response from peer yy.yy.yyy.140:500
Sep 26 13:42:03 isakmpd[81976]: sendmsg (14, 0x7f7ffffd30f0, 0): No route to
host
Sep 26 13:42:03 isakmpd[81976]: transport_send_messages: giving up on exchange
peer-xx.xxx.xx.210-local-zzz.zz.zzz.64, no response from peer xx.xxx.xx.210:500
Sep 26 13:42:10 isakmpd[81976]: message_recv: invalid cookie(s)
aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbb
Sep 26 13:42:10 isakmpd[81976]: dropped message from xx.xxx.xx.210 port 500
due to notification type INVALID_COOKIE
Sep 26 13:42:10 isakmpd[81976]: sendmsg (14, 0x7f7ffffd30f0, 0): No route to
host
Sep 26 13:43:13 /bsd: splassert: sorwakeup: want 5 have 4
Sep 26 13:43:13 /bsd: Starting stack trace...
OpenBSD 6.0-current (GENERIC.MP) #2492: Sun Sep 25 20:10:21 MDT 2016
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8557764608 (8161MB)
avail mem = 8293900288 (7909MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x7f3b9000 (51 entries)
bios0: vendor American Megatrends Inc. version "1.0c" date 02/27/2014
bios0: Supermicro A1SAi
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP FPDT SPMI MCFG WDAT UEFI APIC BDAT HPET SSDT HEST BERT
ERST EINJ
acpi0: wakeup devices PEX1(S0) PEX2(S0) PEX3(S0) EHC1(S0)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz, 2400.44 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu0: 1MB 64b/line 16-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.0.0.0.0.3, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz, 2400.01 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu1: 1MB 64b/line 16-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz, 2400.01 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu2: 1MB 64b/line 16-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz, 2400.01 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu3: 1MB 64b/line 16-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 8 (application processor)
cpu4: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz, 2400.01 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu4: 1MB 64b/line 16-way L2 cache
cpu4: smt 0, core 4, package 0
cpu5 at mainbus0: apid 10 (application processor)
cpu5: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz, 2400.01 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu5: 1MB 64b/line 16-way L2 cache
cpu5: smt 0, core 5, package 0
cpu6 at mainbus0: apid 12 (application processor)
cpu6: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz, 2400.01 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu6: 1MB 64b/line 16-way L2 cache
cpu6: smt 0, core 6, package 0
cpu7 at mainbus0: apid 14 (application processor)
cpu7: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz, 2400.01 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu7: 1MB 64b/line 16-way L2 cache
cpu7: smt 0, core 7, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEX1)
acpiprt2 at acpi0: bus 2 (BR04)
acpiprt3 at acpi0: bus 3 (PEX2)
acpiprt4 at acpi0: bus 4 (PEX3)
acpicpu0 at acpi0: C2(350@41 mwait.3@0x51), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C2(350@41 mwait.3@0x51), C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: C2(350@41 mwait.3@0x51), C1(1000@1 mwait.1), PSS
acpicpu3 at acpi0: C2(350@41 mwait.3@0x51), C1(1000@1 mwait.1), PSS
acpicpu4 at acpi0: C2(350@41 mwait.3@0x51), C1(1000@1 mwait.1), PSS
acpicpu5 at acpi0: C2(350@41 mwait.3@0x51), C1(1000@1 mwait.1), PSS
acpicpu6 at acpi0: C2(350@41 mwait.3@0x51), C1(1000@1 mwait.1), PSS
acpicpu7 at acpi0: C2(350@41 mwait.3@0x51), C1(1000@1 mwait.1), PSS
"PNP0003" at acpi0 not configured
"PNP0501" at acpi0 not configured
"PNP0501" at acpi0 not configured
"IPI0001" at acpi0 not configured
"PNP0C33" at acpi0 not configured
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 2400 MHz: speeds: 2401, 2400, 2300, 2200, 2100, 2000,
1900, 1800, 1700, 1600, 1500, 1400, 1300, 1200 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 vendor "Intel", unknown product 0x1f01 rev 0x02
ppb0 at pci0 dev 1 function 0 "Intel Atom C2000 PCIE" rev 0x02: msi
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03
pci2 at ppb1 bus 2
vga1 at pci2 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb2 at pci0 dev 2 function 0 "Intel Atom C2000 PCIE" rev 0x02: msi
pci3 at ppb2 bus 3
xhci0 at pci3 dev 0 function 0 "Renesas uPD720201 xHCI" rev 0x03: msi
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Renesas xHCI root hub" rev 3.00/1.00
addr 1
ppb3 at pci0 dev 3 function 0 "Intel Atom C2000 PCIE" rev 0x02: msi
pci4 at ppb3 bus 4
pchb1 at pci0 dev 14 function 0 "Intel Atom C2000 RAS" rev 0x02
"Intel Atom C2000 RCEC" rev 0x02 at pci0 dev 15 function 0 not configured
"Intel Atom C2000 SMBus" rev 0x02 at pci0 dev 19 function 0 not configured
em0 at pci0 dev 20 function 0 "Intel I354 SGMII" rev 0x03: msi, address
00:25:90:f1:c1:34
em1 at pci0 dev 20 function 1 "Intel I354 SGMII" rev 0x03: msi, address
00:25:90:f1:c1:35
em2 at pci0 dev 20 function 2 "Intel I354 SGMII" rev 0x03: msi, address
00:25:90:f1:c1:36
em3 at pci0 dev 20 function 3 "Intel I354 SGMII" rev 0x03: msi, address
00:25:90:f1:c1:37
ehci0 at pci0 dev 22 function 0 "Intel Atom C2000 USB" rev 0x02: apic 2 int 23
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
addr 1
ahci0 at pci0 dev 23 function 0 "Intel Atom C2000 AHCI" rev 0x02: msi, AHCI 1.3
scsibus1 at ahci0: 32 targets
ahci1 at pci0 dev 24 function 0 "Intel Atom C2000 AHCI" rev 0x02: msi, AHCI 1.3
ahci1: port 0: 6.0Gb/s
scsibus2 at ahci1: 32 targets
sd0 at scsibus2 targ 0 lun 0: <ATA, INTEL SSDSC2BB08, D201> SCSI3 0/direct
fixed naa.55cd2e404b6471bf
sd0: 76319MB, 512 bytes/sector, 156301488 sectors, thin
pcib0 at pci0 dev 31 function 0 "Intel Atom C2000 PCU" rev 0x02
ichiic0 at pci0 dev 31 function 3 "Intel Atom C2000 PCU SMBus" rev 0x02: apic 2
int 18
iic0 at ichiic0
sdtemp0 at iic0 addr 0x18: at30ts00
sdtemp1 at iic0 addr 0x1a: ts3000gb2
iic0: addr 0x2e 00=3c words 00=3c3c 01=0000 02=0000 03=0000 04=0000 05=0000
06=0000 07=0000
spdmem0 at iic0 addr 0x50: 4GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem1 at iic0 addr 0x52: 4GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
com1: console
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
uhub2 at uhub1 port 1 configuration 1 interface 0 "Intel product 0x07db" rev
2.00/0.02 addr 2
uhub3 at uhub2 port 3 configuration 1 interface 0 "vendor 0x0000 product
0x0001" rev 2.00/0.00 addr 3
uhidev0 at uhub3 port 1 configuration 1 interface 0 "ATEN International product
0x2419" rev 1.10/1.00 addr 4
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub3 port 1 configuration 1 interface 1 "ATEN International product
0x2419" rev 1.10/1.00 addr 4
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse1 at ums0 mux 0
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (7f0e094a72cc105d.a) swap on sd0b dump on sd0b
