Strange kernel panic-like problem
I recently upgraded a box running 6.2 to 6.4 via clean install. After a few
days of running normally it started locking up, usually within a minute or so
after booting up to the login prompt. ddb appears on the console.
I eventually thought to try booting bsd.sp, which has been running for about a
day now without locking up.
Any clues to point me in the right direction would be much appreciated.
Here's some excerpts from the serial console (different sessions) and a dmesg:
ddb{0}> show panic
the kernel did not panic
ddb{0}> trace
acpicpu_idle() at acpicpu_idle+0x1ea
sched_idle(0) at sched_idle+0x245
end trace frame: 0x0, count: -2
login: NMI ... going o debugger
ddb{0}䀿 movq$0x8,%rcxuaei[1;24r c
db{}>
ddb{0}>
d{0}>
ddb{0}
ddb{0}>
ddb{0}>
ddb{�}>
ddb{0>
db{0}> show panic
the kernel did not panic
ddb{0}>
the kernel did not panic
ddb{0}>
the kernel did not panic
ddb{0}>
the kernel did not panic
ddb{0}>
the kernel di no paniddb{0}>
the erne i nic
ddb{0}>
the kernel did not nic
ddb{0}>
thernel id not nc
ddb{0}>
the keel did notpanic
ddb{0}>
theknel did not nic
ddb{0}>
the kernel did not panic
ddb{0}>
the kernel did not panic
ddb{0}>
the kernel did not panc
ddb0}>
the kernel did not panic
ddb{0}> traccce
No such command
ddb{0}>
the kernel did not panic
ddb{0}> tracce
No such command
ddb{0}> traace
No such command
ddb{0}> trace
acpicpu_idle() at acpicpu_idle+0x1ea
sched_idle(0) at sched_idle+0x245
end trace frame: 0x0, count: -2
ddb{0}>
acpicpu_idle() at acpicpu_idle+0x1ea
end trace frame: 0x8000218fca60, count: 0
ddb{0}>
acpicpu_idle() at acpicpu_idle+0x1ea
end trace frame: 0x8000218fca60, count: 0
ddb{0}>
acpicpu_idle() at acpicpu_idle+0x1ea
end trace frame: 0x8000218fca60, count: 0
ddb{0}>
acpicpu_idle() at acpicpu_idle+0x1ea
end trace frame: 0x8000218fca60, count: 0
ddb{0}>
acpicpu_idle() at acpicpu_idle+0x1ea
end trace frame: 0x8000218fca60, count: 0
ddb{0}>
acpicpu_idle() at acpicpu_idle+0x1ea
end trace frame: 0x8000218fca60, count: 0
ddb{0}> boot dump
syncing disks..
ddb{1}> boot dump
syncing disks...panic: kernel diagnostic assertion "p->p_wchan == NULL" failed:
file "/usr/src/sys/kern/kern_sched.c", line 338
Stopped at db_enter+0x12: popq%r11
TIDPIDUID PRFLAGS PFLAGS CPU COMMAND
72309 19595 730x100010 0x800 syslogd
db_enter() at db_enter+0x12
panic() at panic+0x120
__assert(811929f4,80002191fa00,800021750ff0,8000e960) a
t __assert+0x24
sched_chooseproc() at sched_chooseproc+0x241
mi_switch() at mi_switch+0x1b4
sleep_finish(6d3fbc03daad5a02,80002191fb10) at sleep_finish+0x7f
sleep_finish_all(270c6ff115c03531,80002191fb10) at sleep_finish_all+0x1f
tsleep(64263b673ec8ed92,ff02417d4200,ff027f616830,65420) at tsleep+0xcd
getblk(f6e14443bcd449df,ff027f6167d0,80002191fd00,0,ff027f3d3000) a
t getblk+0xf5
bread(80145000,ff027f616a28,ff027f3d3000,0) at bread+0x1b
ffs_update(292bb10918b461bf,ff027f616a28) at ffs_update+0xfc
VOP_FSYNC(f6e14443bcb92266,80002191fe38,2be1d547d68afbf,8000e960) a
t VOP_FSYNC+0x52
ffs_sync_vnode(5116a32c89f5a557,80002191fe38) at ffs_sync_vnode+0xd2
vfs_mount_foreach_vnode(9582de35d54b8f61,2,8000e960) at vfs_mount_forea
ch_vnode+0x4e
end trace frame: 0x80002191fea0, count: 0
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports. Insufficient info makes it difficult to find and fix bugs.
ddb{1}> boot sync
panic: kernel diagnostic assertion "__mp_lock_held(&sched_lock, curcpu()) == 0"
failed: file "/usr/src/sys/kern/kern_lock.c", line 63
Stopped at db_enter+0x12: popq%r11
db_enter() at db_enter+0x12
panic() at panic+0x120
__assert(811929f4,80002191f530,0,ff02369aeae8) at __assert+0x24
_kernel_lock(778d687e7309b96e,1) at _kernel_lock+0xea
solock(537a4000962b3bf3) at solock+0x44
route_input(6e54ffebe841494b,80002191f610,8012f000) at route_input+
0xd1
if_down(8012f000) at if_down+0x94
if_downall() at if_downall+0x62
boot(c) at boot+0x8d
reboot(4800) at reboot+0x5a
nvramattach(81d05260) at nvramattach
db_boot_sync_cmd(819a1c9e,80002191f6c0,81d05260,1) at db_bo
ot_sync_cmd+0xe
db_command(be4ef64b60647db8,0) at db_command+0x2b4
db_command_loop() at db_command_loop+0x96
end trace frame: 0x80002191f820, count: 0
ddb{1}>
ddb{1}> dmesg
OpenBSD 6.4 (GENERIC.MP) #6: Sat Jan 26 20:37:44 CET 2019
r...@syspatch-64-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.
MP
real mem = 8544854016 (8149MB)
avail mem = 8276611072 (7893MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x7f4d8000 (50 entries)
bios0: vendor American Megatrends Inc. version "1.1a" date 08/27/2015
bios0: Supermicro A1SAi
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP FPDT FIDT SPMI MCFG WDA
Re: Strange kernel panic-like problem
On Thu, Feb 07, 2019 at 11:18:40PM -0600, Kyle wrote:
> I recently upgraded a box running 6.2 to 6.4 via clean install. After a few
> days of running normally it started locking up, usually within a minute or so
> after booting up to the login prompt. ddb appears on the console.
>
> I eventually thought to try booting bsd.sp, which has been running for about
> a day now without locking up.
>
> Any clues to point me in the right direction would be much appreciated.
>
> Here's some excerpts from the serial console (different sessions) and a dmesg:
>
>
> ddb{0}> show panic
> the kernel did not panic
> ddb{0}> trace
> acpicpu_idle() at acpicpu_idle+0x1ea
> sched_idle(0) at sched_idle+0x245
> end trace frame: 0x0, count: -2
>
>
>
> login: NMI ... going o debugger
NMIs (Non Maskable Interrupts) are often an indication of hardware problems.
-Otto
> ddb{0}䀿 movq$0x8,%rcxuaei[1;24r c
>db{}>
> ddb{0}>
> d{0}>
> ddb{0}
> ddb{0}>
> ddb{0}>
> ddb{�}>
> ddb{0>
> db{0}> show panic
> the kernel did not panic
> ddb{0}>
> the kernel did not panic
> ddb{0}>
> the kernel did not panic
> ddb{0}>
> the kernel did not panic
> ddb{0}>
> the kernel di no paniddb{0}>
> the erne i nic
> ddb{0}>
> the kernel did not nic
> ddb{0}>
> thernel id not nc
> ddb{0}>
> the keel did notpanic
> ddb{0}>
> theknel did not nic
> ddb{0}>
> the kernel did not panic
> ddb{0}>
> the kernel did not panic
> ddb{0}>
> the kernel did not panc
> ddb0}>
> the kernel did not panic
> ddb{0}> traccce
> No such command
> ddb{0}>
> the kernel did not panic
> ddb{0}> tracce
> No such command
> ddb{0}> traace
> No such command
> ddb{0}> trace
> acpicpu_idle() at acpicpu_idle+0x1ea
> sched_idle(0) at sched_idle+0x245
> end trace frame: 0x0, count: -2
> ddb{0}>
> acpicpu_idle() at acpicpu_idle+0x1ea
> end trace frame: 0x8000218fca60, count: 0
> ddb{0}>
> acpicpu_idle() at acpicpu_idle+0x1ea
> end trace frame: 0x8000218fca60, count: 0
> ddb{0}>
> acpicpu_idle() at acpicpu_idle+0x1ea
> end trace frame: 0x8000218fca60, count: 0
> ddb{0}>
> acpicpu_idle() at acpicpu_idle+0x1ea
> end trace frame: 0x8000218fca60, count: 0
> ddb{0}>
> acpicpu_idle() at acpicpu_idle+0x1ea
> end trace frame: 0x8000218fca60, count: 0
> ddb{0}>
> acpicpu_idle() at acpicpu_idle+0x1ea
> end trace frame: 0x8000218fca60, count: 0
> ddb{0}> boot dump
> syncing disks..
>
>
>
>
> ddb{1}> boot dump
> syncing disks...panic: kernel diagnostic assertion "p->p_wchan == NULL"
> failed: file "/usr/src/sys/kern/kern_sched.c", line 338
> Stopped at db_enter+0x12: popq%r11
> TIDPIDUID PRFLAGS PFLAGS CPU COMMAND
> 72309 19595 730x100010 0x800 syslogd
> db_enter() at db_enter+0x12
> panic() at panic+0x120
> __assert(811929f4,80002191fa00,800021750ff0,8000e960)
> a
> t __assert+0x24
> sched_chooseproc() at sched_chooseproc+0x241
> mi_switch() at mi_switch+0x1b4
> sleep_finish(6d3fbc03daad5a02,80002191fb10) at sleep_finish+0x7f
> sleep_finish_all(270c6ff115c03531,80002191fb10) at sleep_finish_all+0x1f
> tsleep(64263b673ec8ed92,ff02417d4200,ff027f616830,65420) at
> tsleep+0xcd
>
> getblk(f6e14443bcd449df,ff027f6167d0,80002191fd00,0,ff027f3d3000)
> a
> t getblk+0xf5
> bread(80145000,ff027f616a28,ff027f3d3000,0) at bread+0x1b
> ffs_update(292bb10918b461bf,ff027f616a28) at ffs_update+0xfc
> VOP_FSYNC(f6e14443bcb92266,80002191fe38,2be1d547d68afbf,8000e960)
> a
> t VOP_FSYNC+0x52
> ffs_sync_vnode(5116a32c89f5a557,80002191fe38) at ffs_sync_vnode+0xd2
> vfs_mount_foreach_vnode(9582de35d54b8f61,2,8000e960) at
> vfs_mount_forea
> ch_vnode+0x4e
> end trace frame: 0x80002191fea0, count: 0
> https://www.openbsd.org/ddb.html describes the minimum info required in bug
> reports. Insufficient info makes it difficult to find and fix bugs.
> ddb{1}> boot sync
> panic: kernel diagnostic assertion "__mp_lock_held(&sched_lock, curcpu()) ==
> 0" failed: file "/usr/src/sys/kern/kern_lock.c", line 63
> Stopped at db_enter+0x12: popq%r11
> db_enter() at db_enter+0x12
> panic() at panic+0x120
> __assert(811929f4,80002191f530,0,ff02369aeae8) at
> __assert+0x24
>
> _kernel_lock(778d687e7309b96e,1) at _kernel_lock+0xea
> solock(537a4000962b3bf3) at solock+0x44
> route_input(6e54ffebe841494b,80002191f610,8012f000) at
> route_input+
> 0xd1
> if_down(8012f000) at if_down+0x94
> if_downall() at if_downall+0x62
> boot(c) at boot+0x8d
> reboot(4800) at reboot+0x5a
> nvramattach(81d05260) at nvramattach
> db_boot_sync_cmd(819a1c9e,80002191f6c0,81d05260,1) at
> db_bo
> ot_sync_cmd+0xe
> db_command(be4ef64b60647db8,0) at db_command+0x2b4
> db_command_loop() at db_command_loop+0x96
> end trace frame: 0x80002191f820, count: 0
> ddb{1}>
>
>
>
> ddb{1}> dmesg
> OpenBSD 6.4 (GENERIC.MP) #6: Sat Jan 26 20:37
