[Bug 204633] If INVARIANTS is enabled, free() may attempt to acquire sleeping lock

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204633

Bug ID: 204633
   Summary: If INVARIANTS is enabled, free() may attempt to
acquire sleeping lock
   Product: Base System
   Version: 11.0-CURRENT
  Hardware: Any
OS: Any
Status: New
  Severity: Affects Some People
  Priority: ---
 Component: kern
  Assignee: freebsd-bugs@FreeBSD.org
  Reporter: j...@freebsd.org

While testing new code with WITNESS enabled, I saw this panic:

panic: acquiring blockable sleep lock with spinlock or critical section held
(sleep mutex) 64 Bucket @ /usr/src/sys/vm/uma_dbg.c:217
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe21c690
vpanic() at vpanic+0x182/frame 0xfe21c710
kassert_panic() at kassert_panic+0x126/frame 0xfe21c780
witness_checkorder() at witness_checkorder+0x13b/frame 0xfe21c800
__mtx_lock_flags() at __mtx_lock_flags+0xa4/frame 0xfe21c850
uma_dbg_getslab() at uma_dbg_getslab+0x42/frame 0xfe21c880
uma_dbg_alloc() at uma_dbg_alloc+0x36/frame 0xfe21c8b0
uma_zalloc_arg() at uma_zalloc_arg+0x53e/frame 0xfe21c910
bucket_alloc() at bucket_alloc+0xa6/frame 0xfe21c930
uma_zfree_arg() at uma_zfree_arg+0x293/frame 0xfe21c980
free() at free+0x8b/frame 0xfe21c9c0
pmc_hook_handler() at pmc_hook_handler+0xbbb/frame 0xfe21ca50
thread_exit() at thread_exit+0x1b7/frame 0xfe21ca90
kern_thr_exit() at kern_thr_exit+0x119/frame 0xfe21cac0
sys_thr_exit() at sys_thr_exit+0x62/frame 0xfe21cae0
amd64_syscall() at amd64_syscall+0x2de/frame 0xfe21cbf0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe21cbf0
--- syscall (431, FreeBSD ELF64, sys_thr_exit), rip = 0x40b70a, rsp =
0x7fffde9f2ee8, rbp = 0x7fffde9f2f00 ---
KDB: enter: panic


The free() man page promises that free() will not sleep. However, if INVARIANTS
is enabled, free() may very well attempt to acquire a sleeping lock. In fact,
it appears that malloc() may even attempt to acquire a sleeping lock if
INVARIANTS is enabled.

The problem is that uma_zalloc_arg() calls uma_dbg_alloc(). uma_dbg_alloc()
calls uma_dbg_getslab(). uma_dbg_getslab() may acquire the zone lock, which is
a normal (sleepable) mutex.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-bugs@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"

[Bug 204633] If INVARIANTS is enabled, free() may attempt to acquire sleeping lock

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204633

Mark Johnston  changed:

   What|Removed |Added

 CC||ma...@freebsd.org

--- Comment #1 from Mark Johnston  ---
I think this indicates a bug in the caller rather than UMA. The nomenclature is
a bit confusing: a sleep mutex is just a "default" mutex, i.e. a non-spin
mutex. When a thread blocks on a sleep mutex, it enters bounded sleep; "sleep"
in the free(9) man page refers to unbounded sleep.

The assertion is failing because the thread holds a spin mutex or a critical
section, in which case it is not valid to try and acquire a sleep mutex. It
could probably be triggered in a non-INVARIANTS kernel too, since
uma_zfree_arg() will attempt to acquire the corresponding zone lock, which is
also a sleep mutex.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-bugs@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"

[Bug 204633] If INVARIANTS is enabled, free() may attempt to acquire sleeping lock

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204633

Jonathan T. Looney  changed:

   What|Removed |Added

   Assignee|freebsd-bugs@FreeBSD.org|j...@freebsd.org

--- Comment #5 from Jonathan T. Looney  ---
I have a proposed change out for review. See https://reviews.freebsd.org/D4197.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-bugs@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"

[Bug 204633] If INVARIANTS is enabled, free() may attempt to acquire sleeping lock

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204633

--- Comment #3 from Mark Johnston  ---
(In reply to Jonathan T. Looney from comment #2)
I think it would be correct and reasonable to assert that
curthread->td_critnest == 0 at the beginning of uma_zalloc_arg() and
uma_zfree_arg(). In most cases, UMA can satisfy allocation and free requests
without acquiring a lock, so the checking currently done by witness is not
consistent.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-bugs@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"

[Bug 204633] If INVARIANTS is enabled, free() may attempt to acquire sleeping lock

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204633

--- Comment #2 from Jonathan T. Looney  ---
(In reply to Mark Johnston from comment #1)
> The assertion is failing because the thread holds a spin mutex or a critical 
> section, 
> in which case it is not valid to try and acquire a sleep mutex. It could 
> probably be 
> triggered in a non-INVARIANTS kernel too, since uma_zfree_arg() will attempt 
> to 
> acquire the corresponding zone lock, which is also a sleep mutex.

Good point. That makes this very much look like intended behavior.

I only saw this when I tried testing my changes with an unusually high load.
Presumably, that caused the allocator to need to acquire the zone lock when it
would normally not need to do so. I wonder how many other things have slipped
through without enough testing to actually trigger the assert?

Perhaps, I should propose a man page change to make this more clear. And/or add
a check in malloc/free to catch these problems more reliably.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-bugs@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"

[Bug 204633] If INVARIANTS is enabled, free() may attempt to acquire sleeping lock

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204633

--- Comment #4 from Ed Maste  ---
> Perhaps, I should propose a man page change to make this more clear.
> And/or add a check in malloc/free to catch these problems more reliably.

Indeed, it would be useful to have this clarified in the man page, and an
explicit check under INVARIANTS in malloc/free would be sensible.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-bugs@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"