Add a debug option that makes hazptr_acquire() always enter __hazptr_acquire(), even when a per-CPU slot is available.
This allows the slow path, including its lock acquisition, to be exercised by lockdep-heavy workloads. It also lets hazptrtorture exercise the slow path without requiring the per-CPU slots to be exhausted. Suggested-by: Boqun Feng <[email protected]> Signed-off-by: Kunwu Chan <[email protected]> --- include/linux/hazptr.h | 3 ++- kernel/rcu/Kconfig.debug | 10 ++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/linux/hazptr.h b/include/linux/hazptr.h index 275cbe54b075..4ee02d23abc9 100644 --- a/include/linux/hazptr.h +++ b/include/linux/hazptr.h @@ -245,7 +245,8 @@ void *hazptr_acquire(struct hazptr_ctx *ctx, void * const *addr_p) ctx->acquire_cpu = smp_processor_id(); ctx->acquire_caller = _THIS_IP_; #endif - if (unlikely(slot->addr)) + if (IS_ENABLED(CONFIG_HAZPTR_ACQUIRE_FORCE_SLOWPATH) || + unlikely(slot->addr)) return __hazptr_acquire(ctx, addr_p); WRITE_ONCE(slot->addr, READ_ONCE(hazptr_wildcard)); /* Store B */ diff --git a/kernel/rcu/Kconfig.debug b/kernel/rcu/Kconfig.debug index 7c3c6017b266..3a2e9d7f58f8 100644 --- a/kernel/rcu/Kconfig.debug +++ b/kernel/rcu/Kconfig.debug @@ -265,4 +265,14 @@ config HAZPTR_DEBUG Say Y here if you want to enable those assert, N otherwise. +config HAZPTR_ACQUIRE_FORCE_SLOWPATH + bool "Force hazard-pointer acquire into the slow path" + depends on DEBUG_KERNEL + default n + help + Make hazptr_acquire() always enter __hazptr_acquire(). + This is useful for testing slow-path locking, especially + with lockdep. It also lets hazptrtorture exercise the slow + path without filling the per-CPU slots. + endmenu # "RCU Debugging" -- 2.43.0
