On 2 October 2026 18:08:32 BST, Kunwu Chan <[email protected]> wrote: >This series extends Paul McKenney's v3 hazptr implementation [1] >with a shared scan path for concurrent hazptr_synchronize() callers, >and adapts the lockdep dynamic-key hashlist use case from Boqun >Feng's 2025 shazptr series [2] to the current hazptr API. > >The series also adds rcuscale support and torture coverage for >the hazptr implementation. > >The lockdep conversion replaces the expedited RCU wait in >lockdep_unregister_key() with hazptr_synchronize(). This limits >the wait to hazard pointers protecting the target hash bucket >instead of waiting for a system-wide expedited RCU grace period. > >[1] >https://lore.kernel.org/all/[email protected]/ >[2] >https://lore.kernel.org/lkml/[email protected]/ >
This is well needed. Thanks for the patch, I'll maybe review maybe test if I have time, cough cough, school! >Performance data >================ > >All measurements are on an ARM64 KVM guest with >HAZPTR_SCALE_NR_OBJS=8 and round-robin updater selection; unless >noted otherwise, nreaders=0. Scale tests run with >PROVE_LOCKING=n. > >rcuscale synchronize latency (96 CPUs, nw=1): > > nreaders=0 nreaders=1 nreaders=4 nreaders=96 > scale avg avg avg avg > -------------------------------------------------------------------- > hazptr 46 us 47 us 102 us 106 ms > RCU 8.3 ms 8.5 ms 16.6 ms 54.4 ms > SRCU 8.2 ms 8.0 ms 11.7 ms 11.8 ms > >Hazptr single-writer latency remains below 110 us with up to >four readers on this 96-CPU guest, but rises to 106 ms when >all 96 CPUs hold hazard pointers. > >Under 16 concurrent synchronize callers (nw=16), per-writer >latency at four CPU counts: > > hazptr nw=16 RCU nw=16 SRCU nw=16 > CPUs avg avg avg > ----------------------------------------------------------- > 24 8.0 ms 13.6 ms 9.2 ms > 96 8.0 ms 13.3 ms 8.6 ms > 128 7.9 ms 13.0 ms 11.7 ms > 256 8.0 ms 20.8 ms 9.6 ms > >Hazptr remains around 8.0 ms across these CPU counts, with the >scan-kthread retry interval contributing to the latency. >RCU rises to ~21 ms at 256 CPUs while SRCU stays around 9-12 ms. > >Reader-side overhead (refscale, 96 readers on the 96-CPU guest, >3 runs each): > > hazptr 25.7 ns/op > RCU 96.7 ns/op > SRCU 134.6 ns/op > >lockdep workload -- tc qdisc mq x100, ARM64 KVM, PROVE_LOCKING=y, >96 background hazptr readers, function-call IPIs per 100 ops: > > CPUs hazptr exp RCU reduction > ------------------------------------------------------------ > 24 196 207 5% > 96 189 265 29% > 128 193 266 27% > 256 222 384 42% > >Hazptr issues fewer function-call IPIs at each CPU count, with >the difference reaching 42% at 256 CPUs. > >The hazptr.sh test suite passes, including the lockdep scenarios >and the 8-to-256 CPU sweep, with no lockdep warnings, deadlocks, >or crashes. >Additional x86 server testing with Lian Wang is planned(maybe after >LPC). > >Changes since RFC/WIP >===================== > >- RFC/WIP: >https://lore.kernel.org/all/[email protected]/ > >- Split the original 4-patch RFC/WIP into smaller commits covering > shared scanning, correctness, API support, lockdep, scaling, and > torture testing. > >- Incorporated Boqun Feng's review feedback: use a Bloom filter to > avoid per-waiter allocation, add scoped_guard() support, and add > a debug option to force the hazptr acquire slow path. > >- Fixed scan ordering around backup-slot promotion by scanning all > per-CPU slots before the overflow lists, with a separate > overflow-list phase. > >- Simplified the scan cycle to flip first and drain only the old > wildcard generation, with herd7-verified LKMM tests for both the > in-flight and resolved publication cases. > >- Extended rcuscale and hazptrtorture coverage, added a selftest > script for the torture configurations, and fixed the > hazptr_release() kernel-doc. > >Kunwu Chan (15): > hazptr: add shared scan kthread > hazptr: use Bloom filter for shared scan waiters > hazptr: scan all per-CPU slots before overflow lists > hazptr: add scoped_guard() support > hazptr: add debug option to force the acquire slow path > hazptr: elide redundant first drain pass > Documentation/litmus-tests: add hazptr wildcard-flip escape test > locking/lockdep: use hazptr to wait for dynamic key lookups > rcuscale: add hazptr scale type > hazptr: fix kernel-doc of hazptr_release() > Documentation/litmus-tests: add hazptr acquire-before-scan test > hazptrtorture: add slowpath and lockdep scenarios > hazptrtorture: add READERS4 and READERS0 torture configs > hazptrtorture: add 128- and 256-CPU configs > selftests/rcutorture: add hazptr torture test script > > Documentation/litmus-tests/README | 13 + > .../hazptr/hazptr-acquire-before-scan.litmus | 45 +++ > .../hazptr/hazptr-wildcard-flip-escape.litmus | 46 +++ > include/linux/hazptr.h | 56 ++- > kernel/hazptr.c | 336 +++++++++++++++++- > kernel/locking/lockdep.c | 25 +- > kernel/rcu/Kconfig.debug | 10 + > kernel/rcu/hazptrtorture.c | 57 ++- > kernel/rcu/rcuscale.c | 70 +++- > .../selftests/rcutorture/bin/hazptr.sh | 146 ++++++++ > .../rcutorture/configs/hazptr/CFLIST | 6 + > .../rcutorture/configs/hazptr/CPU128 | 16 + > .../rcutorture/configs/hazptr/CPU128.boot | 1 + > .../rcutorture/configs/hazptr/CPU256 | 16 + > .../rcutorture/configs/hazptr/CPU256.boot | 1 + > .../rcutorture/configs/hazptr/LOCKDEP | 17 + > .../rcutorture/configs/hazptr/LOCKDEP.boot | 1 + > .../rcutorture/configs/hazptr/READERS0 | 16 + > .../rcutorture/configs/hazptr/READERS0.boot | 2 + > .../rcutorture/configs/hazptr/READERS4 | 16 + > .../rcutorture/configs/hazptr/READERS4.boot | 2 + > .../rcutorture/configs/hazptr/SLOWPATH | 16 + > 22 files changed, 881 insertions(+), 33 deletions(-) > create mode 100644 > Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus > create mode 100644 > Documentation/litmus-tests/hazptr/hazptr-wildcard-flip-escape.litmus > create mode 100755 tools/testing/selftests/rcutorture/bin/hazptr.sh > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/CPU128 > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/CPU128.boot > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/CPU256 > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/CPU256.boot > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/LOCKDEP > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/LOCKDEP.boot > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/READERS0 > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/READERS0.boot > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/READERS4 > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/READERS4.boot > create mode 100644 > tools/testing/selftests/rcutorture/configs/hazptr/SLOWPATH > > --- Thanks! "I'm not a very positive person" - Linus torvalds
