From: Kunwu Chan <[email protected]>
srcu_read_lock_atomic() and srcu_read_unlock_atomic() arm and disarm
might_sleep() checks via non_block_start()/non_block_end(), skipping
hardirq so the update does not land on the interrupted task's
->non_block_count.
Inline softirqs run on the interrupted task's stack as well, so a
timer callback running atomic-SRCU readers races with the interrupted
task's own ->non_block_count updates, as KCSAN reports:
BUG: KCSAN: data-race in srcu_torture_read_lock /
srcu_torture_read_unlock
write to 0xffffa00f818ea418 of 4 bytes by interrupt on cpu 0:
srcu_torture_read_lock+0x422/0x470
rcutorture_one_extend+0xdc/0x600
rcu_torture_one_read+0xd1/0x330
rcu_torture_timer+0x75/0x140
call_timer_fn+0xe6/0x2f0
...
run_timer_softirq+0xb7/0x130
handle_softirqs+0xfc/0x3f0
__irq_exit_rcu+0x8e/0x100
Use in_task() so the annotation is applied only in task context; it
is redundant elsewhere because might_sleep() already warns about
sleeping from atomic context.
Signed-off-by: Kunwu Chan <[email protected]>
---
include/linux/srcu.h | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/include/linux/srcu.h b/include/linux/srcu.h
index 0de21155abc1..403fbe57ab15 100644
--- a/include/linux/srcu.h
+++ b/include/linux/srcu.h
@@ -346,11 +346,13 @@ static inline int srcu_read_lock_atomic(struct
srcu_struct *ssp)
/*
* Arm might_sleep() to catch even a *potentially* sleeping call
* in the section, not just an actual schedule: the atomic-domain
- * promise must hold on every path, contended or not. In hardirq
- * the annotation would land on the interrupted task; it is also
+ * promise must hold on every path, contended or not. In hardirq,
+ * softirq, or NMI the annotation would land on the interrupted
+ * task, and can also result in data races against that task's
+ * own non_block_start()/non_block_end() invocations; it is also
* redundant there, so skip it.
*/
- if (!in_hardirq())
+ if (in_task())
non_block_start();
srcu_check_read_flavor(ssp, SRCU_READ_FLAVOR_ATOMIC);
retval = __srcu_read_lock(ssp);
@@ -562,7 +564,7 @@ static inline void srcu_read_unlock_atomic(struct
srcu_struct *ssp, int idx)
srcu_check_read_flavor(ssp, SRCU_READ_FLAVOR_ATOMIC);
srcu_lock_release(&ssp->dep_map);
__srcu_read_unlock(ssp, idx);
- if (!in_hardirq())
+ if (in_task())
non_block_end();
preempt_enable();
}
--
2.43.0
