During grace period initialization, when rcu_gp_init() reports QS for offline CPUs, any tasks blocked on those CPUs' per-CPU blocked lists must first be promoted to the rcu_node's blkd_tasks list.
Without this promotion, blocked tasks on offline CPUs' per-CPU lists won't have gp_tasks point to them, so the GP machinery won't wait for them. This can cause "Wrong-GP reads" errors where a GP completes while readers are still in their critical sections. Therefore, call call rcu_promote_blocked_tasks_rdp() for each offline CPU before reporting QS for them. Signed-off-by: Joel Fernandes <[email protected]> --- kernel/rcu/tree.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 5e73ebb260e3..468388970c98 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2001,8 +2001,21 @@ static noinline_for_stack bool rcu_gp_init(void) */ mask = rnp->qsmask & ~rnp->qsmaskinitnext; rnp->rcu_gp_init_mask = mask; - if ((mask || rnp->wait_blkd_tasks) && rcu_is_leaf_node(rnp)) + if ((mask || rnp->wait_blkd_tasks) && rcu_is_leaf_node(rnp)) { + int cpu; + + /* + * Promote blocked tasks from offline CPUs before + * reporting QS, so they properly block the GP. + */ + for_each_leaf_node_cpu_mask(rnp, cpu, mask) { + struct rcu_data *rdp_cpu; + + rdp_cpu = per_cpu_ptr(&rcu_data, cpu); + rcu_promote_blocked_tasks_rdp(rdp_cpu, rnp); + } rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags); + } else raw_spin_unlock_irq_rcu_node(rnp); cond_resched_tasks_rcu_qs(); -- 2.34.1
