Hi, On 04/07/18 15:24, Matt Fleming wrote: > It's possible that the CPU doing nohz idle balance hasn't had its own > load updated for many seconds. This can lead to huge deltas between > rq->avg_stamp and rq->clock when rebalancing, and has been seen to > cause the following crash: > > divide error: 0000 [#1] SMP > Call Trace: > [<ffffffff810bcba8>] update_sd_lb_stats+0xe8/0x560 > [<ffffffff810bd04d>] find_busiest_group+0x2d/0x4b0 > [<ffffffff810bd640>] load_balance+0x170/0x950 > [<ffffffff810be3ff>] rebalance_domains+0x13f/0x290 > [<ffffffff810852bc>] __do_softirq+0xec/0x300 > [<ffffffff8108578a>] irq_exit+0xfa/0x110 > [<ffffffff816167d9>] reschedule_interrupt+0xc9/0xd0 >
Do you have some sort of reproducer for that crash? If not I guess I can cook something up with a quiet userspace & rt-app, though I've never seen that one on arm64. > Make sure we update the rq clock and load before balancing. > > Cc: Ingo Molnar <[email protected]> > Cc: Mike Galbraith <[email protected]> > Cc: Peter Zijlstra <[email protected]> > Signed-off-by: Matt Fleming <[email protected]> > --- > kernel/sched/fair.c | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 2f0a0be4d344..2c81662c858a 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -9597,6 +9597,16 @@ static bool _nohz_idle_balance(struct rq *this_rq, > unsigned int flags, > */ > smp_mb(); > > + /* > + * Ensure this_rq's clock and load are up-to-date before we > + * rebalance since it's possible that they haven't been > + * updated for multiple schedule periods, i.e. many seconds. > + */ > + raw_spin_lock_irq(&this_rq->lock); > + update_rq_clock(this_rq); > + cpu_load_update_idle(this_rq); > + raw_spin_unlock_irq(&this_rq->lock); > + I'm failing to understand why the updates further down below are seemingly not enough. After we've potentially done update_rq_clock(rq); cpu_load_update_idle(rq); for all nohz cpus != this_cpu, we still end up doing: if (idle != CPU_NEWLY_IDLE) { update_blocked_averages(this_cpu); has_blocked_load |= this_rq->has_blocked_load; } which should properly update this_rq's clock and load before we attempt to do any balancing on it. > for_each_cpu(balance_cpu, nohz.idle_cpus_mask) { > if (balance_cpu == this_cpu || !idle_cpu(balance_cpu)) > continue; >
