On Tue, Apr 20, 2021 at 04:39:04PM +0200, Peter Zijlstra wrote: > On Tue, Apr 20, 2021 at 04:20:56PM +0200, Peter Zijlstra wrote: > > On Tue, Apr 20, 2021 at 10:46:33AM +0100, Vincent Donnefort wrote: > > > > > Found the issue: > > > > > > $ cat hotplug/states: > > > 219: sched:active > > > 220: online > > > > > > CPU0: > > > > > > $ echo 219 > hotplug/fail > > > $ echo 0 > online > > > > > > => cpu_active = 1 cpu_dying = 1 > > > > > > which means that later on, for another CPU hotunplug, in > > > __balance_push_cpu_stop(), the fallback rq for a kthread can select that > > > CPU0, but __migrate_task() would fail and we end-up in an infinite loop, > > > trying to migrate that task to CPU0. > > > > > > The problem is that for a failure in sched:active, as "online" has no > > > callback, > > > there will be no call to cpuhp_invoke_callback(). Hence, the cpu_dying > > > bit would > > > not be reset. > > > > Urgh! Good find.
> I seem to have triggered the BUG() in select_fallback_rq() with your recipie. > Have cpu0 fail on sched:active, then offline all other CPUs. > > Now lemme add that patch. (which obviously didn't actually build) seems to fix it. --- diff --git a/kernel/cpu.c b/kernel/cpu.c index 838dcf238f92..e538518556f4 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -63,6 +63,7 @@ struct cpuhp_cpu_state { bool rollback; bool single; bool bringup; + int cpu; struct hlist_node *node; struct hlist_node *last; enum cpuhp_state cb_state; @@ -160,9 +161,6 @@ static int cpuhp_invoke_callback(unsigned int cpu, enum cpuhp_state state, int (*cb)(unsigned int cpu); int ret, cnt; - if (cpu_dying(cpu) != !bringup) - set_cpu_dying(cpu, !bringup); - if (st->fail == state) { st->fail = CPUHP_INVALID; return -EAGAIN; @@ -467,13 +465,16 @@ static inline enum cpuhp_state cpuhp_set_state(struct cpuhp_cpu_state *st, enum cpuhp_state target) { enum cpuhp_state prev_state = st->state; + bool bringup = st->state < target; st->rollback = false; st->last = NULL; st->target = target; st->single = false; - st->bringup = st->state < target; + st->bringup = bringup; + if (cpu_dying(st->cpu) != !bringup) + set_cpu_dying(st->cpu, !bringup); return prev_state; } @@ -481,6 +482,8 @@ cpuhp_set_state(struct cpuhp_cpu_state *st, enum cpuhp_state target) static inline void cpuhp_reset_state(struct cpuhp_cpu_state *st, enum cpuhp_state prev_state) { + bool bringup = !st->bringup; + st->target = prev_state; /* @@ -503,7 +506,9 @@ cpuhp_reset_state(struct cpuhp_cpu_state *st, enum cpuhp_state prev_state) st->state++; } - st->bringup = !st->bringup; + st->bringup = bringup; + if (cpu_dying(st->cpu) != !bringup) + set_cpu_dying(st->cpu, !bringup); } /* Regular hotplug invocation of the AP hotplug thread */ @@ -693,6 +698,7 @@ static void cpuhp_create(unsigned int cpu) init_completion(&st->done_up); init_completion(&st->done_down); + st->cpu = cpu; } static int cpuhp_should_run(unsigned int cpu)