On Mon, Nov 16, 2020 at 03:29:46PM +0000, Mel Gorman wrote:
> On Mon, Nov 16, 2020 at 01:58:03PM +0100, Peter Zijlstra wrote:
> > > sched_ttwu_pending()
> > > if (WARN_ON_ONCE(p->on_cpu))
> > > smp_cond_load_acquire(&p->on_cpu)
> > >
> > > ttwu_do_activate()
> > > if (p->sched_contributes_to_load)
> > > ...
> > >
> > > on the other (for the remote case, which is the only 'interesting' one).
> >
>
> But this side is interesting because I'm having trouble convincing
> myself it's 100% correct for sched_contributes_to_load. The write of
> prev->sched_contributes_to_load in the schedule() path has a big gap
> before it hits the smp_store_release(prev->on_cpu).
>
> On the ttwu path, we have
>
> if (smp_load_acquire(&p->on_cpu) &&
> ttwu_queue_wakelist(p, task_cpu(p), wake_flags | WF_ON_CPU))
> goto unlock;
>
> ttwu_queue_wakelist queues task on the wakelist, sends IPI
> and on the receiver side it calls ttwu_do_activate and reads
> sched_contributes_to_load
>
> sched_ttwu_pending() is not necessarily using the same rq lock so no
> protection there. The smp_load_acquire() has just been hit but it still
> leaves a gap between when sched_contributes_to_load is written and a
> parallel read of sched_contributes_to_load.
>
> So while we might be able to avoid a smp_rmb() before the read of
> sched_contributes_to_load and rely on p->on_cpu ordering there,
> we may still need a smp_wmb() after nr_interruptible() increments
> instead of waiting until the smp_store_release() is hit while a task
> is scheduling. That would be a real memory barrier on arm64 and a plain
> compiler barrier on x86-64.
I'm mighty confused by your words here; and the patch below. What actual
scenario are you worried about?
If we take the WF_ON_CPU path, we IPI the CPU the task is ->on_cpu on.
So the IPI lands after the schedule() that clears ->on_cpu on the very
same CPU.
>
> > Also see the "Notes on Program-Order guarantees on SMP systems."
> > comment.
>
> I did, it was the on_cpu ordering for the blocking case that had me
> looking at the smp_store_release and smp_cond_load_acquire in arm64 in
> the first place thinking that something in there must be breaking the
> on_cpu ordering. I'm re-reading it every so often while trying to figure
> out where the gap is or whether I'm imagining things.
>
> Not fully tested but did not instantly break either
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index d2003a7d5ab5..877eaeba45ac 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -4459,14 +4459,26 @@ static void __sched notrace __schedule(bool preempt)
> if (signal_pending_state(prev_state, prev)) {
> prev->state = TASK_RUNNING;
> } else {
> - prev->sched_contributes_to_load =
> + int acct_load =
> (prev_state & TASK_UNINTERRUPTIBLE) &&
> !(prev_state & TASK_NOLOAD) &&
> !(prev->flags & PF_FROZEN);
>
> - if (prev->sched_contributes_to_load)
> + prev->sched_contributes_to_load = acct_load;
> + if (acct_load) {
> rq->nr_uninterruptible++;
>
> + /*
> + * Pairs with p->on_cpu ordering, either a
> + * smp_load_acquire or smp_cond_load_acquire
> + * in the ttwu path before ttwu_do_activate
> + * p->sched_contributes_to_load. It's only
> + * after the nr_interruptible update happens
> + * that the ordering is critical.
> + */
> + smp_wmb();
> + }
Sorry, I can't follow, at all.
