On 2017-04-06 10:01:39 [+0200], Ingo Molnar wrote:
>
> * Sebastian Andrzej Siewior <[email protected]> wrote:
>
> > On 2017-04-06 08:16:22 [+0200], Ingo Molnar wrote:
> > >
> > > * Sebastian Andrzej Siewior <[email protected]> wrote:
> > >
> > > > On 2017-04-05 09:39:43 [+0200], Ingo Molnar wrote:
> > > > >
> > > > > So maybe we could add the following facility:
> > > > >
> > > > > ptr = sched_migrate_to_cpu_save(cpu);
> > > > >
> > > > > ...
> > > > >
> > > > > sched_migrate_to_cpu_restore(ptr);
> > >
> > > BTW., and I'm sure this has come up before, but why doesn't
> > > migrate_disable() use
> > > a simple per task flag that the scheduler migration code takes into
> > > account?
> >
> > we could add that. But right now there are two spots which look at the
> > counter to decide whether or not migration is disabled.
> >
> > > It should be functionally equivalent to the current solution, and it
> > > appears to
> > > have a heck of a smaller cross section with the rest of the scheduler.
> > >
> > > I.e.:
> > >
> > > static inline void migrate_disable(void)
> > > {
> > > current->migration_disabled++;
> > > }
> > >
> > > ...
> > >
> > > static inline void migrate_enable(void)
> > > {
> > > current->migration_disabled--;
> > > }
> > >
> > > or so? Then add this flag as a condition to can_migrate_task() et al.
> > >
> > > While we generally dislike such flags as they wreck havoc with the
> > > scheduler if
> > > overused, the cpus_allowed based solution has the exact same effect so
> > > it's not
> > > like it's a step backwards - and it should also be much faster and less
> > > intrusive.
> >
> > So you are saying that we drop the cpus_ptr + cpus_mask fields again and
> > instead add a task-flag to ensure that the scheduler does not migrate
> > the task to another CPU?
>
> Yeah - but no need to add a per-task flag if we already have a counter.
>
> > > Am I missing some complication?
> >
> > We do have the counter. We have need to ensure that the CPU is not going
> > away
> > while we are in a migrate_disable() region since we can be scheduled out.
> > So the
> > CPU can't go offline until we leave that region.
>
> Yeah. But it should be relatively straightforward to extend the logic that
> makes
> sure that a CPU does not go away from under tasks pinned to that CPU alone,
> right?
I used get_online_cpus() that is enough.
> > #define migrate_disable() sched_migrate_to_cpu_save(-1)
> >
> > int sched_migrate_to_cpu_save(int cpu)
>
> So if we have a ->migration_disabled counter then we don't need the
> sched_migrate_to_cpu_save()/restore() complication, right?
correct. Unless (as you suggested) we want a migrate to specific CPU we
just the function until the BUG() statement and don't need to check the
CPU we are on.
> Sorry if this is a back and forth - I was somehow convinced that we do need
> to
> frob the cpus_allowed mask to get this functionality - but in hindsight I
> think
> the counter should be enough.
>
> I.e. just have a counter and these two APIs:
>
> static inline void migrate_disable(void)
> {
> current->migration_disabled++;
plus
if (current->migration_disabled == 1)
get_online_cpus()
> }
>
> ...
>
> static inline void migrate_enable(void)
> {
> current->migration_disabled--;
> }
>
> ... and make sure the scheduler migration code plus the CPU hotplug code
> considers
> the counter.
So on the sched part I need go through all places where it looks at the
mask and make sure it ignores the CPU switch decision. This could be
doable.
On the CPU hotplug I don't see a way around get_online_cpus(). It is
essentially there to ensure a CPU does not go away. The other way around
it would be to remove the CPU from the online_mask followed by going
through all task left on the CPU waiting for them leave the CPU. And I
remember peterz saying that he wanted to make get_online_cpus() fast and
stay with that.
> Would this work, and would this be the simplest all around solution?
Let me try this with get_online_cpus() (as suggested) and without
cpus_ptr.
> Thanks,
>
> Ingo
Sebastian
