On Tue, Apr 16, 2013 at 01:32:56PM +0200, Ingo Molnar wrote:
>
> * Robin Holt <h...@sgi.com> wrote:
>
> > We recently noticed that reboot of a 1024 cpu machine takes approx 16
> > minutes of just stopping the cpus. The slowdown was tracked to commit
> > f96972f.
> >
> > The current implementation does all the work of hot removing the cpus
> > before halting the system. We are switching to just migrating to the
> > boot cpu and then continuing with shutdown/reboot.
> >
> > This also has the effect of not breaking x86's command line parameter for
> > specifying the reboot cpu. Note, this code was shamelessly copied from
> > arch/x86/kernel/reboot.c with bits removed pertaining to the reboot_cpu
> > command line parameter.
> >
> > Signed-off-by: Robin Holt <h...@sgi.com>
> > Tested-by: Shawn Guo <shawn....@linaro.org>
> > To: Ingo Molnar <mi...@redhat.com>
> > To: Russ Anderson <r...@sgi.com>
> > To: Oleg Nesterov <o...@redhat.com>
> > Cc: Andrew Morton <a...@linux-foundation.org>
> > Cc: "H. Peter Anvin" <h...@zytor.com>
> > Cc: Lai Jiangshan <la...@cn.fujitsu.com>
> > Cc: Linus Torvalds <torva...@linux-foundation.org>
> > Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
> > Cc: Michel Lespinasse <wal...@google.com>
> > Cc: Oleg Nesterov <o...@redhat.com>
> > Cc: "Paul E. McKenney" <paul...@linux.vnet.ibm.com>
> > Cc: Paul Mackerras <pau...@samba.org>
> > Cc: Peter Zijlstra <pet...@infradead.org>
> > Cc: Robin Holt <h...@sgi.com>
> > Cc: "ru...@rustcorp.com.au" <ru...@rustcorp.com.au>
> > Cc: Tejun Heo <t...@kernel.org>
> > Cc: the arch/x86 maintainers <x...@kernel.org>
> > Cc: Thomas Gleixner <t...@linutronix.de>
> > Cc: <sta...@vger.kernel.org>
> >
> > ---
> >
> > Changes since -v1.
> > - Set PF_THREAD_BOUND before migrating to eliminate potential race.
> > - Modified kernel_power_off to also migrate instead of using
> > disable_nonboot_cpus().
> > ---
> > kernel/sys.c | 22 +++++++++++++++++++---
> > 1 file changed, 19 insertions(+), 3 deletions(-)
> >
> > diff --git a/kernel/sys.c b/kernel/sys.c
> > index 0da73cf..5ef7aa2 100644
> > --- a/kernel/sys.c
> > +++ b/kernel/sys.c
> > @@ -357,6 +357,22 @@ int unregister_reboot_notifier(struct notifier_block
> > *nb)
> > }
> > EXPORT_SYMBOL(unregister_reboot_notifier);
> >
> > +void migrate_to_reboot_cpu(void)
>
> It appears to be file-scope, so should be static I guess?
Done.
> > +{
> > + /* The boot cpu is always logical cpu 0 */
> > + int reboot_cpu_id = 0;
> > +
> > + /* Make certain the cpu I'm about to reboot on is online */
> > + if (!cpu_online(reboot_cpu_id))
> > + reboot_cpu_id = smp_processor_id();
>
> Shouldn't we pick the first online CPU instead, to make it deterministic?
Done.
reboot_cpu_id = cpumask_first(cpu_online_mask);
> Also, does this codepath prevent hotplug from going on in parallel?
Not sure. I have not considered hotplug. I will look that over when I
am in the office.
> ( Plus, the smp_processor_id() is in a preemptible section AFAICS, so it will
> throw a warning with preempt debug on. )
>
> > +
> > + /* Prevent races with other tasks migrating this task. */
>
> (I guess the colon can be dropped here, like in the other comments.)
Done.
>
> > + current->flags |= PF_THREAD_BOUND;
> > +
> > + /* Make certain I only run on the appropriate processor */
> > + set_cpus_allowed_ptr(current, cpumask_of(reboot_cpu_id));
> > +}
I will resubmit when I have the hotplug stuff understood and after giving
the set some more time for comments.
Robin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
