Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2021-02-16 Thread David Woodhouse
On Tue, 2021-02-16 at 15:10 +, David Woodhouse wrote: > Actually it breaks before that, in rcu_cpu_starting(). A spinlock > around that, an atomic_t to let the APs do their TSC sync one at a time > (both in the above tree now), and I have a 75% saving on CPU bringup > time for my 28-thread

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2021-02-16 Thread David Woodhouse
On Tue, 2021-02-16 at 13:53 +, David Woodhouse wrote: > I threw it into my tree at > https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/parallel > > It seems to work fairly nicely. The parallel SIPI seems to win be about > a third of the bringup time on my 28-thread Haswell

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2021-02-16 Thread David Woodhouse
On Fri, 2021-02-12 at 18:30 +0100, Thomas Gleixner wrote: > On Fri, Feb 12 2021 at 01:29, Thomas Gleixner wrote: > > On Thu, Feb 11 2021 at 22:58, David Woodhouse wrote: > > I have no problem with making that jump based. It does not matter at > > all. But you can't use the idle task stack before

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2021-02-01 Thread David Woodhouse
On Thu, 2021-01-21 at 15:55 +0100, Thomas Gleixner wrote: > > Here's the hack we're testing with, for reference. It's kind of ugly > > but you can see where it's going. Note that the CMOS mangling for the > > warm reset vector is going to need to be lifted out of the per-cpu > > loop, and done

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2021-01-21 Thread David Woodhouse
On Thu, 2021-01-21 at 15:42 +, David Woodhouse wrote: > [2.289283] BUG: kernel NULL pointer dereference, address: > [2.289283] #PF: supervisor write access in kernel mode > [2.289283] #PF: error_code(0x0002) - not-present page > [2.289283] PGD 0 P4D 0 > [

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2021-01-21 Thread David Woodhouse
On Thu, 2021-01-21 at 15:55 +0100, Thomas Gleixner wrote: > > Testing on real hardware has been more interesting and less useful so > > far. We started with the CPUHP_BRINGUP_KICK_CPU state being > > *immediately* before CPUHP_BRINGUP_CPU. On my 28-thread Haswell box, > > that didn't come up at

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2021-01-21 Thread Thomas Gleixner
David, On Tue, Jan 19 2021 at 12:12, David Woodhouse wrote: > On Tue, 2020-12-15 at 22:20 +0100, Thomas Gleixner wrote: > We've been playing with this a little. There's a proof-of-concept hack > below; don't look too hard because it's only really for figuring out > the timing etc. > > Basically

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2021-01-19 Thread David Woodhouse
On Tue, 2020-12-15 at 22:20 +0100, Thomas Gleixner wrote: > Since the rewrite of the CPU hotplug infrastructure to a state machine > it's pretty obvious that the bringup of APs can changed from the fully > serialized: > > for_each_present_cpu(cpu) { > if (!cpu_online(cpu)) >

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2021-01-07 Thread David Woodhouse
On Wed, 2020-12-16 at 16:31 +0100, Thomas Gleixner wrote: > But obviously the C-state in which the APs are waiting is not really > relevant, as you demonstrated that the cost is due to INIT/SIPI even > with spinwait, which is what I suspected. > > OTOH, the advantage of INIT/SIPI is that the AP

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2020-12-17 Thread shenkai (D)
在 2020/12/16 23:31, Thomas Gleixner 写道: OTOH, the advantage of INIT/SIPI is that the AP comes up in a well known state. We can set APs to a known state explicitly like BSP will do in kexec case (what we also tried to do in the patch). Maybe it is not a big problem? Best regards Kai

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2020-12-16 Thread Thomas Gleixner
Kai, On Wed, Dec 16 2020 at 22:18, shenkai wrote: > After some tests, the conclusion that time cost is from deep C-state > turns out to be wrong > > Sorry for that. No problem. > In kexec case, first let APs spinwait like what I did  in that patch, > but wake APs up by sending apic INIT and

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2020-12-16 Thread shenkai (D)
在 2020/12/16 18:12, Thomas Gleixner 写道: Kai, On Wed, Dec 16 2020 at 16:45, shenkai wrote: 在 2020/12/16 5:20, Thomas Gleixner 写道: Thanks for your and Andy's precious comments. I would like to take a try on reconstructing this patch to make it more decent and generic. It would be

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2020-12-16 Thread Thomas Gleixner
Kai, On Wed, Dec 16 2020 at 16:45, shenkai wrote: > 在 2020/12/16 5:20, Thomas Gleixner 写道: >> >> > Thanks for your and Andy's precious comments. I would like to take a try on > > reconstructing this patch to make it more decent and generic. >> It would be interesting to see the numbers just with

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2020-12-16 Thread shenkai (D)
在 2020/12/16 5:20, Thomas Gleixner 写道: On Tue, Dec 15 2020 at 08:31, Andy Lutomirski wrote: On Tue, Dec 15, 2020 at 6:46 AM shenkai (D) wrote: From: shenkai Date: Tue, 15 Dec 2020 01:58:06 + Subject: [PATCH] use x86 cpu park to speedup smp_init in kexec situation In kexec reboot on x86

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2020-12-15 Thread Thomas Gleixner
On Tue, Dec 15 2020 at 08:31, Andy Lutomirski wrote: > On Tue, Dec 15, 2020 at 6:46 AM shenkai (D) wrote: >> >> From: shenkai >> Date: Tue, 15 Dec 2020 01:58:06 + >> Subject: [PATCH] use x86 cpu park to speedup smp_init in kexec situation >> >> In kexec reboot on x86 machine, APs will be

Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation

2020-12-15 Thread Andy Lutomirski
On Tue, Dec 15, 2020 at 6:46 AM shenkai (D) wrote: > > From: shenkai > Date: Tue, 15 Dec 2020 01:58:06 + > Subject: [PATCH] use x86 cpu park to speedup smp_init in kexec situation > > In kexec reboot on x86 machine, APs will be halted and then waked up > by the apic INIT and SIPI interrupt.