On Thu, 26 Nov 2020 17:23:20 +0100
Igor Mammedov <imamm...@redhat.com> wrote:
> On Thu, 26 Nov 2020 10:10:27 +0100
> Greg Kurz <gr...@kaod.org> wrote:
>
> > On Thu, 26 Nov 2020 15:57:37 +1100
> > David Gibson <da...@gibson.dropbear.id.au> wrote:
> >
> > > On Wed, Nov 25, 2020 at 10:51:05AM +0100, Greg Kurz wrote:
> > > > On Wed, 25 Nov 2020 13:39:47 +1100
> > > > David Gibson <da...@gibson.dropbear.id.au> wrote:
> > > >
> > > > > On Mon, Nov 23, 2020 at 12:51:08PM +0100, Greg Kurz wrote:
> > > > > > On Mon, 23 Nov 2020 16:11:30 +1100
> > > > > > David Gibson <da...@gibson.dropbear.id.au> wrote:
> > > > > >
> > > > > > > On Sat, Nov 21, 2020 at 12:42:03AM +0100, Greg Kurz wrote:
> > > > > > > > When it comes to resetting the compat mode of the vCPUS, there
> > > > > > > > are
> > > > > > > > two situations to consider:
> > > > > > > > (1) machine reset should set the compat mode back to the
> > > > > > > > machine default,
> > > > > > > > ie. spapr->max_compat_pvr
> > > > > > > > (2) hot plugged vCPUs should set their compat mode to mach the
> > > > > > > > boot vCPU,
> > > > > > > > ie. POWERPC_CPU(first_cpu)->compat_pvr
> > > > > > > >
> > > > > > > > This is currently handled in two separate places: globally for
> > > > > > > > all vCPUs
> > > > > > > > from the machine reset code for (1) and for each thread of a
> > > > > > > > core from
> > > > > > > > the hotplug path for (2).
> > > > > > > >
> > > > > > > > Since the machine reset code already resets all vCPUs, starting
> > > > > > > > with boot
> > > > > > > > vCPU, consolidate the logic in spapr_reset_vcpu(). Special case
> > > > > > > > the boot
> > > > > > > > vCPU so that it resets its compat mode back to the machine
> > > > > > > > default. Any
> > > > > > > > other vCPU just need to match the compat mode of the boot vCPU.
> > > > > > > >
> > > > > > > > Failing to set the compat mode during machine reset is a fatal
> > > > > > > > error,
> > > > > > > > but not for hot plugged vCPUs. This is arguable because if
> > > > > > > > we've been
> > > > > > > > able to set the boot vCPU compat mode at CAS or during machine
> > > > > > > > reset,
> > > > > > > > it should definitely not fail for other vCPUs. Since
> > > > > > > > spapr_reset_vcpu()
> > > > > > > > already has a fatal error path for kvm_check_mmu() failures, do
> > > > > > > > the
> > > > > > > > same for ppc_set_compat().
> > > > > > > >
> > > > > > > > This gets rid of an error path in spapr_core_plug(). It will
> > > > > > > > allow
> > > > > > > > further simplifications.
> > > > > > > >
> > > > > > > > Signed-off-by: Greg Kurz <gr...@kaod.org>
> > > > > > > > ---
> > > > > > > > hw/ppc/spapr.c | 16 ----------------
> > > > > > > > hw/ppc/spapr_cpu_core.c | 13 +++++++++++++
> > > > > > > > 2 files changed, 13 insertions(+), 16 deletions(-)
> > > > > > > >
> > > > > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > > > > > index f58f77389e8e..da7586f548df 100644
> > > > > > > > --- a/hw/ppc/spapr.c
> > > > > > > > +++ b/hw/ppc/spapr.c
> > > > > > > > @@ -1606,8 +1606,6 @@ static void
> > > > > > > > spapr_machine_reset(MachineState *machine)
> > > > > > > > spapr_ovec_cleanup(spapr->ov5_cas);
> > > > > > > > spapr->ov5_cas = spapr_ovec_new();
> > > > > > > >
> > > > > > > > - ppc_set_compat_all(spapr->max_compat_pvr, &error_fatal);
> > > > > > > > -
> > > > > > > > /*
> > > > > > > > * This is fixing some of the default configuration of the
> > > > > > > > XIVE
> > > > > > > > * devices. To be called after the reset of the machine
> > > > > > > > devices.
> > > > > > > > @@ -3785,20 +3783,6 @@ static void
> > > > > > > > spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > > > > > >
> > > > > > > > core_slot->cpu = OBJECT(dev);
> > > > > > > >
> > > > > > > > - /*
> > > > > > > > - * Set compatibility mode to match the boot CPU, which was
> > > > > > > > either set
> > > > > > > > - * by the machine reset code or by CAS.
> > > > > > > > - */
> > > > > > > > - if (hotplugged) {
> > > > > > > > - for (i = 0; i < cc->nr_threads; i++) {
> > > > > > > > - if (ppc_set_compat(core->threads[i],
> > > > > > > > -
> > > > > > > > POWERPC_CPU(first_cpu)->compat_pvr,
> > > > > > > > - errp) < 0) {
> > > > > > > > - return;
> > > > > > > > - }
> > > > > > > > - }
> > > > > > > > - }
> > > > > > > > -
> > > > > > > > if (smc->pre_2_10_has_unused_icps) {
> > > > > > > > for (i = 0; i < cc->nr_threads; i++) {
> > > > > > > > cs = CPU(core->threads[i]);
> > > > > > > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > > > > > > > index 2f7dc3c23ded..17741a3fb77f 100644
> > > > > > > > --- a/hw/ppc/spapr_cpu_core.c
> > > > > > > > +++ b/hw/ppc/spapr_cpu_core.c
> > > > > > > > @@ -27,6 +27,7 @@
> > > > > > > >
> > > > > > > > static void spapr_reset_vcpu(PowerPCCPU *cpu)
> > > > > > > > {
> > > > > > > > + PowerPCCPU *first_ppc_cpu = POWERPC_CPU(first_cpu);
> > > > > > > > CPUState *cs = CPU(cpu);
> > > > > > > > CPUPPCState *env = &cpu->env;
> > > > > > > > PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
> > > > > > > > @@ -69,6 +70,18 @@ static void spapr_reset_vcpu(PowerPCCPU *cpu)
> > > > > > > > kvm_check_mmu(cpu, &error_fatal);
> > > > > > > >
> > > > > > > > spapr_irq_cpu_intc_reset(spapr, cpu);
> > > > > > > > +
> > > > > > > > + /*
> > > > > > > > + * The boot CPU is only reset during machine reset : reset
> > > > > > > > its
> > > > > > > > + * compatibility mode to the machine default. For other
> > > > > > > > CPUs,
> > > > > > > > + * either cold plugged or hot plugged, set the
> > > > > > > > compatibility mode
> > > > > > > > + * to match the boot CPU, which was either set by the
> > > > > > > > machine reset
> > > > > > > > + * code or by CAS.
> > > > > > > > + */
> > > > > > > > + ppc_set_compat(cpu,
> > > > > > > > + cpu == first_ppc_cpu ?
> > > > > > > > + spapr->max_compat_pvr :
> > > > > > > > first_ppc_cpu->compat_pvr,
> > > > > > > > + &error_fatal);
> > > > > > >
> > > > > > > This assumes that when it is called for a non-boot CPU, it has
> > > > > > > already
> > > > > > > been called for the boot CPU.. Are we certain that's guaranteed
> > > > > > > by
> > > > > > > the sequence of reset calls during a full machine reset?
> > > > > > >
> > > > > >
> > > > > > This happens to be the case. Basically because the boot CPU core
> > > > > > is created (including registering its reset handler) first and
> > > > > > qemu_devices_reset() calls handlers in the same order they were
> > > > > > registered.
> > > > >
> > > > > Right, I assumed it works for now, but it seems rather fragile, since
> > > > > I'm not sure we're relying on guaranteed properties of the interface.
> > > > >
> > > >
> > > > The reset handler interface is absolutely undocumented, so I guess we
> > > > have no formal guarantees at the present time. But since the current
> > > > implementation has the property, would it be acceptable to carve it
> > > > in stone with added documentation ? In the event of unlikely changes
> > > > to the reset handler logic, people would _just_ need to make sure
> > > > handlers are called in the same order they were registered.
> > >
> > > Yeah, maybe.
> > >
> > > One other thing occurs to me: will we still do things in the right
> > > order if the (initial) boot cpu is hot unplugged, then replugged
> > > before a reset?
> > >
> >
> > This can't happen AFAICT.
> >
> > (qemu) qom-get /machine/unattached/device[1] core-id
> > 0
> > (qemu) device_del /machine/unattached/device[1]
> > Error: Boot CPU core may not be unplugged
> >
> > commit 62be8b044adf47327ebefdefb25f28a40316ebd0
> > Author: Bharata B Rao <bhar...@linux.vnet.ibm.com>
> > Date: Wed Jul 27 10:44:42 2016 +0530
> >
> > spapr: Prevent boot CPU core removal
> >
> >
> > So yes, this adds yet another road block on the way to support hot
> > unplug of the boot CPU. Is this a concern ?
> >
> > If we go forward with this patch, maybe I should mention in the
> > changelog/documentation the various assumptions which this patch
> > is made under:
> > - reset handlers are called in the same order they were registered
> > - boot CPU registers its reset handler before other CPUs
> > - boot CPU cannot be hot unplugged
> >
> > These guarantee that the boot core is always reset before other
> > cores during reset.
> it might work for now but it seems fragile to me.
>
> What if we make compat mode a property and move setting it to machine code,
> more precisely treat it like any other cpu feature property.
>
> if(need_compat_more)
> register_global_property(compat_mode)
>
> that way when any cpu is created it will have this property set
> and it won't depend on the order CPUs are created/reset
Ah it's more complicated, ignore this nonsense pls.
>
> >
> > > > > Is there any way we could at least assert() if things are called out
> > > > > of order?
> > > > >
> > > >
> > > > Maybe. I'll look into it.
> > > >
> > > > > >
> > > > > > > > }
> > > > > > > >
> > > > > > > > void spapr_cpu_set_entry_state(PowerPCCPU *cpu, target_ulong
> > > > > > > > nip,
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> >
>
>
