Re: [Qemu-devel] Segfault with coalesced mmio and boot CPU removal

[David Gibson]

On Wed, Jul 27, 2016 at 10:01:41AM +0530, Bharata B Rao wrote:
> On Wed, Jul 27, 2016 at 08:22:51AM +0530, Bharata B Rao wrote:
> > Hi,
> > 
> > Coalesced mmio buffer is part of vCPU 0's kvm_run mmap'ed area
> > and with the introduction of CPU hotplug, vCPU 0 can be removed on
> > PowerPC leading to the below seen segfault in QEMU.
> > 
> > 
> > This happens because during CPU removal, though we park the kvm_fd
> > corresponding to the removed vCPU thread, we unmap the kvm_run (and
> > hence coalesced mmio ring).
> > 
> 
> > What would be the best way to fix this ? Is disassociating 
> > coalesced_mmio_ring
> > from vCPU 0's kvm_run the correct solution ?
> 
> May be PowerPC too should do what x86 does like below for now ?

I think that's what we need to do short term.  In the 2.8 timeframe,
separating the mmio ring out from the vcpu state sounds like a good
idea, but I don't really know how complicated that will be.

> 
> commit 73360e27850b213327011f7e22e03865b8c0dd5b
> Author: Igor Mammedov 
> Date:   Mon Jul 18 10:31:22 2016 +0200
> 
> pc: Forbid BSP removal
> 
> Boot CPU is assumed to always present in QEMU code, so
> untile that assumptions are gone, deny removal request,
> In another words QEMU won't support BSP hot-unplug.
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] Segfault with coalesced mmio and boot CPU removal

[Bharata B Rao]

On Wed, Jul 27, 2016 at 08:22:51AM +0530, Bharata B Rao wrote:
> Hi,
> 
> Coalesced mmio buffer is part of vCPU 0's kvm_run mmap'ed area
> and with the introduction of CPU hotplug, vCPU 0 can be removed on
> PowerPC leading to the below seen segfault in QEMU.
> 
> 
> This happens because during CPU removal, though we park the kvm_fd
> corresponding to the removed vCPU thread, we unmap the kvm_run (and
> hence coalesced mmio ring).
> 

> What would be the best way to fix this ? Is disassociating coalesced_mmio_ring
> from vCPU 0's kvm_run the correct solution ?

May be PowerPC too should do what x86 does like below for now ?

commit 73360e27850b213327011f7e22e03865b8c0dd5b
Author: Igor Mammedov 
Date:   Mon Jul 18 10:31:22 2016 +0200

pc: Forbid BSP removal

Boot CPU is assumed to always present in QEMU code, so
untile that assumptions are gone, deny removal request,
In another words QEMU won't support BSP hot-unplug.

[Qemu-devel] Segfault with coalesced mmio and boot CPU removal

[Bharata B Rao]

Hi,

Coalesced mmio buffer is part of vCPU 0's kvm_run mmap'ed area
and with the introduction of CPU hotplug, vCPU 0 can be removed on
PowerPC leading to the below seen segfault in QEMU.

0x100a1d34 in kvm_flush_coalesced_mmio_buffer ()
at qemu/kvm-all.c:1828
1828while (ring->first != ring->last) {

#0  0x100a1d34 in kvm_flush_coalesced_mmio_buffer ()
at qemu/kvm-all.c:1828
#1  0x100160e0 in qemu_flush_coalesced_mmio_buffer ()
at qemu/exec.c:1206
#2  0x100a961c in memory_region_transaction_begin ()
at qemu/memory.c:904
#3  0x100add90 in memory_region_set_enabled (mr=0x3fff96420310, 
enabled=false)
at qemu/memory.c:1974
#4  0x104577b0 in pci_default_write_config (d=0x3fff96420010, addr=4, 
val_in=258, 
l=2) at hw/pci/pci.c:1340
#5  0x10465d0c in pci_host_config_write_common (pci_dev=0x3fff96420010, 
addr=4, 
limit=4096, val=258, len=2) at hw/pci/pci_host.c:66
#6  0x10170500 in finish_write_pci_config (spapr=0x10fc6290, 
buid=576460752840294400, addr=4, size=2, val=258, rets=20547656)
at qemu/hw/ppc/spapr_pci.c:199
#7  0x10170620 in rtas_ibm_write_pci_config (cpu=0x3fffac590010, 
spapr=0x10fc6290, 
token=8215, nargs=5, args=20547636, nret=1, rets=20547656)
at qemu/hw/ppc/spapr_pci.c:223
#8  0x1016e540 in spapr_rtas_call (cpu=0x3fffac590010, 
spapr=0x10fc6290, token=8215, 
nargs=5, args=20547636, nret=1, rets=20547656)
at qemu/hw/ppc/spapr_rtas.c:675
#9  0x10167dfc in h_rtas (cpu=0x3fffac590010, spapr=0x10fc6290, 
opcode=61440, 
args=0x3fffac570030) at qemu/hw/ppc/spapr_hcall.c:665
#10 0x101693ec in spapr_hypercall (cpu=0x3fffac590010, opcode=61440, 
args=0x3fffac570030) at qemu/hw/ppc/spapr_hcall.c:1094
#11 0x1026c82c in kvm_arch_handle_exit (cs=0x3fffac590010, 
run=0x3fffac57)
at qemu/target-ppc/kvm.c:1731
#12 0x100a246c in kvm_cpu_exec (cpu=0x3fffac590010)
at qemu/kvm-all.c:2005
#13 0x1007d8d4 in qemu_kvm_cpu_thread_fn (arg=0x3fffac590010) 

This happens because during CPU removal, though we park the kvm_fd
corresponding to the removed vCPU thread, we unmap the kvm_run (and
hence coalesced mmio ring).

What would be the best way to fix this ? Is disassociating coalesced_mmio_ring
from vCPU 0's kvm_run the correct solution ?

Regards,
Bharata.