Re: [Qemu-devel] [PATCH v3] virtio-pci: fix migration for pci bus master
On Mon, Oct 20, 2014 at 06:08:56PM +0200, Alexander Graf wrote: On 20.10.14 08:58, Michael S. Tsirkin wrote: Current support for bus master (clearing OK bit) together with the need to support guests which do not enable PCI bus mastering, leads to extra state in VIRTIO_PCI_FLAG_BUS_MASTER_BUG bit, which isn't robust in case of cross-version migration for the case when guests use the device before setting DRIVER_OK. Rip out this code, and replace it: - Modern QEMU doesn't need VIRTIO_PCI_FLAG_BUS_MASTER_BUG so just drop it for latest machine type. - For compat machine types, set PCI_COMMAND if DRIVER_OK is set. As this is needed for 2.1 for both pc and ppc, move PC_COMPAT macros from pc.h to a new common header. Reviewed-by: Greg Kurz gk...@linux.vnet.ibm.com Tested-by: Greg Kurz gk...@linux.vnet.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Alexander, could you pls ack me merging this? Thanks! Have you tried whether this works with old kernel versions? We introduced the broken flag for very specific old kernel versions that got their flags wrong IIRC. Alex Yes. The point is that since 2.1 the broken flag isn't effective anymore: when bus master is off, PCI blocks outgoing transactions and setting internal flags isn't going to help. Instead, we detect a buggy guest and immediately enable bus master for it - no need for a separate flag. -- MST
Re: [Qemu-devel] [PATCH v3] virtio-pci: fix migration for pci bus master
On 21.10.14 12:16, Michael S. Tsirkin wrote: On Mon, Oct 20, 2014 at 06:08:56PM +0200, Alexander Graf wrote: On 20.10.14 08:58, Michael S. Tsirkin wrote: Current support for bus master (clearing OK bit) together with the need to support guests which do not enable PCI bus mastering, leads to extra state in VIRTIO_PCI_FLAG_BUS_MASTER_BUG bit, which isn't robust in case of cross-version migration for the case when guests use the device before setting DRIVER_OK. Rip out this code, and replace it: - Modern QEMU doesn't need VIRTIO_PCI_FLAG_BUS_MASTER_BUG so just drop it for latest machine type. - For compat machine types, set PCI_COMMAND if DRIVER_OK is set. As this is needed for 2.1 for both pc and ppc, move PC_COMPAT macros from pc.h to a new common header. Reviewed-by: Greg Kurz gk...@linux.vnet.ibm.com Tested-by: Greg Kurz gk...@linux.vnet.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Alexander, could you pls ack me merging this? Thanks! Have you tried whether this works with old kernel versions? We introduced the broken flag for very specific old kernel versions that got their flags wrong IIRC. Alex Yes. The point is that since 2.1 the broken flag isn't effective anymore: when bus master is off, PCI blocks outgoing transactions and setting internal flags isn't going to help. Instead, we detect a buggy guest and immediately enable bus master for it - no need for a separate flag. Ok, works for me then. Alex
Re: [Qemu-devel] [PATCH v3] virtio-pci: fix migration for pci bus master
On Tue, Oct 21, 2014 at 01:35:39PM +0200, Alexander Graf wrote: On 21.10.14 12:16, Michael S. Tsirkin wrote: On Mon, Oct 20, 2014 at 06:08:56PM +0200, Alexander Graf wrote: On 20.10.14 08:58, Michael S. Tsirkin wrote: Current support for bus master (clearing OK bit) together with the need to support guests which do not enable PCI bus mastering, leads to extra state in VIRTIO_PCI_FLAG_BUS_MASTER_BUG bit, which isn't robust in case of cross-version migration for the case when guests use the device before setting DRIVER_OK. Rip out this code, and replace it: - Modern QEMU doesn't need VIRTIO_PCI_FLAG_BUS_MASTER_BUG so just drop it for latest machine type. - For compat machine types, set PCI_COMMAND if DRIVER_OK is set. As this is needed for 2.1 for both pc and ppc, move PC_COMPAT macros from pc.h to a new common header. Reviewed-by: Greg Kurz gk...@linux.vnet.ibm.com Tested-by: Greg Kurz gk...@linux.vnet.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Alexander, could you pls ack me merging this? Thanks! Have you tried whether this works with old kernel versions? We introduced the broken flag for very specific old kernel versions that got their flags wrong IIRC. Alex Yes. The point is that since 2.1 the broken flag isn't effective anymore: when bus master is off, PCI blocks outgoing transactions and setting internal flags isn't going to help. Instead, we detect a buggy guest and immediately enable bus master for it - no need for a separate flag. Ok, works for me then. Alex cool. Gerd tells me there's a conflict with your next tree: would you like for me to wait until it's merged, or go ahead and send pull request and rebase yours on top? -- MST
Re: [Qemu-devel] [PATCH v3] virtio-pci: fix migration for pci bus master
On 21.10.14 14:10, Michael S. Tsirkin wrote: On Tue, Oct 21, 2014 at 01:35:39PM +0200, Alexander Graf wrote: On 21.10.14 12:16, Michael S. Tsirkin wrote: On Mon, Oct 20, 2014 at 06:08:56PM +0200, Alexander Graf wrote: On 20.10.14 08:58, Michael S. Tsirkin wrote: Current support for bus master (clearing OK bit) together with the need to support guests which do not enable PCI bus mastering, leads to extra state in VIRTIO_PCI_FLAG_BUS_MASTER_BUG bit, which isn't robust in case of cross-version migration for the case when guests use the device before setting DRIVER_OK. Rip out this code, and replace it: - Modern QEMU doesn't need VIRTIO_PCI_FLAG_BUS_MASTER_BUG so just drop it for latest machine type. - For compat machine types, set PCI_COMMAND if DRIVER_OK is set. As this is needed for 2.1 for both pc and ppc, move PC_COMPAT macros from pc.h to a new common header. Reviewed-by: Greg Kurz gk...@linux.vnet.ibm.com Tested-by: Greg Kurz gk...@linux.vnet.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Alexander, could you pls ack me merging this? Thanks! Have you tried whether this works with old kernel versions? We introduced the broken flag for very specific old kernel versions that got their flags wrong IIRC. Alex Yes. The point is that since 2.1 the broken flag isn't effective anymore: when bus master is off, PCI blocks outgoing transactions and setting internal flags isn't going to help. Instead, we detect a buggy guest and immediately enable bus master for it - no need for a separate flag. Ok, works for me then. Alex cool. Gerd tells me there's a conflict with your next tree: would you like for me to wait until it's merged, or go ahead and send pull request and rebase yours on top? Whoever sends it first wins I'd say ;) Alex
Re: [Qemu-devel] [PATCH v3] virtio-pci: fix migration for pci bus master
On Tue, 21 Oct 2014 15:10:22 +0300 Michael S. Tsirkin m...@redhat.com wrote: On Tue, Oct 21, 2014 at 01:35:39PM +0200, Alexander Graf wrote: On 21.10.14 12:16, Michael S. Tsirkin wrote: On Mon, Oct 20, 2014 at 06:08:56PM +0200, Alexander Graf wrote: On 20.10.14 08:58, Michael S. Tsirkin wrote: Current support for bus master (clearing OK bit) together with the need to support guests which do not enable PCI bus mastering, leads to extra state in VIRTIO_PCI_FLAG_BUS_MASTER_BUG bit, which isn't robust in case of cross-version migration for the case when guests use the device before setting DRIVER_OK. Rip out this code, and replace it: - Modern QEMU doesn't need VIRTIO_PCI_FLAG_BUS_MASTER_BUG so just drop it for latest machine type. - For compat machine types, set PCI_COMMAND if DRIVER_OK is set. As this is needed for 2.1 for both pc and ppc, move PC_COMPAT macros from pc.h to a new common header. Reviewed-by: Greg Kurz gk...@linux.vnet.ibm.com Tested-by: Greg Kurz gk...@linux.vnet.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Alexander, could you pls ack me merging this? Thanks! Have you tried whether this works with old kernel versions? We introduced the broken flag for very specific old kernel versions that got their flags wrong IIRC. Alex Yes. The point is that since 2.1 the broken flag isn't effective anymore: when bus master is off, PCI blocks outgoing transactions and setting internal flags isn't going to help. Instead, we detect a buggy guest and immediately enable bus master for it - no need for a separate flag. Ok, works for me then. Alex cool. Gerd tells me there's a conflict with your next tree: would you like for me to wait until it's merged, or go ahead and send pull request and rebase yours on top? This patch also needs to be rebased on current master actually because PC_COMPAT_2_1 in include/hw/i386/pc.h got changed. Cheers. -- Greg
[Qemu-devel] [PATCH v3] virtio-pci: fix migration for pci bus master
Current support for bus master (clearing OK bit) together with the need to support guests which do not enable PCI bus mastering, leads to extra state in VIRTIO_PCI_FLAG_BUS_MASTER_BUG bit, which isn't robust in case of cross-version migration for the case when guests use the device before setting DRIVER_OK. Rip out this code, and replace it: - Modern QEMU doesn't need VIRTIO_PCI_FLAG_BUS_MASTER_BUG so just drop it for latest machine type. - For compat machine types, set PCI_COMMAND if DRIVER_OK is set. As this is needed for 2.1 for both pc and ppc, move PC_COMPAT macros from pc.h to a new common header. Reviewed-by: Greg Kurz gk...@linux.vnet.ibm.com Tested-by: Greg Kurz gk...@linux.vnet.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Alexander, could you pls ack me merging this? Thanks! changes from v2: drop default = -1 from ppc - was a typo, reported by Greg hw/virtio/virtio-pci.h | 5 + include/hw/compat.h| 16 include/hw/i386/pc.h | 10 ++ hw/i386/pc_piix.c | 2 +- hw/i386/pc_q35.c | 2 +- hw/ppc/spapr.c | 7 +++ hw/virtio/virtio-pci.c | 29 +++-- 7 files changed, 43 insertions(+), 28 deletions(-) create mode 100644 include/hw/compat.h diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h index 1cea157..8873b6d 100644 --- a/hw/virtio/virtio-pci.h +++ b/hw/virtio/virtio-pci.h @@ -53,6 +53,11 @@ typedef struct VirtioBusClass VirtioPCIBusClass; #define VIRTIO_PCI_BUS_CLASS(klass) \ OBJECT_CLASS_CHECK(VirtioPCIBusClass, klass, TYPE_VIRTIO_PCI_BUS) +/* Need to activate work-arounds for buggy guests at vmstate load. */ +#define VIRTIO_PCI_FLAG_BUS_MASTER_BUG_MIGRATION_BIT 0 +#define VIRTIO_PCI_FLAG_BUS_MASTER_BUG_MIGRATION \ +(1 VIRTIO_PCI_FLAG_BUS_MASTER_BUG_MIGRATION_BIT) + /* Performance improves when virtqueue kick processing is decoupled from the * vcpu thread using ioeventfd for some devices. */ #define VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT 1 diff --git a/include/hw/compat.h b/include/hw/compat.h new file mode 100644 index 000..47f6ff5 --- /dev/null +++ b/include/hw/compat.h @@ -0,0 +1,16 @@ +#ifndef HW_COMPAT_H +#define HW_COMPAT_H + +#define HW_COMPAT_2_1 \ +{\ +.driver = intel-hda,\ +.property = old_msi_addr,\ +.value= on,\ +},\ +{\ +.driver = virtio-pci,\ +.property = virtio-pci-bus-master-bug-migration,\ +.value= on,\ +} + +#endif /* HW_COMPAT_H */ diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h index bae023a..82ad046 100644 --- a/include/hw/i386/pc.h +++ b/include/hw/i386/pc.h @@ -14,6 +14,7 @@ #include sysemu/sysemu.h #include hw/pci/pci.h #include hw/boards.h +#include hw/compat.h #define HPET_INTCAP hpet-intcap @@ -307,15 +308,8 @@ int e820_add_entry(uint64_t, uint64_t, uint32_t); int e820_get_num_entries(void); bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *); -#define PC_COMPAT_2_1 \ -{\ -.driver = intel-hda,\ -.property = old_msi_addr,\ -.value= on,\ -} - #define PC_COMPAT_2_0 \ -PC_COMPAT_2_1, \ +HW_COMPAT_2_1, \ {\ .driver = virtio-scsi-pci,\ .property = any_layout,\ diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index 553afdd..a1634ab 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -502,7 +502,7 @@ static QEMUMachine pc_i440fx_machine_v2_1 = { .name = pc-i440fx-2.1, .init = pc_init_pci, .compat_props = (GlobalProperty[]) { -PC_COMPAT_2_1, +HW_COMPAT_2_1, { /* end of list */ } }, }; diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c index a199043..f330f7a 100644 --- a/hw/i386/pc_q35.c +++ b/hw/i386/pc_q35.c @@ -365,7 +365,7 @@ static QEMUMachine pc_q35_machine_v2_1 = { .name = pc-q35-2.1, .init = pc_q35_init, .compat_props = (GlobalProperty[]) { -PC_COMPAT_2_1, +HW_COMPAT_2_1, { /* end of list */ } }, }; diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 2becc9f..623f626 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -57,6 +57,8 @@ #include trace.h #include hw/nmi.h +#include hw/compat.h + #include libfdt.h /* SLOF memory layout: @@ -1689,10 +1691,15 @@ static const TypeInfo spapr_machine_info = { static void spapr_machine_2_1_class_init(ObjectClass *oc, void *data) { MachineClass *mc = MACHINE_CLASS(oc); +static GlobalProperty compat_props[] = { +HW_COMPAT_2_1, +{ /* end of list */ } +}; mc-name = pseries-2.1; mc-desc = pSeries Logical Partition (PAPR compliant) v2.1; mc-is_default = 0; +mc-compat_props = compat_props; } static const TypeInfo spapr_machine_2_1_info = { diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index a827cd4..a499a3c 100644 --- a/hw/virtio/virtio-pci.c +++
Re: [Qemu-devel] [PATCH v3] virtio-pci: fix migration for pci bus master
On Mon, 20 Oct 2014 09:58:50 +0300 Michael S. Tsirkin m...@redhat.com wrote: Current support for bus master (clearing OK bit) together with the need to support guests which do not enable PCI bus mastering, leads to extra state in VIRTIO_PCI_FLAG_BUS_MASTER_BUG bit, which isn't robust in case of cross-version migration for the case when guests use the device before setting DRIVER_OK. Rip out this code, and replace it: - Modern QEMU doesn't need VIRTIO_PCI_FLAG_BUS_MASTER_BUG so just drop it for latest machine type. - For compat machine types, set PCI_COMMAND if DRIVER_OK is set. As this is needed for 2.1 for both pc and ppc, move PC_COMPAT macros from pc.h to a new common header. Reviewed-by: Greg Kurz gk...@linux.vnet.ibm.com Tested-by: Greg Kurz gk...@linux.vnet.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Alexander, could you pls ack me merging this? Thanks! This patch conflicts with the following commit in Alex's ppc-next branch: commit 6dd65a940532eaac42a302c30964b0d42967e187 Author: David Gibson da...@gibson.dropbear.id.au Date: Mon Sep 8 15:30:31 2014 +1000 spapr: Cleanup machine naming conventions, and prepare for 2.2 release Easy to fix though. Alex, I can send the rebased version if you want. Cheers. -- Greg changes from v2: drop default = -1 from ppc - was a typo, reported by Greg hw/virtio/virtio-pci.h | 5 + include/hw/compat.h| 16 include/hw/i386/pc.h | 10 ++ hw/i386/pc_piix.c | 2 +- hw/i386/pc_q35.c | 2 +- hw/ppc/spapr.c | 7 +++ hw/virtio/virtio-pci.c | 29 +++-- 7 files changed, 43 insertions(+), 28 deletions(-) create mode 100644 include/hw/compat.h diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h index 1cea157..8873b6d 100644 --- a/hw/virtio/virtio-pci.h +++ b/hw/virtio/virtio-pci.h @@ -53,6 +53,11 @@ typedef struct VirtioBusClass VirtioPCIBusClass; #define VIRTIO_PCI_BUS_CLASS(klass) \ OBJECT_CLASS_CHECK(VirtioPCIBusClass, klass, TYPE_VIRTIO_PCI_BUS) +/* Need to activate work-arounds for buggy guests at vmstate load. */ +#define VIRTIO_PCI_FLAG_BUS_MASTER_BUG_MIGRATION_BIT 0 +#define VIRTIO_PCI_FLAG_BUS_MASTER_BUG_MIGRATION \ +(1 VIRTIO_PCI_FLAG_BUS_MASTER_BUG_MIGRATION_BIT) + /* Performance improves when virtqueue kick processing is decoupled from the * vcpu thread using ioeventfd for some devices. */ #define VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT 1 diff --git a/include/hw/compat.h b/include/hw/compat.h new file mode 100644 index 000..47f6ff5 --- /dev/null +++ b/include/hw/compat.h @@ -0,0 +1,16 @@ +#ifndef HW_COMPAT_H +#define HW_COMPAT_H + +#define HW_COMPAT_2_1 \ +{\ +.driver = intel-hda,\ +.property = old_msi_addr,\ +.value= on,\ +},\ +{\ +.driver = virtio-pci,\ +.property = virtio-pci-bus-master-bug-migration,\ +.value= on,\ +} + +#endif /* HW_COMPAT_H */ diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h index bae023a..82ad046 100644 --- a/include/hw/i386/pc.h +++ b/include/hw/i386/pc.h @@ -14,6 +14,7 @@ #include sysemu/sysemu.h #include hw/pci/pci.h #include hw/boards.h +#include hw/compat.h #define HPET_INTCAP hpet-intcap @@ -307,15 +308,8 @@ int e820_add_entry(uint64_t, uint64_t, uint32_t); int e820_get_num_entries(void); bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *); -#define PC_COMPAT_2_1 \ -{\ -.driver = intel-hda,\ -.property = old_msi_addr,\ -.value= on,\ -} - #define PC_COMPAT_2_0 \ -PC_COMPAT_2_1, \ +HW_COMPAT_2_1, \ {\ .driver = virtio-scsi-pci,\ .property = any_layout,\ diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index 553afdd..a1634ab 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -502,7 +502,7 @@ static QEMUMachine pc_i440fx_machine_v2_1 = { .name = pc-i440fx-2.1, .init = pc_init_pci, .compat_props = (GlobalProperty[]) { -PC_COMPAT_2_1, +HW_COMPAT_2_1, { /* end of list */ } }, }; diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c index a199043..f330f7a 100644 --- a/hw/i386/pc_q35.c +++ b/hw/i386/pc_q35.c @@ -365,7 +365,7 @@ static QEMUMachine pc_q35_machine_v2_1 = { .name = pc-q35-2.1, .init = pc_q35_init, .compat_props = (GlobalProperty[]) { -PC_COMPAT_2_1, +HW_COMPAT_2_1, { /* end of list */ } }, }; diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 2becc9f..623f626 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -57,6 +57,8 @@ #include trace.h #include hw/nmi.h +#include hw/compat.h + #include libfdt.h /* SLOF memory layout: @@ -1689,10 +1691,15 @@ static const TypeInfo
Re: [Qemu-devel] [PATCH v3] virtio-pci: fix migration for pci bus master
On 20.10.14 08:58, Michael S. Tsirkin wrote: Current support for bus master (clearing OK bit) together with the need to support guests which do not enable PCI bus mastering, leads to extra state in VIRTIO_PCI_FLAG_BUS_MASTER_BUG bit, which isn't robust in case of cross-version migration for the case when guests use the device before setting DRIVER_OK. Rip out this code, and replace it: - Modern QEMU doesn't need VIRTIO_PCI_FLAG_BUS_MASTER_BUG so just drop it for latest machine type. - For compat machine types, set PCI_COMMAND if DRIVER_OK is set. As this is needed for 2.1 for both pc and ppc, move PC_COMPAT macros from pc.h to a new common header. Reviewed-by: Greg Kurz gk...@linux.vnet.ibm.com Tested-by: Greg Kurz gk...@linux.vnet.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Alexander, could you pls ack me merging this? Thanks! Have you tried whether this works with old kernel versions? We introduced the broken flag for very specific old kernel versions that got their flags wrong IIRC. Alex
Re: [Qemu-devel] [PATCH v3] virtio-pci: fix migration for pci bus master
On Mon, 20 Oct 2014 18:08:56 +0200 Alexander Graf ag...@suse.de wrote: On 20.10.14 08:58, Michael S. Tsirkin wrote: Current support for bus master (clearing OK bit) together with the need to support guests which do not enable PCI bus mastering, leads to extra state in VIRTIO_PCI_FLAG_BUS_MASTER_BUG bit, which isn't robust in case of cross-version migration for the case when guests use the device before setting DRIVER_OK. Rip out this code, and replace it: - Modern QEMU doesn't need VIRTIO_PCI_FLAG_BUS_MASTER_BUG so just drop it for latest machine type. - For compat machine types, set PCI_COMMAND if DRIVER_OK is set. As this is needed for 2.1 for both pc and ppc, move PC_COMPAT macros from pc.h to a new common header. Reviewed-by: Greg Kurz gk...@linux.vnet.ibm.com Tested-by: Greg Kurz gk...@linux.vnet.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Alexander, could you pls ack me merging this? Thanks! Have you tried whether this works with old kernel versions? We introduced the broken flag for very specific old kernel versions that got their flags wrong IIRC. This is a follow-up to the following commit: commit e43c0b2ea5574efb0bedebf6a7d05916eefeba52 Author: Michael S. Tsirkin m...@redhat.com Date: Thu Sep 11 18:45:33 2014 +0200 virtio-pci: enable bus master for old guests The goal was to support *again* buggy guests that do DMA without enabling bus master, since QEMU now uses the bus master address space for delivering MSI/MSI-x messages. We were missing some bits to support cross version migration. Thanks to Michael's patch, I could migrate a buggy RHEL 6.5 guest from QEMU v2.1 to QEMU master. Alex Cheers. -- Greg