Re: [PULL 00/39] Migration 20231024 patches

2023-10-25 Thread Stefan Hajnoczi
On Tue, 24 Oct 2023 at 23:45, Juan Quintela wrote: > > The following changes since commit a95260486aa7e78d7c7194eba65cf03311ad94ad: > > Merge tag 'pull-tcg-20231023' of https://gitlab.com/rth7680/qemu into > staging (2023-10-23 14:45:46 -0700) > > are available in the Git repository at: > >

Re: [PATCH v3 28/28] docs: update Xen-on-KVM documentation

2023-10-25 Thread David Woodhouse
On Wed, 2023-10-25 at 19:56 +0100, Andrew Cooper wrote: > On 25/10/2023 7:26 pm, David Woodhouse wrote: >   > > On Wed, 2023-10-25 at 13:20 -0500, Eric Blake wrote: > >   > > > On Wed, Oct 25, 2023 at 03:50:42PM +0100, David Woodhouse wrote: > >   > > >   > > > > + > > > > +Booting Xen PV guests >

Re: [PATCH v3 28/28] docs: update Xen-on-KVM documentation

2023-10-25 Thread Andrew Cooper
On 25/10/2023 7:26 pm, David Woodhouse wrote: > On Wed, 2023-10-25 at 13:20 -0500, Eric Blake wrote: >> On Wed, Oct 25, 2023 at 03:50:42PM +0100, David Woodhouse wrote: >>> + >>> +Booting Xen PV guests >>> +- >>> + >>> +Booting PV guest kernels is possible by using the Xen PV

Re: [PATCH v3 28/28] docs: update Xen-on-KVM documentation

2023-10-25 Thread David Woodhouse
On Wed, 2023-10-25 at 13:20 -0500, Eric Blake wrote: > On Wed, Oct 25, 2023 at 03:50:42PM +0100, David Woodhouse wrote: > > From: David Woodhouse > > > > Add notes about console and network support, and how to launch PV guests. > > Clean up the disk configuration examples now that that's

Re: [PATCH v3 28/28] docs: update Xen-on-KVM documentation

2023-10-25 Thread Eric Blake
On Wed, Oct 25, 2023 at 03:50:42PM +0100, David Woodhouse wrote: > From: David Woodhouse > > Add notes about console and network support, and how to launch PV guests. > Clean up the disk configuration examples now that that's simpler, and > remove the comment about IDE unplug on q35/AHCI now

Re: [PATCH v3 0/9] mirror: allow switching from background to active mode

2023-10-25 Thread Kevin Wolf
Am 25.10.2023 um 14:27 hat Fiona Ebner geschrieben: > Am 23.10.23 um 13:39 schrieb Fiona Ebner: > > Am 19.10.23 um 15:36 schrieb Kevin Wolf: > >> Most of this series looks good to me. Apart from the comments I made in > >> the individual patches, I would like to see iotests coverage of changing >

[PATCH v3 06/28] hw/xen: take iothread mutex in xen_evtchn_reset_op()

2023-10-25 Thread David Woodhouse
From: David Woodhouse The xen_evtchn_soft_reset() function requires the iothread mutex, but is also called for the EVTCHNOP_reset hypercall. Ensure the mutex is taken in that case. Fixes: a15b10978fe6 ("hw/xen: Implement EVTCHNOP_reset") Signed-off-by: David Woodhouse ---

[PATCH v3 05/28] hw/xen: fix XenStore watch delivery to guest

2023-10-25 Thread David Woodhouse
From: David Woodhouse When fire_watch_cb() found the response buffer empty, it would call deliver_watch() to generate the XS_WATCH_EVENT message in the response buffer and send an event channel notification to the guest… without actually *copying* the response buffer into the ring. So there was

[PATCH v3 02/28] i386/xen: fix per-vCPU upcall vector for Xen emulation

2023-10-25 Thread David Woodhouse
From: David Woodhouse The per-vCPU upcall vector support had three problems. Firstly it was using the wrong hypercall argument and would always return -EFAULT when the guest tried to set it up. Secondly it was using the wrong ioctl() to pass the vector to the kernel and thus the *kernel* would

[PATCH v3 25/28] hw/pci: add pci_init_nic_devices(), pci_init_nic_in_slot()

2023-10-25 Thread David Woodhouse
From: David Woodhouse The loop over nd_table[] to add PCI NICs is repeated in quite a few places. Add a helper function to do it. Some platforms also try to instantiate a specific model in a specific slot, to match the real hardware. Add pci_init_nic_in_slot() for that purpose. Signed-off-by:

[PATCH v3 13/28] hw/xen: automatically assign device index to block devices

2023-10-25 Thread David Woodhouse
From: David Woodhouse There's no need to force the user to assign a vdev. We can automatically assign one, starting at xvda and searching until we find the first disk name that's unused. This means we can now allow '-drive if=xen,file=xxx' to work without an explicit separate -driver argument,

[PATCH v3 08/28] i386/xen: Ignore VCPU_SSHOTTMR_future flag in set_singleshot_timer()

2023-10-25 Thread David Woodhouse
From: David Woodhouse Upstream Xen now ignores this flag¹, since the only guest kernel ever to use it was buggy. ¹ https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=19c6cbd909 Signed-off-by: David Woodhouse Reviewed-by: Paul Durrant --- target/i386/kvm/xen-emu.c | 20

[PATCH v3 12/28] hw/xen: populate store frontend nodes with XenStore PFN/port

2023-10-25 Thread David Woodhouse
From: David Woodhouse This is kind of redundant since without being able to get these through some other method (HVMOP_get_param) the guest wouldn't be able to access XenStore in order to find them. Signed-off-by: David Woodhouse Reviewed-by: Paul Durrant --- hw/i386/kvm/xen_xenstore.c | 11

[PATCH v3 18/28] hw/xen: only remove peers of PCI NICs on unplug

2023-10-25 Thread David Woodhouse
From: David Woodhouse When the Xen guest asks to unplug *emulated* NICs, it's kind of unhelpful also to unplug the peer of the *Xen* PV NIC. Signed-off-by: David Woodhouse --- hw/i386/xen/xen_platform.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git

[PATCH v3 15/28] hw/xen: do not repeatedly try to create a failing backend device

2023-10-25 Thread David Woodhouse
From: David Woodhouse If xen_backend_device_create() fails to instantiate a device, the XenBus code will just keep trying over and over again each time the bus is re-enumerated, as long as the backend appears online and in XenbusStateInitialising. The only thing which prevents the XenBus code

[PATCH v3 04/28] hw/xen: don't clear map_track[] in xen_gnttab_reset()

2023-10-25 Thread David Woodhouse
From: David Woodhouse The refcounts actually correspond to 'active_ref' structures stored in a GHashTable per "user" on the backend side (mostly, per XenDevice). If we zero map_track[] on reset, then when the backend drivers get torn down and release their mapping we hit the

[PATCH v3 24/28] net: add qemu_create_nic_bus_devices()

2023-10-25 Thread David Woodhouse
From: David Woodhouse This will instantiate any NICs which live on a given bus type. Each bus is allowed *one* substitution (for PCI it's virtio → virtio-net-pci, for Xen it's xen → xen-net-device; no point in overengineering it unless we actually want more). Signed-off-by: David Woodhouse ---

[PATCH v3 17/28] hw/xen: add support for Xen primary console in emulated mode

2023-10-25 Thread David Woodhouse
From: David Woodhouse The primary console is special because the toolstack maps a page into the guest for its ring, and also allocates the guest-side event channel. The guest's grant table is even primed to export that page using a known grant ref#. Add support for all that in emulated mode, so

[PATCH v3 09/28] hw/xen: Clean up event channel 'type_val' handling to use union

2023-10-25 Thread David Woodhouse
From: David Woodhouse A previous implementation of this stuff used a 64-bit field for all of the port information (vcpu/type/type_val) and did atomic exchanges on them. When I implemented that in Qemu I regretted my life choices and just kept it simple with locking instead. So there's no need

[PATCH v3 22/28] net: add qemu_{configure, create}_nic_device(), qemu_find_nic_info()

2023-10-25 Thread David Woodhouse
From: David Woodhouse Most code which directly accesses nd_table[] and nb_nics uses them for one of two things. Either "I have created a NIC device and I'd like a configuration for it", or "I will create a NIC device *if* there is a configuration for it". With some variants on the theme around

[PATCH v3 20/28] net: do not delete nics in net_cleanup()

2023-10-25 Thread David Woodhouse
From: David Woodhouse In net_cleanup() we only need to delete the netdevs, as those may have state which outlives Qemu when it exits, and thus may actually need to be cleaned up on exit. The nics, on the other hand, are owned by the device which created them. Most devices don't bother to clean

[PATCH v3 16/28] hw/xen: update Xen console to XenDevice model

2023-10-25 Thread David Woodhouse
From: David Woodhouse This allows (non-primary) console devices to be created on the command line and hotplugged. Signed-off-by: David Woodhouse Reviewed-by: Paul Durrant --- hw/char/trace-events| 8 + hw/char/xen_console.c | 532 +++-

[PATCH v3 11/28] i386/xen: advertise XEN_HVM_CPUID_UPCALL_VECTOR in CPUID

2023-10-25 Thread David Woodhouse
From: David Woodhouse This will allow Linux guests (since v6.0) to use the per-vCPU upcall vector delivered as MSI through the local APIC. Signed-off-by: David Woodhouse Reviewed-by: Paul Durrant --- target/i386/kvm/kvm.c | 4 1 file changed, 4 insertions(+) diff --git

[PATCH v3 10/28] include: update Xen public headers to Xen 4.17.2 release

2023-10-25 Thread David Woodhouse
From: David Woodhouse ... in order to advertise the XEN_HVM_CPUID_UPCALL_VECTOR feature, which will come in a subsequent commit. Signed-off-by: David Woodhouse Acked-by: Paul Durrant --- hw/i386/kvm/xen_xenstore.c| 2 +- include/hw/xen/interface/arch-arm.h | 37

[PATCH v3 00/28] Get Xen PV shim running in QEMU, add net & console

2023-10-25 Thread David Woodhouse
Round up a handful of outstanding fixes, add console support and fix up per-vCPU upcall vector support (which was previously untested), and that allows us to boot PV guests using the Xen "PV shim". Having fixed up the per-vCPU upcall vector support, pull in slightly newer Xen headers just for

[PATCH v3 19/28] hw/xen: update Xen PV NIC to XenDevice model

2023-10-25 Thread David Woodhouse
From: David Woodhouse This allows us to use Xen PV networking with emulated Xen guests, and to add them on the command line or hotplug. Signed-off-by: David Woodhouse --- hw/net/meson.build| 2 +- hw/net/trace-events | 11 + hw/net/xen_nic.c | 484

[PATCH v3 03/28] hw/xen: select kernel mode for per-vCPU event channel upcall vector

2023-10-25 Thread David Woodhouse
From: David Woodhouse A guest which has configured the per-vCPU upcall vector may set the HVM_PARAM_CALLBACK_IRQ param to fairly much anything other than zero. For example, Linux v6.0+ after commit b1c3497e604 ("x86/xen: Add support for HVMOP_set_evtchn_upcall_vector") will just do this after

[PATCH v3 28/28] docs: update Xen-on-KVM documentation

2023-10-25 Thread David Woodhouse
From: David Woodhouse Add notes about console and network support, and how to launch PV guests. Clean up the disk configuration examples now that that's simpler, and remove the comment about IDE unplug on q35/AHCI now that it's fixed. Also update stale avocado test filename in MAINTAINERS.

[PATCH v3 01/28] i386/xen: Don't advertise XENFEAT_supervisor_mode_kernel

2023-10-25 Thread David Woodhouse
From: David Woodhouse This confuses lscpu into thinking it's running in PVH mode. Fixes: bedcc139248 ("i386/xen: implement HYPERVISOR_xen_version") Signed-off-by: David Woodhouse Reviewed-by: Paul Durrant --- target/i386/kvm/xen-emu.c | 1 - 1 file changed, 1 deletion(-) diff --git

[PATCH v3 21/28] xen-platform: unplug AHCI disks

2023-10-25 Thread David Woodhouse
From: David Woodhouse To support Xen guests using the Q35 chipset, the unplug protocol needs to also remove AHCI disks. Make pci_xen_ide_unplug() more generic, iterating over the children of the PCI device and destroying the "ide-hd" devices. That works the same for both AHCI and IDE, as does

[PATCH v3 27/28] hw/xen: use qemu_create_nic_bus_devices() to instantiate Xen NICs

2023-10-25 Thread David Woodhouse
From: David Woodhouse When instantiating XenBus itself, for each NIC which is configured with either the model unspecified, or set to to "xen" or "xen-net-device", create a corresponding xen-net-device for it. Now we can launch emulated Xen guests with '-nic user', and this fixes the setup for

[PATCH v3 26/28] hw/i386/pc: use qemu_get_nic_info() and pci_init_nic_devices()

2023-10-25 Thread David Woodhouse
From: David Woodhouse Eliminate direct access to nd_table[] and nb_nics by processing the the ISA NICs first and then calling pci_init_nic_devices() for the test. It's important to do this *before* the subsequent patch which registers the Xen PV network devices, because the code being remove

[PATCH v3 14/28] hw/xen: add get_frontend_path() method to XenDeviceClass

2023-10-25 Thread David Woodhouse
From: David Woodhouse The primary Xen console is special. The guest's side is set up for it by the toolstack automatically and not by the standard PV init sequence. Accordingly, its *frontend* doesn't appear in …/device/console/0 either; instead it appears under …/console in the guest's

[PATCH v3 07/28] hw/xen: use correct default protocol for xen-block on x86

2023-10-25 Thread David Woodhouse
From: David Woodhouse Even on x86_64 the default protocol is the x86-32 one if the guest doesn't specifically ask for x86-64. Fixes: b6af8926fb85 ("xen: add implementations of xen-block connect and disconnect functions...") Signed-off-by: David Woodhouse --- hw/block/xen-block.c | 6 +-

[PATCH v3 23/28] net: report list of available models according to platform

2023-10-25 Thread David Woodhouse
From: David Woodhouse By noting the models for which a configuration was requested, we can give the user an accurate list of which NIC models were actually available on the platform/configuration that was otherwise chosen. Signed-off-by: David Woodhouse --- net/net.c | 94

Re: [PATCH v2 01/12] qemu-file: Don't increment qemu_file_transferred at qemu_file_fill_buffer

2023-10-25 Thread Fabiano Rosas
Juan Quintela writes: > We only call qemu_file_transferred_* on the sending side. Remove the > increment at qemu_file_fill_buffer() and add asserts to > qemu_file_transferred* functions. > > Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas

Re: [PATCH v3 0/9] mirror: allow switching from background to active mode

2023-10-25 Thread Fiona Ebner
Am 23.10.23 um 13:39 schrieb Fiona Ebner: > Am 19.10.23 um 15:36 schrieb Kevin Wolf: >> Most of this series looks good to me. Apart from the comments I made in >> the individual patches, I would like to see iotests coverage of changing >> the mirroring mode. At the least to show that the query

Re: [PATCH 0/3] fix a few blockjob-related deadlocks when using iothread

2023-10-25 Thread Kevin Wolf
Am 19.10.2023 um 15:19 hat Fiona Ebner geschrieben: > The discussion leading up to this series can be found here: > https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg00729.html > > Fiona Ebner (3): > blockjob: drop AioContext lock before calling bdrv_graph_wrlock() > block: avoid

Re: [PATCH 12/12] hw/xen: add support for Xen primary console in emulated mode

2023-10-25 Thread Paul Durrant
On 25/10/2023 10:00, David Woodhouse wrote: On Wed, 2023-10-25 at 09:31 +0100, Paul Durrant wrote: On 24/10/2023 17:34, David Woodhouse wrote: On Tue, 2023-10-24 at 17:25 +0100, Paul Durrant wrote: On 24/10/2023 16:49, David Woodhouse wrote: On Tue, 2023-10-24 at 16:39 +0100, Paul Durrant

[PATCH v2 07/12] migration: migration_transferred_bytes() don't need the QEMUFile

2023-10-25 Thread Juan Quintela
Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Juan Quintela --- migration/migration-stats.h | 4 +--- migration/migration-stats.c | 6 +++--- migration/migration.c | 6 +++--- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/migration/migration-stats.h

[PATCH v2 05/12] qemu_file: Remove unused qemu_file_transferred()

2023-10-25 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/qemu-file.h | 18 -- migration/qemu-file.c | 7 --- 2 files changed, 25 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index a29c37b0d0..8b71152754 100644 ---

[PATCH v2 03/12] qemu_file: total_transferred is not used anymore

2023-10-25 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/qemu-file.c | 4 1 file changed, 4 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 384985f534..641ab703cc 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -41,9 +41,6 @@

[PATCH v2 01/12] qemu-file: Don't increment qemu_file_transferred at qemu_file_fill_buffer

2023-10-25 Thread Juan Quintela
We only call qemu_file_transferred_* on the sending side. Remove the increment at qemu_file_fill_buffer() and add asserts to qemu_file_transferred* functions. Signed-off-by: Juan Quintela --- migration/qemu-file.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git

[PATCH v2 04/12] migration: Use the number of transferred bytes directly

2023-10-25 Thread Juan Quintela
We only use migration_transferred_bytes() to calculate the rate_limit, for that we don't need to flush whatever is on the qemu_file buffer. Remember that the buffer is really small (normal case is 32K if we use iov's can be 64 * TARGET_PAGE_SIZE), so this is not relevant to calculations.

[PATCH v2 06/12] qemu-file: Remove _noflush from qemu_file_transferred_noflush()

2023-10-25 Thread Juan Quintela
qemu_file_transferred() don't exist anymore, so we can reuse the name. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- v2: Update the documentation (thanks fabiano) --- migration/qemu-file.h | 9 - migration/block.c | 4 ++-- migration/qemu-file.c | 2 +-

[PATCH v2 02/12] qemu_file: Use a stat64 for qemu_file_transferred

2023-10-25 Thread Juan Quintela
This way we can read it from any thread. I checked that it gives the same value as the current one. We never use two qemu_files at the same time. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/migration-stats.h | 4 migration/qemu-file.c | 5 +++-- 2 files

[PATCH v2 08/12] migration: migration_rate_limit_reset() don't need the QEMUFile

2023-10-25 Thread Juan Quintela
Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Juan Quintela --- migration/migration-stats.h | 4 +--- migration/migration-stats.c | 2 +- migration/migration.c | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/migration/migration-stats.h

[PATCH v2 09/12] qemu-file: Simplify qemu_file_get_error()

2023-10-25 Thread Juan Quintela
If we pass a NULL error is the same that returning directly the value. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/qemu-file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index

[PATCH v2 11/12] migration: Remove transferred atomic counter

2023-10-25 Thread Juan Quintela
After last commit, it is a write only variable. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/migration-stats.h | 4 migration/multifd.c | 3 --- migration/ram.c | 1 - 3 files changed, 8 deletions(-) diff --git a/migration/migration-stats.h

[PATCH v2 12/12] qemu-file: Make qemu_fflush() return errors

2023-10-25 Thread Juan Quintela
This let us simplify code of this shape. qemu_fflush(f); int ret = qemu_file_get_error(f); if (ret) { return ret; } into: int ret = qemu_fflush(f); if (ret) { return ret; } I updated all callers where there is any error check. qemu_fclose() don't need to check

[PATCH v2 10/12] migration: Use migration_transferred_bytes()

2023-10-25 Thread Juan Quintela
There are only two differnces with the old value: - the amount of QEMUFile that hasn't yet been flushed. It can be discussed what is more exact, the new or the old one. - the amount of transferred bytes that we forgot to account for (the newer is better, i.e. exact). Notice that this two

[PATCH v2 00/12] migration: Yet another round of atomic counters

2023-10-25 Thread Juan Quintela
Hi On this v2: - Redid commint for patch 1 - Every other patch is reviewed now. Please review pending patch. Thanks, Juan. [v1] Goal of the whole series was to be able to move rate_limit logic to not use qemu_file. Goal achieved. Removal of trasnferred atomic counter. After this series, we

Re: [PATCH 02/12] qemu_file: Use a stat64 for qemu_file_transferred

2023-10-25 Thread Juan Quintela
Eric Blake wrote: > On Tue, Oct 24, 2023 at 05:10:32PM +0200, Juan Quintela wrote: >> This way we can read it from any thread. >> I checked that it gives the same value than the current one. We never > > s/than/as/ Done >> use to qemu_files at the same time. > > s/to/two/ Done Thanks.

Re: [PATCH 01/12] qemu-file: We only call qemu_file_transferred_* on the sending side

2023-10-25 Thread Juan Quintela
Fabiano Rosas wrote: > Juan Quintela writes: > >> Remove the increase in qemu_file_fill_buffer() and add asserts to >> qemu_file_transferred* functions. > > Patch looks ok, but I would rewrite the whole commit message like this: > > Don't increment qemu_file_transferred at qemu_file_fill_buffer

Re: [PATCH 12/12] hw/xen: add support for Xen primary console in emulated mode

2023-10-25 Thread David Woodhouse
On Wed, 2023-10-25 at 09:31 +0100, Paul Durrant wrote: > On 24/10/2023 17:34, David Woodhouse wrote: > > On Tue, 2023-10-24 at 17:25 +0100, Paul Durrant wrote: > > > On 24/10/2023 16:49, David Woodhouse wrote: > > > > On Tue, 2023-10-24 at 16:39 +0100, Paul Durrant wrote: > > > > > On 24/10/2023

Re: [PATCH 12/12] hw/xen: add support for Xen primary console in emulated mode

2023-10-25 Thread Paul Durrant
On 24/10/2023 17:34, David Woodhouse wrote: On Tue, 2023-10-24 at 17:25 +0100, Paul Durrant wrote: On 24/10/2023 16:49, David Woodhouse wrote: On Tue, 2023-10-24 at 16:39 +0100, Paul Durrant wrote: On 24/10/2023 16:37, David Woodhouse wrote: On Tue, 2023-10-24 at 15:20 +0100, Paul Durrant

Re: [PATCH 06/12] hw/xen: add get_frontend_path() method to XenDeviceClass

2023-10-25 Thread David Woodhouse
On Tue, 2023-10-24 at 14:37 +0100, Paul Durrant wrote: > > > > Ok, but a prepend to say that a lack of path there means we skip > > > frontend creation seems reasonable? > > > > No, it *is* returning an error. Perhaps I can make it > > > > I understand it is returning an error. I thought the