Re: [PATCH v2 00/10] mirror: allow switching from background to active mode

2024-02-28 Thread Markus Armbruster
Vladimir Sementsov-Ogievskiy writes: > On 03.11.23 18:56, Markus Armbruster wrote: >> Is the job abstraction a failure? >> >> We have >> >> block-job- command since job- commandsince >> - >> block-job-set-speed 1.1

Re: [PATCH v8 03/15] pcie_sriov: Reset SR-IOV extended capability

2024-02-28 Thread Akihiko Odaki
On 2024/02/29 1:23, Sriram Yagnaraman wrote: -Original Message- From: Akihiko Odaki Sent: Wednesday, 28 February 2024 12:33 To: Philippe Mathieu-Daudé ; Michael S. Tsirkin ; Marcel Apfelbaum ; Alex Williamson ; Cédric Le Goater ; Paolo Bonzini ; Daniel P. Berrangé ; Eduardo Habkost ;

Re: [PATCH v2 00/10] mirror: allow switching from background to active mode

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
On 03.11.23 18:56, Markus Armbruster wrote: Kevin Wolf writes: Am 03.11.2023 um 10:36 hat Markus Armbruster geschrieben: Vladimir Sementsov-Ogievskiy writes: On 11.10.23 13:18, Fiona Ebner wrote: Am 10.10.23 um 19:55 schrieb Vladimir Sementsov-Ogievskiy: On 09.10.23 12:46, Fiona Ebner wr

RE: [PATCH v8 03/15] pcie_sriov: Reset SR-IOV extended capability

2024-02-28 Thread Sriram Yagnaraman
> -Original Message- > From: Akihiko Odaki > Sent: Wednesday, 28 February 2024 12:33 > To: Philippe Mathieu-Daudé ; Michael S. Tsirkin > ; Marcel Apfelbaum ; > Alex Williamson ; Cédric Le Goater > ; Paolo Bonzini ; Daniel P. > Berrangé ; Eduardo Habkost > ; Sriram Yagnaraman > ; Jason Wa

RE: [PATCH v8 02/15] pcie_sriov: Validate NumVFs

2024-02-28 Thread Sriram Yagnaraman
> -Original Message- > From: Akihiko Odaki > Sent: Wednesday, 28 February 2024 12:33 > To: Philippe Mathieu-Daudé ; Michael S. Tsirkin > ; Marcel Apfelbaum ; > Alex Williamson ; Cédric Le Goater > ; Paolo Bonzini ; Daniel P. > Berrangé ; Eduardo Habkost > ; Sriram Yagnaraman > ; Jason Wa

RE: [PATCH v8 05/15] hw/pci: Always call pcie_sriov_pf_reset()

2024-02-28 Thread Sriram Yagnaraman
> -Original Message- > From: Akihiko Odaki > Sent: Wednesday, 28 February 2024 12:33 > To: Philippe Mathieu-Daudé ; Michael S. Tsirkin > ; Marcel Apfelbaum ; > Alex Williamson ; Cédric Le Goater > ; Paolo Bonzini ; Daniel P. > Berrangé ; Eduardo Habkost > ; Sriram Yagnaraman > ; Jason Wa

Re: [PATCH v2] block-backend: per-device throttling of BLOCK_IO_ERROR reports

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
On 09.01.24 16:13, Vladimir Sementsov-Ogievskiy wrote: From: Leonid Kaplan BLOCK_IO_ERROR events comes from guest, so we must throttle them. We still want per-device throttling, so let's use device id as a key. Signed-off-by: Leonid Kaplan Signed-off-by: Vladimir Sementsov-Ogievskiy ping) --

Re: [PATCH v3] blockcommit: Reopen base image as RO after abort

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
On 09.02.24 15:29, Alexander Ivanov wrote: Could you please review the patch? Sorry for long delay. Honestly, I don't like refcnt in block-driver. It violate incapsulation, refcnt is interal thing of common block layer. And actually, you can't make any assumptions from value of refcnt, as yo

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
On 16.02.24 13:55, Fiona Ebner wrote: Previous discussion from when this was sent upstream [0] (it's been a while). I rebased the patches and re-ordered and squashed like suggested back then [1]. This implements two new mirror modes: - bitmap mirror mode with always/on-success/never bitmap sync

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
On 28.02.24 19:00, Vladimir Sementsov-Ogievskiy wrote: On 16.02.24 13:55, Fiona Ebner wrote: Previous discussion from when this was sent upstream [0] (it's been a while). I rebased the patches and re-ordered and squashed like suggested back then [1]. This implements two new mirror modes: - bit

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
On 16.02.24 13:55, Fiona Ebner wrote: Previous discussion from when this was sent upstream [0] (it's been a while). I rebased the patches and re-ordered and squashed like suggested back then [1]. This implements two new mirror modes: - bitmap mirror mode with always/on-success/never bitmap sync

[PATCH v3 0/5] backup: discard-source parameter

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
Hi all! The main patch is 04, please look at it for description and diagram. v3: 02: new patch 04: take WRITE permission only when discard_source is required Vladimir Sementsov-Ogievskiy (5): block/copy-before-write: fix permission block/copy-before-write: support unligned snapshot-discard

[PATCH v3 1/5] block/copy-before-write: fix permission

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
In case when source node does not have any parents, the condition still works as required: backup job do create the parent by block_job_create -> block_job_add_bdrv -> bdrv_root_attach_child Still, in this case checking @perm variable doesn't work, as backup job creates the root blk with empty

[PATCH v3 3/5] block/copy-before-write: create block_copy bitmap in filter node

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
Currently block_copy creates copy_bitmap in source node. But that is in bad relation with .independent_close=true of copy-before-write filter: source node may be detached and removed before .bdrv_close() handler called, which should call block_copy_state_free(), which in turn should remove copy_bit

[PATCH v3 4/5] qapi: blockdev-backup: add discard-source parameter

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
Add a parameter that enables discard-after-copy. That is mostly useful in "push backup with fleecing" scheme, when source is snapshot-access format driver node, based on copy-before-write filter snapshot-access API: [guest] [snapshot-access] ~~ blockdev-backup ~~> [backup target] |

[PATCH v3 2/5] block/copy-before-write: support unligned snapshot-discard

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
First thing that crashes on unligned access here is bdrv_reset_dirty_bitmap(). Correct way is to align-down the snapshot-discard request. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/copy-before-write.c | 16 +--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git

[PATCH v3 5/5] iotests: add backup-discard-source

2024-02-28 Thread Vladimir Sementsov-Ogievskiy
Add test for a new backup option: discard-source. Signed-off-by: Vladimir Sementsov-Ogievskiy --- .../qemu-iotests/tests/backup-discard-source | 151 ++ .../tests/backup-discard-source.out | 5 + 2 files changed, 156 insertions(+) create mode 100755 tests/qemu-iotes

Re: [PATCH v8 14/15] hw/pci: Determine if rombar is explicitly enabled

2024-02-28 Thread Markus Armbruster
Akihiko Odaki writes: > vfio determines if rombar is explicitly enabled by inspecting QDict. > Inspecting QDict is not nice because QDict is untyped and depends on the > details on the external interface. Add an infrastructure to determine if > rombar is explicitly enabled to hw/pci. > > This cha

Re: [PATCH v8 13/15] hw/pci: Use UINT32_MAX as a default value for rombar

2024-02-28 Thread Markus Armbruster
Akihiko Odaki writes: > Currently there is no way to distinguish the case that rombar is > explicitly specified as 1 and the case that rombar is not specified. > > Set rombar UINT32_MAX by default to distinguish these cases just as it > is done for addr and romsize. It was confirmed that changing

Re: [PATCH 9/9] hostmem-file: support POSIX shm_open()

2024-02-28 Thread Markus Armbruster
Stefano Garzarella writes: > Add a new `shm` bool option for `-object memory-backend-file`. > > When this option is set to true, the POSIX shm_open(3) is used instead > of open(2). > > So a file will not be created in the filesystem, but a "POSIX shared > memory object" will be instantiated. In L

Re: [PATCH v4 12/21] parallels: Let image extensions work in RW mode

2024-02-28 Thread Denis V. Lunev
On 2/28/24 11:25, Alexander Ivanov wrote: On 1/18/24 14:31, Denis V. Lunev wrote: On 1/16/24 15:45, Denis V. Lunev wrote: On 12/28/23 11:12, Alexander Ivanov wrote: Now we support extensions saving and can let to work with them in read-write mode. Signed-off-by: Alexander Ivanov ---   bloc

Re: [PATCH 9/9] hostmem-file: support POSIX shm_open()

2024-02-28 Thread Daniel P . Berrangé
On Wed, Feb 28, 2024 at 12:47:59PM +0100, Stefano Garzarella wrote: > Add a new `shm` bool option for `-object memory-backend-file`. > > When this option is set to true, the POSIX shm_open(3) is used instead > of open(2). > > So a file will not be created in the filesystem, but a "POSIX shared >

Re: [PATCH 9/9] hostmem-file: support POSIX shm_open()

2024-02-28 Thread David Hildenbrand
On 28.02.24 12:47, Stefano Garzarella wrote: Add a new `shm` bool option for `-object memory-backend-file`. When this option is set to true, the POSIX shm_open(3) is used instead of open(2). So a file will not be created in the filesystem, but a "POSIX shared memory object" will be instantiated

Re: [PATCH 0/9] vhost-user: support any POSIX system (tested on macOS and FreeBSD)

2024-02-28 Thread Stefano Garzarella
I just noticed that I forgot to add RFC tag and fix Author to match SOB in some patches, sorry! Stefano On Wed, Feb 28, 2024 at 12:48 PM Stefano Garzarella wrote: > > The vhost-user protocol is not really Linux-specific, so let's try support > QEMU's frontends and backends (including libvhost-us

[PATCH 6/9] vhost-user: enable frontends on any POSIX system

2024-02-28 Thread Stefano Garzarella
The vhost-user protocol is not really Linux-specific so let's enable vhost-user frontends for any POSIX system. In vhost_net.c we use VHOST_FILE_UNBIND which is defined in a Linux specific header, let's define it for other systems as well. Signed-off-by: Stefano Garzarella --- If we want to be m

[PATCH 4/9] vhost-user-server: don't abort if we can't set fd non-blocking

2024-02-28 Thread Stefano Garzarella
From: Stefano Garzarella In vhost-user-server we set all fd received from the other peer in non-blocking mode. For some of them (e.g. memfd, shm_open, etc.) if we fail, it's not really a problem, because we don't use these fd with blocking operations, but only to map memory. In these cases a fai

[PATCH 3/9] libvhost-user: mask F_INFLIGHT_SHMFD if memfd is not supported

2024-02-28 Thread Stefano Garzarella
libvhost-user will panic when receiving VHOST_USER_GET_INFLIGHT_FD message if MFD_ALLOW_SEALING is not defined, since it's not able to create a memfd. VHOST_USER_GET_INFLIGHT_FD is used only if VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD is negotiated. So, let's mask that feature if the backend is not ab

[PATCH 5/9] contrib/vhost-user-blk: fix bind() using the right size of the address

2024-02-28 Thread Stefano Garzarella
From: Stefano Garzarella On macOS passing `-s /tmp/vhost.socket` parameter to the vhost-user-blk application, the bind was done on `/tmp/vhost.socke` pathname, missing the last character. This sounds like one of the portability problems described in the unix(7) manpage: Pathname sockets

[PATCH 7/9] libvhost-user: enable it on any POSIX system

2024-02-28 Thread Stefano Garzarella
The vhost-user protocol is not really Linux-specific so let's enable libvhost-user for any POSIX system. Compiling it on macOS and FreeBSD some problems came up: - avoid to include linux/vhost.h which is avaibale only on Linux (vhost_types.h contains many of the things we need) - macOS doesn't p

[PATCH 9/9] hostmem-file: support POSIX shm_open()

2024-02-28 Thread Stefano Garzarella
Add a new `shm` bool option for `-object memory-backend-file`. When this option is set to true, the POSIX shm_open(3) is used instead of open(2). So a file will not be created in the filesystem, but a "POSIX shared memory object" will be instantiated. In Linux this turns into a file in /dev/shm,

[PATCH 8/9] contrib/vhost-user-blk: enabled it on any POSIX system

2024-02-28 Thread Stefano Garzarella
Let's make the code more portable by using the "qemu/bswap.h" API and adding defines from block/file-posix.c to support O_DIRECT in other systems (e.g. macOS). vhost-user-server.c is a dependency, let's enable it for any POSIX system. Signed-off-by: Stefano Garzarella --- meson.build

[PATCH 0/9] vhost-user: support any POSIX system (tested on macOS and FreeBSD)

2024-02-28 Thread Stefano Garzarella
The vhost-user protocol is not really Linux-specific, so let's try support QEMU's frontends and backends (including libvhost-user) in any POSIX system with this series. The main use case is to be able to use virtio devices that we don't have built-in in QEMU (e.g. virtiofsd, vhost-user-vsock, etc.)

[PATCH 2/9] libvhost-user: fail vu_message_write() if sendmsg() is failing

2024-02-28 Thread Stefano Garzarella
In vu_message_write() we use sendmsg() to send the message header, then a write() to send the payload. If sendmsg() fails we should avoid sending the payload, since we were unable to send the header. Discovered before fixing the issue with the previous patch, where sendmsg() failed on macOS due t

[PATCH 1/9] libvhost-user: set msg.msg_control to NULL when it is empty

2024-02-28 Thread Stefano Garzarella
From: Stefano Garzarella On some OS (e.g. macOS) sendmsg() returns -1 (errno EINVAL) if the `struct msghdr` has the field `msg_controllen` set to 0, but `msg_control` is not NULL. Signed-off-by: Stefano Garzarella --- subprojects/libvhost-user/libvhost-user.c | 1 + 1 file changed, 1 insertion

[PATCH v8 00/15] hw/pci: SR-IOV related fixes and improvements

2024-02-28 Thread Akihiko Odaki
I submitted a RFC series[1] to add support for SR-IOV emulation to virtio-net-pci. During the development of the series, I fixed some trivial bugs and made improvements that I think are independently useful. This series extracts those fixes and improvements from the RFC series. [1]: https://patche

[PATCH v8 02/15] pcie_sriov: Validate NumVFs

2024-02-28 Thread Akihiko Odaki
The guest may write NumVFs greater than TotalVFs and that can lead to buffer overflow in VF implementations. Cc: qemu-sta...@nongnu.org Fixes: CVE-2024-26327 Fixes: 7c0fa8dff811 ("pcie: Add support for Single Root I/O Virtualization (SR/IOV)") Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.

[PATCH v8 14/15] hw/pci: Determine if rombar is explicitly enabled

2024-02-28 Thread Akihiko Odaki
vfio determines if rombar is explicitly enabled by inspecting QDict. Inspecting QDict is not nice because QDict is untyped and depends on the details on the external interface. Add an infrastructure to determine if rombar is explicitly enabled to hw/pci. This changes the semantics of UINT32_MAX, w

[PATCH v8 09/15] pcie_sriov: Release VFs failed to realize

2024-02-28 Thread Akihiko Odaki
Release VFs failed to realize just as we do in unregister_vfs(). Fixes: 7c0fa8dff811 ("pcie: Add support for Single Root I/O Virtualization (SR/IOV)") Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sri

[PATCH v8 10/15] pcie_sriov: Remove num_vfs from PCIESriovPF

2024-02-28 Thread Akihiko Odaki
num_vfs is not migrated so use PCI_SRIOV_CTRL_VFE and PCI_SRIOV_NUM_VF instead. Signed-off-by: Akihiko Odaki --- include/hw/pci/pcie_sriov.h | 1 - hw/pci/pcie_sriov.c | 28 hw/pci/trace-events | 2 +- 3 files changed, 21 insertions(+), 10 deletions

[PATCH v8 15/15] hw/qdev: Remove opts member

2024-02-28 Thread Akihiko Odaki
It is no longer used. Signed-off-by: Akihiko Odaki Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Markus Armbruster --- include/hw/qdev-core.h | 4 hw/core/qdev.c | 1 - system/qdev-monitor.c | 12 +++- 3 files changed, 7 insertions(+), 10 deletions(-) diff --git a/i

[PATCH v8 13/15] hw/pci: Use UINT32_MAX as a default value for rombar

2024-02-28 Thread Akihiko Odaki
Currently there is no way to distinguish the case that rombar is explicitly specified as 1 and the case that rombar is not specified. Set rombar UINT32_MAX by default to distinguish these cases just as it is done for addr and romsize. It was confirmed that changing the default value to UINT32_MAX

[PATCH v8 04/15] pcie_sriov: Do not reset NumVFs after disabling VFs

2024-02-28 Thread Akihiko Odaki
The spec does not NumVFs is reset after disabling VFs except when resetting the PF. Clearing it is guest visible and out of spec, even though Linux doesn't rely on this value being preserved, so we never noticed. Fixes: 7c0fa8dff811 ("pcie: Add support for Single Root I/O Virtualization (SR/IOV)"

[PATCH v8 07/15] pcie_sriov: Do not manually unrealize

2024-02-28 Thread Akihiko Odaki
A device gets automatically unrealized when being unparented. Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 5 - 1 file changed, 5 deletions(-) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index e9b23221d713..8b1fd2a89ad7 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie

[PATCH v8 12/15] hw/pci: Replace -1 with UINT32_MAX for romsize

2024-02-28 Thread Akihiko Odaki
romsize is an uint32_t variable. Specifying -1 as an uint32_t value is obscure way to denote UINT32_MAX. Worse, if int is wider than 32-bit, it will change the behavior of a construct like the following: romsize = -1; if (romsize != -1) { ... } When -1 is assigned to romsize, -1 will be impli

[PATCH v8 08/15] pcie_sriov: Reuse SR-IOV VF device instances

2024-02-28 Thread Akihiko Odaki
Disable SR-IOV VF devices by reusing code to power down PCI devices instead of removing them when the guest requests to disable VFs. This allows to realize devices and report VF realization errors at PF realization time. Signed-off-by: Akihiko Odaki --- docs/pcie_sriov.txt | 8 ++-- in

[PATCH v8 06/15] hw/pci: Rename has_power to enabled

2024-02-28 Thread Akihiko Odaki
The renamed state will not only represent powering state of PFs, but also represent SR-IOV VF enablement in the future. Signed-off-by: Akihiko Odaki --- include/hw/pci/pci.h| 7 ++- include/hw/pci/pci_device.h | 2 +- hw/pci/pci.c| 14 +++--- hw/pci/pci_host

[PATCH v8 11/15] pcie_sriov: Register VFs after migration

2024-02-28 Thread Akihiko Odaki
pcie_sriov doesn't have code to restore its state after migration, but igb, which uses pcie_sriov, naively claimed its migration capability. Add code to register VFs after migration and fix igb migration. Fixes: 3a977deebe6b ("Intrdocue igb device emulation") Signed-off-by: Akihiko Odaki --- in

[PATCH v8 05/15] hw/pci: Always call pcie_sriov_pf_reset()

2024-02-28 Thread Akihiko Odaki
Call pcie_sriov_pf_reset() from pci_do_device_reset() just as we do for msi_reset() and msix_reset() to prevent duplicating code for each SR-IOV PF. Signed-off-by: Akihiko Odaki --- hw/net/igb.c | 2 -- hw/nvme/ctrl.c | 4 hw/pci/pci.c | 1 + 3 files changed, 1 insertion(+), 6 deletions

[PATCH v8 01/15] hw/nvme: Use pcie_sriov_num_vfs()

2024-02-28 Thread Akihiko Odaki
nvme_sriov_pre_write_ctrl() used to directly inspect SR-IOV configurations to know the number of VFs being disabled due to SR-IOV configuration writes, but the logic was flawed and resulted in out-of-bound memory access. It assumed PCI_SRIOV_NUM_VF always has the number of currently enabled VFs, b

[PATCH v8 03/15] pcie_sriov: Reset SR-IOV extended capability

2024-02-28 Thread Akihiko Odaki
pcie_sriov_pf_disable_vfs() is called when resetting the PF, but it only disables VFs and does not reset SR-IOV extended capability, leaking the state and making the VF Enable register inconsistent with the actual state. Replace pcie_sriov_pf_disable_vfs() with pcie_sriov_pf_reset(), which does no

Re: [PATCH v4 12/21] parallels: Let image extensions work in RW mode

2024-02-28 Thread Alexander Ivanov
On 1/18/24 14:31, Denis V. Lunev wrote: On 1/16/24 15:45, Denis V. Lunev wrote: On 12/28/23 11:12, Alexander Ivanov wrote: Now we support extensions saving and can let to work with them in read-write mode. Signed-off-by: Alexander Ivanov ---   block/parallels-ext.c |  4   block/parall

Re: [PATCH v7 15/16] vfio: Avoid inspecting option QDict for rombar

2024-02-28 Thread Markus Armbruster
Akihiko Odaki writes: > Use pci_rom_bar_explicitly_enabled() to determine if rombar is explicitly > enabled. > > Signed-off-by: Akihiko Odaki > --- > hw/vfio/pci.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c > index 4fa387f0430d..647f

Re: [PATCH v7 14/16] hw/pci: Determine if rombar is explicitly enabled

2024-02-28 Thread Markus Armbruster
Akihiko Odaki writes: > vfio determines if rombar is explicitly enabled by inspecting QDict. > Inspecting QDict is not nice because QDict is untyped and depends on the > details on the external interface. Add an infrastructure to determine if > rombar is explicitly enabled to hw/pci. > > Signed-o

Re: [PATCH v7 12/16] hw/pci: Replace -1 with UINT32_MAX for romsize

2024-02-28 Thread Markus Armbruster
Akihiko Odaki writes: > romsize is an uint32_t variable. Specifying -1 as an uint32_t value is > obscure way to denote UINT32_MAX. > > Worse, if int is wider than 32-bit, it will change the behavior of a > construct like the following: > romsize = -1; > if (romsize != -1) { > ... > } > > When