[PATCH v4 00/12] vhost-user: support any POSIX system (tested on macOS, FreeBSD, OpenBSD)

2024-05-08 Thread Stefano Garzarella
v1: https://patchew.org/QEMU/20240228114759.44758-1-sgarz...@redhat.com/ v2: https://patchew.org/QEMU/20240326133936.125332-1-sgarz...@redhat.com/ v3: https://patchew.org/QEMU/20240404122330.92710-1-sgarz...@redhat.com/ v4: - rebased on master (commit e116b92d01c2cd75957a9f8ad1d4932292867b81) -

[PATCH v4 01/12] libvhost-user: set msg.msg_control to NULL when it is empty

2024-05-08 Thread Stefano Garzarella
On some OS (e.g. macOS) sendmsg() returns -1 (errno EINVAL) if the `struct msghdr` has the field `msg_controllen` set to 0, but `msg_control` is not NULL. Reviewed-by: Eric Blake Reviewed-by: David Hildenbrand Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefano Garzarella --- subproje

[PATCH v4 06/12] contrib/vhost-user-*: use QEMU bswap helper functions

2024-05-08 Thread Stefano Garzarella
Let's replace the calls to le*toh() and htole*() with qemu/bswap.h helpers to make the code more portable. Suggested-by: Philippe Mathieu-Daudé Signed-off-by: Stefano Garzarella --- contrib/vhost-user-blk/vhost-user-blk.c | 9 + contrib/vhost-user-input/main.c | 16 ---

[PATCH v4 07/12] vhost-user: enable frontends on any POSIX system

2024-05-08 Thread Stefano Garzarella
The vhost-user protocol is not really Linux-specific so let's enable vhost-user frontends for any POSIX system. In vhost_net.c we use VHOST_FILE_UNBIND which is defined in a Linux specific header, let's define it for other systems as well. Signed-off-by: Stefano Garzarella --- meson.build

[PATCH v4 09/12] contrib/vhost-user-blk: enable it on any POSIX system

2024-05-08 Thread Stefano Garzarella
Let's make the code more portable by adding defines from block/file-posix.c to support O_DIRECT in other systems (e.g. macOS). vhost-user-server.c is a dependency, let's enable it for any POSIX system. Signed-off-by: Stefano Garzarella --- v4: - moved using of "qemu/bswap.h" API in a separate pa

[PATCH v4 03/12] libvhost-user: mask F_INFLIGHT_SHMFD if memfd is not supported

2024-05-08 Thread Stefano Garzarella
libvhost-user will panic when receiving VHOST_USER_GET_INFLIGHT_FD message if MFD_ALLOW_SEALING is not defined, since it's not able to create a memfd. VHOST_USER_GET_INFLIGHT_FD is used only if VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD is negotiated. So, let's mask that feature if the backend is not ab

[PATCH v4 08/12] libvhost-user: enable it on any POSIX system

2024-05-08 Thread Stefano Garzarella
The vhost-user protocol is not really Linux-specific so let's enable libvhost-user for any POSIX system. Compiling it on macOS and FreeBSD some problems came up: - avoid to include linux/vhost.h which is avaibale only on Linux (vhost_types.h contains many of the things we need) - macOS doesn't p

[PATCH v4 11/12] tests/qtest/vhost-user-blk-test: use memory-backend-shm

2024-05-08 Thread Stefano Garzarella
`memory-backend-memfd` is available only on Linux while the new `memory-backend-shm` can be used on any POSIX-compliant operating system. Let's use it so we can run the test in multiple environments. Signed-off-by: Stefano Garzarella --- tests/qtest/vhost-user-blk-test.c | 2 +- 1 file changed,

[PATCH v4 05/12] contrib/vhost-user-blk: fix bind() using the right size of the address

2024-05-08 Thread Stefano Garzarella
On macOS passing `-s /tmp/vhost.socket` parameter to the vhost-user-blk application, the bind was done on `/tmp/vhost.socke` pathname, missing the last character. This sounds like one of the portability problems described in the unix(7) manpage: Pathname sockets When binding a socket

[PATCH v4 12/12] tests/qtest/vhost-user-test: add a test case for memory-backend-shm

2024-05-08 Thread Stefano Garzarella
`memory-backend-shm` can be used with vhost-user devices, so let's add a new test case for it. Signed-off-by: Stefano Garzarella --- tests/qtest/vhost-user-test.c | 23 +++ 1 file changed, 23 insertions(+) diff --git a/tests/qtest/vhost-user-test.c b/tests/qtest/vhost-user-t

[PATCH v4 10/12] hostmem: add a new memory backend based on POSIX shm_open()

2024-05-08 Thread Stefano Garzarella
shm_open() creates and opens a new POSIX shared memory object. A POSIX shared memory object allows creating memory backend with an associated file descriptor that can be shared with external processes (e.g. vhost-user). The new `memory-backend-shm` can be used as an alternative when `memory-backen

[PATCH v4 02/12] libvhost-user: fail vu_message_write() if sendmsg() is failing

2024-05-08 Thread Stefano Garzarella
In vu_message_write() we use sendmsg() to send the message header, then a write() to send the payload. If sendmsg() fails we should avoid sending the payload, since we were unable to send the header. Discovered before fixing the issue with the previous patch, where sendmsg() failed on macOS due t

[PATCH v4 04/12] vhost-user-server: do not set memory fd non-blocking

2024-05-08 Thread Stefano Garzarella
In vhost-user-server we set all fd received from the other peer in non-blocking mode. For some of them (e.g. memfd, shm_open, etc.) it's not really needed, because we don't use these fd with blocking operations, but only to map memory. In addition, in some systems this operation can fail (e.g. in

Re: [PATCH v4 01/12] libvhost-user: set msg.msg_control to NULL when it is empty

2024-05-08 Thread Daniel P . Berrangé
On Wed, May 08, 2024 at 09:44:45AM +0200, Stefano Garzarella wrote: > On some OS (e.g. macOS) sendmsg() returns -1 (errno EINVAL) if > the `struct msghdr` has the field `msg_controllen` set to 0, but > `msg_control` is not NULL. > > Reviewed-by: Eric Blake > Reviewed-by: David Hildenbrand > Revi

Re: [PATCH v4 02/12] libvhost-user: fail vu_message_write() if sendmsg() is failing

2024-05-08 Thread Daniel P . Berrangé
On Wed, May 08, 2024 at 09:44:46AM +0200, Stefano Garzarella wrote: > In vu_message_write() we use sendmsg() to send the message header, > then a write() to send the payload. > > If sendmsg() fails we should avoid sending the payload, since we > were unable to send the header. > > Discovered befo

Re: [PATCH v4 04/12] vhost-user-server: do not set memory fd non-blocking

2024-05-08 Thread Daniel P . Berrangé
On Wed, May 08, 2024 at 09:44:48AM +0200, Stefano Garzarella wrote: > In vhost-user-server we set all fd received from the other peer > in non-blocking mode. For some of them (e.g. memfd, shm_open, etc.) > it's not really needed, because we don't use these fd with blocking > operations, but only to

Re: [PATCH v4 01/12] libvhost-user: set msg.msg_control to NULL when it is empty

2024-05-08 Thread Stefano Garzarella
On Wed, May 08, 2024 at 09:57:13AM GMT, Daniel P. Berrangé wrote: On Wed, May 08, 2024 at 09:44:45AM +0200, Stefano Garzarella wrote: On some OS (e.g. macOS) sendmsg() returns -1 (errno EINVAL) if the `struct msghdr` has the field `msg_controllen` set to 0, but `msg_control` is not NULL. Review

Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling

2024-05-08 Thread Daniel P . Berrangé
On Tue, May 07, 2024 at 06:52:50AM +0200, Jinpu Wang wrote: > Hi Peter, hi Daniel, > On Mon, May 6, 2024 at 5:29 PM Peter Xu wrote: > > > > On Mon, May 06, 2024 at 12:08:43PM +0200, Jinpu Wang wrote: > > > Hi Peter, hi Daniel, > > > > Hi, Jinpu, > > > > Thanks for sharing this test results. Sound

Re: [PATCH v4 06/12] contrib/vhost-user-*: use QEMU bswap helper functions

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 09:44, Stefano Garzarella wrote: Let's replace the calls to le*toh() and htole*() with qemu/bswap.h helpers to make the code more portable. Suggested-by: Philippe Mathieu-Daudé Signed-off-by: Stefano Garzarella --- contrib/vhost-user-blk/vhost-user-blk.c | 9 + contrib/vh

Re: [RFC 0/1] hw/nvme: add atomic write support

2024-05-08 Thread Klaus Jensen
On Apr 15 16:46, Alan Adamson wrote: > Since there is discussion in the Linux NVMe Driver community to add NVMe > Atomic Write > support, it would be desirable to test it with qemu nvme emulation. > > Initially, this RFC will focus on supporting NVMe controller atomic write > parameters(AWUN, >

Re: [PATCH v4 02/12] libvhost-user: fail vu_message_write() if sendmsg() is failing

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 09:44, Stefano Garzarella wrote: In vu_message_write() we use sendmsg() to send the message header, then a write() to send the payload. If sendmsg() fails we should avoid sending the payload, since we were unable to send the header. Discovered before fixing the issue with the previous

Re: [PATCH v4 01/12] libvhost-user: set msg.msg_control to NULL when it is empty

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 09:44, Stefano Garzarella wrote: On some OS (e.g. macOS) sendmsg() returns -1 (errno EINVAL) if the `struct msghdr` has the field `msg_controllen` set to 0, but `msg_control` is not NULL. Reviewed-by: Eric Blake Reviewed-by: David Hildenbrand Reviewed-by: Philippe Mathieu-Daudé Sign

Re: [PATCH v4 05/12] contrib/vhost-user-blk: fix bind() using the right size of the address

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 09:44, Stefano Garzarella wrote: On macOS passing `-s /tmp/vhost.socket` parameter to the vhost-user-blk application, the bind was done on `/tmp/vhost.socke` pathname, missing the last character. This sounds like one of the portability problems described in the unix(7) manpage: Pa

Re: [PATCH v4 06/12] contrib/vhost-user-*: use QEMU bswap helper functions

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 12:13, Philippe Mathieu-Daudé wrote: On 8/5/24 09:44, Stefano Garzarella wrote: Let's replace the calls to le*toh() and htole*() with qemu/bswap.h helpers to make the code more portable. Suggested-by: Philippe Mathieu-Daudé Signed-off-by: Stefano Garzarella ---   contrib/vhost-user-

Re: [PATCH v4 07/12] vhost-user: enable frontends on any POSIX system

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 09:44, Stefano Garzarella wrote: The vhost-user protocol is not really Linux-specific so let's enable vhost-user frontends for any POSIX system. In vhost_net.c we use VHOST_FILE_UNBIND which is defined in a Linux specific header, let's define it for other systems as well. Signed-off-b

Re: [PATCH v4 09/12] contrib/vhost-user-blk: enable it on any POSIX system

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 09:44, Stefano Garzarella wrote: Let's make the code more portable by adding defines from block/file-posix.c to support O_DIRECT in other systems (e.g. macOS). vhost-user-server.c is a dependency, let's enable it for any POSIX system. Signed-off-by: Stefano Garzarella --- v4: - moved

Re: [PATCH v4 08/12] libvhost-user: enable it on any POSIX system

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 09:44, Stefano Garzarella wrote: The vhost-user protocol is not really Linux-specific so let's enable libvhost-user for any POSIX system. Compiling it on macOS and FreeBSD some problems came up: - avoid to include linux/vhost.h which is avaibale only on Linux "available" (vhost_

Re: [PATCH v4 03/12] libvhost-user: mask F_INFLIGHT_SHMFD if memfd is not supported

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 09:44, Stefano Garzarella wrote: libvhost-user will panic when receiving VHOST_USER_GET_INFLIGHT_FD message if MFD_ALLOW_SEALING is not defined, since it's not able to create a memfd. VHOST_USER_GET_INFLIGHT_FD is used only if VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD is negotiated. So, let

Re: [PATCH v4 00/12] vhost-user: support any POSIX system (tested on macOS, FreeBSD, OpenBSD)

2024-05-08 Thread Philippe Mathieu-Daudé
On 8/5/24 09:44, Stefano Garzarella wrote: The vhost-user protocol is not really Linux-specific, so let's try support QEMU's frontends and backends (including libvhost-user) in any POSIX system with this series. The main use case is to be able to use virtio devices that we don't have built-in in

Re: [PATCH 8/9] hw/nvme: add reservation protocal command

2024-05-08 Thread Klaus Jensen
On May 8 17:36, Changqi Lu wrote: > Add reservation acquire, reservation register, > reservation release and reservation report commands > in the nvme device layer. > > By introducing these commands, this enables the nvme > device to perform reservation-related tasks, including > querying keys, q

[PATCH] virtio-blk: remove SCSI passthrough functionality

2024-05-08 Thread Paolo Bonzini
The legacy SCSI passthrough functionality has never been enabled for VIRTIO 1.0 and was deprecated more than four years ago. Get rid of it---almost, because QEMU is advertising it unconditionally for legacy virtio-blk devices. Just parse the header and return a nonzero status. Signed-off-by: Pao

Re: [PATCH v4 10/12] hostmem: add a new memory backend based on POSIX shm_open()

2024-05-08 Thread Markus Armbruster
Stefano Garzarella writes: > shm_open() creates and opens a new POSIX shared memory object. > A POSIX shared memory object allows creating memory backend with an > associated file descriptor that can be shared with external processes > (e.g. vhost-user). > > The new `memory-backend-shm` can be us

[PATCH v3 3/4] hw/nvme: Allocate sec-ctrl-list as a dynamic array

2024-05-08 Thread Minwoo Im
From: Minwoo Im To prevent further bumping up the number of maximum VF te support, this patch allocates a dynamic array (NvmeCtrl *)->sec_ctrl_list based on number of VF supported by sriov_max_vfs property. Signed-off-by: Minwoo Im --- hw/nvme/ctrl.c | 8 +--- hw/nvme/nvme.h | 5 ++---

[PATCH v3 0/4] hw/nvme: FDP and SR-IOV enhancements

2024-05-08 Thread Minwoo Im
Hello, This patchset has rebased on the latest master and replaced 3rd patch to one which allocates a dynamic array for secondary controller list based on the maximum number of VFs (sriov_max_vfs) rather than a static size of static array as Klaus suggested. Rest of the patchset are the same with

[PATCH v3 2/4] hw/nvme: separate identify data for sec. ctrl list

2024-05-08 Thread Minwoo Im
From: Minwoo Im Secondary controller list for virtualization has been managed by Identify Secondary Controller List data structure with NvmeSecCtrlList where up to 127 secondary controller entries can be managed. The problem hasn't arisen so far because NVME_MAX_VFS has been 127. This patch sep

[PATCH v3 1/4] hw/nvme: add Identify Endurance Group List

2024-05-08 Thread Minwoo Im
From: Minwoo Im Commit 73064edfb864 ("hw/nvme: flexible data placement emulation") intorudced NVMe FDP feature to nvme-subsys and nvme-ctrl with a single endurance group #1 supported. This means that controller should return proper identify data to host with Identify Endurance Group List (CNS 19

[PATCH v3 4/4] hw/nvme: Expand VI/VQ resource to uint32

2024-05-08 Thread Minwoo Im
From: Minwoo Im VI and VQ resources cover queue resources in each VFs in SR-IOV. Current maximum I/O queue pair size is 0x, we can expand them to cover the full number of I/O queue pairs. This patch also fixed Identify Secondary Controller List overflow due to expand of number of secondary c

Re: [PATCH v2 2/4] mirror: allow specifying working bitmap

2024-05-08 Thread Fiona Ebner
Am 07.05.24 um 14:15 schrieb Fiona Ebner: > Am 02.04.24 um 22:14 schrieb Vladimir Sementsov-Ogievskiy: >> On 07.03.24 16:47, Fiona Ebner wrote: >>> +# @bitmap: The name of a bitmap to use as a working bitmap for >>> +# sync=full mode.  This argument must be not be present for other >>> +# s

[PATCH 4/9] scsi/util: add helper functions for persistent reservation types conversion

2024-05-08 Thread Changqi Lu
This commit introduces two helper functions that facilitate the conversion between the persistent reservation types used in the SCSI protocol and those used in the block layer. Signed-off-by: Changqi Lu Signed-off-by: zhenwei pi --- include/scsi/utils.h | 5 + scsi/utils.c | 40 +++

[PATCH 3/9] scsi/constant: add persistent reservation in/out protocol constants

2024-05-08 Thread Changqi Lu
Add constants for the persistent reservation in/out protocol in the scsi/constant module. The constants include the persistent reservation command, type, and scope values defined in sections 6.13 and 6.14 of the SCSI Primary Commands-4 (SPC-4) specification. Signed-off-by: Changqi Lu Signed-off-b

[PATCH 0/9] Support persistent reservation operations

2024-05-08 Thread Changqi Lu
Hi, I am going to introduce persistent reservation for QEMU block. There are three parts in this series: Firstly, at the block layer, the commit abstracts seven APIs related to the persistent reservation command. These APIs including reading keys, reading reservations, registering, reserving, rel

[PATCH 8/9] hw/nvme: add reservation protocal command

2024-05-08 Thread Changqi Lu
Add reservation acquire, reservation register, reservation release and reservation report commands in the nvme device layer. By introducing these commands, this enables the nvme device to perform reservation-related tasks, including querying keys, querying reservation status, registering reservati

[PATCH 5/9] hw/scsi: add persistent reservation in/out api for scsi device

2024-05-08 Thread Changqi Lu
Add persistent reservation in/out operations in the SCSI device layer. By introducing the persistent reservation in/out api, this enables the SCSI device to perform reservation-related tasks, including querying keys, querying reservation status, registering reservation keys, initiating and releasin

[PATCH 6/9] block/nvme: add reservation command protocol constants

2024-05-08 Thread Changqi Lu
Add constants for the NVMe persistent command protocol. The constants include the reservation command opcode and reservation type values defined in section 7 of the NVMe 2.0 specification. Signed-off-by: Changqi Lu Signed-off-by: zhenwei pi --- include/block/nvme.h | 30

[PATCH 1/9] block: add persistent reservation in/out api

2024-05-08 Thread Changqi Lu
Add persistent reservation in/out operations at the block level. The following operations are included: - read_keys:retrieves the list of registered keys. - read_reservation: retrieves the current reservation status. - register: registers a new reservation key. - reserve:

[PATCH 2/9] block/raw: add persistent reservation in/out driver

2024-05-08 Thread Changqi Lu
Add persistent reservation in/out operations for raw driver. The following methods are implemented: bdrv_co_pr_read_keys, bdrv_co_pr_read_reservation, bdrv_co_pr_register, bdrv_co_pr_reserve, bdrv_co_pr_release, bdrv_co_pr_clear and bdrv_co_pr_preempt. Signed-off-by: Changqi Lu Signed-off-by: zhe

[PATCH 7/9] hw/nvme: add helper functions for converting reservation types

2024-05-08 Thread Changqi Lu
This commit introduces two helper functions that facilitate the conversion between the reservation types used in the NVME protocol and those used in the block layer. Signed-off-by: Changqi Lu Signed-off-by: zhenwei pi --- hw/nvme/nvme.h | 40 1 file chan

[PATCH 9/9] block/iscsi: add persistent reservation in/out driver

2024-05-08 Thread Changqi Lu
Add persistent reservation in/out operations for iscsi driver. The following methods are implemented: bdrv_co_pr_read_keys, bdrv_co_pr_read_reservation, bdrv_co_pr_register, bdrv_co_pr_reserve, bdrv_co_pr_release, bdrv_co_pr_clear and bdrv_co_pr_preempt. Signed-off-by: Changqi Lu Signed-off-by: z

Re: [PATCH] hw/nvme: fix mo field in io mgnt send

2024-05-08 Thread Vincent Fu
On 5/7/24 10:05, Vincent Fu wrote: On 5/6/24 04:06, Klaus Jensen wrote: The Management Operation field of I/O Management Send is only 8 bits, not 16. Fixes: 73064edfb864 ("hw/nvme: flexible data placement emulation") Signed-off-by: Klaus Jensen ---   hw/nvme/ctrl.c | 2 +-   1 file changed, 1 i

Re: [PATCH] misc: Use QEMU header path relative to include/ directory

2024-05-08 Thread Philippe Mathieu-Daudé
On 7/5/24 16:27, Philippe Mathieu-Daudé wrote: QEMU headers are relative to the include/ directory, not to the project root directory. Remove "include/". See also: https://www.qemu.org/docs/master/devel/style.html#include-directives Signed-off-by: Philippe Mathieu-Daudé --- hw/audio/virtio-s

Re: [PATCH v3 00/28] glib: Replace g_memdup() by g_memdup2()

2024-05-08 Thread Philippe Mathieu-Daudé
On 3/9/21 19:44, Philippe Mathieu-Daudé wrote: Per https://discourse.gnome.org/t/port-your-module-from-g-memdup-to-g-memdup2-now/5538 The old API took the size of the memory to duplicate as a guint, whereas most memory functions take memory sizes as a gsize. This made it easy to accide