Re: [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library
On Tue, Feb 08, 2022 at 02:42:41PM +0800, Yongji Xie wrote:
> On Mon, Feb 7, 2022 at 10:01 PM Stefan Hajnoczi wrote:
> >
> > On Tue, Jan 25, 2022 at 09:17:57PM +0800, Xie Yongji wrote:
> > > +int vduse_dev_handler(VduseDev *dev)
> > > +{
> > > +struct vduse_dev_request req;
> > > +struct vduse_dev_response resp = { 0 };
> > > +VduseVirtq *vq;
> > > +int i, ret;
> > > +
> > > +ret = read(dev->fd, , sizeof(req));
> >
> > This file descriptor is blocking? I guess the assumption is that the
> > kernel VDUSE code always enqueues at least one struct vduse_dev_request,
> > so userspace will not block when the file descriptor becomes readable?
> >
>
> Yes, that's true. We can always get one entire request if the file
> descriptor becomes readable.
Okay, then the code is fine. We trust the kernel not to block us. While
it is possible to get spurious select(2)/poll(2) ready file descriptors
in the general case (e.g. multiple processes monitoring the same file),
I don't think that can happen in this case.
signature.asc
Description: PGP signature
Re: [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library
On Mon, Feb 7, 2022 at 10:01 PM Stefan Hajnoczi wrote:
>
> On Tue, Jan 25, 2022 at 09:17:57PM +0800, Xie Yongji wrote:
> > VDUSE [1] is a linux framework that makes it possible to implement
> > software-emulated vDPA devices in userspace. This adds a library
> > as a subproject to help implementing VDUSE backends in QEMU.
> >
> > [1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
>
> This library assumes that the program is allowed to access the control
> device (/dev/vduse/control). Is that always the case or should the
> library also support access to /dev/vduse/ only (maybe even with
> file descriptor passing) so a privileged process can create/destroy
> VDUSE devices?
>
Make sense. I will add two new API to support these two cases.
> I didn't review the vring code in detail.
>
> >
> > Signed-off-by: Xie Yongji
> > ---
> > meson.build | 15 +
> > meson_options.txt |2 +
> > scripts/meson-buildoptions.sh |3 +
> > subprojects/libvduse/include/atomic.h |1 +
> > subprojects/libvduse/libvduse.c | 1025 +++
> > subprojects/libvduse/libvduse.h | 193
> > subprojects/libvduse/meson.build| 10 +
> > subprojects/libvduse/standard-headers/linux |1 +
> > 8 files changed, 1250 insertions(+)
> > create mode 12 subprojects/libvduse/include/atomic.h
> > create mode 100644 subprojects/libvduse/libvduse.c
> > create mode 100644 subprojects/libvduse/libvduse.h
> > create mode 100644 subprojects/libvduse/meson.build
> > create mode 12 subprojects/libvduse/standard-headers/linux
> >
> > diff --git a/meson.build b/meson.build
> > index 333c61deba..864fb50ade 100644
> > --- a/meson.build
> > +++ b/meson.build
> > @@ -1305,6 +1305,21 @@ if not get_option('fuse_lseek').disabled()
> >endif
> > endif
> >
> > +have_libvduse = (targetos == 'linux')
> > +if get_option('libvduse').enabled()
> > +if targetos != 'linux'
> > +error('libvduse requires linux')
> > +endif
> > +elif get_option('libvduse').disabled()
> > +have_libvduse = false
> > +endif
> > +
> > +libvduse = not_found
> > +if have_libvduse
> > + libvduse_proj = subproject('libvduse')
> > + libvduse = libvduse_proj.get_variable('libvduse_dep')
> > +endif
> > +
> > # libbpf
> > libbpf = dependency('libbpf', required: get_option('bpf'), method:
> > 'pkg-config')
> > if libbpf.found() and not cc.links('''
> > diff --git a/meson_options.txt b/meson_options.txt
> > index 921967eddb..16790d1814 100644
> > --- a/meson_options.txt
> > +++ b/meson_options.txt
> > @@ -195,6 +195,8 @@ option('virtfs', type: 'feature', value: 'auto',
> > description: 'virtio-9p support')
> > option('virtiofsd', type: 'feature', value: 'auto',
> > description: 'build virtiofs daemon (virtiofsd)')
> > +option('libvduse', type: 'feature', value: 'auto',
> > + description: 'build VDUSE Library')
> >
> > option('capstone', type: 'combo', value: 'auto',
> > choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
> > diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
> > index a4af02c527..af5c75d758 100644
> > --- a/scripts/meson-buildoptions.sh
> > +++ b/scripts/meson-buildoptions.sh
> > @@ -58,6 +58,7 @@ meson_options_help() {
> >printf "%s\n" ' libssh ssh block device support'
> >printf "%s\n" ' libudev Use libudev to enumerate host devices'
> >printf "%s\n" ' libusb libusb support for USB passthrough'
> > + printf "%s\n" ' libvdusebuild VDUSE Library'
> >printf "%s\n" ' libxml2 libxml2 support for Parallels image
> > format'
> >printf "%s\n" ' linux-aio Linux AIO support'
> >printf "%s\n" ' linux-io-uring Linux io_uring support'
> > @@ -188,6 +189,8 @@ _meson_option_parse() {
> > --disable-libudev) printf "%s" -Dlibudev=disabled ;;
> > --enable-libusb) printf "%s" -Dlibusb=enabled ;;
> > --disable-libusb) printf "%s" -Dlibusb=disabled ;;
> > +--enable-libvduse) printf "%s" -Dlibvduse=enabled ;;
> > +--disable-libvduse) printf "%s" -Dlibvduse=disabled ;;
> > --enable-libxml2) printf "%s" -Dlibxml2=enabled ;;
> > --disable-libxml2) printf "%s" -Dlibxml2=disabled ;;
> > --enable-linux-aio) printf "%s" -Dlinux_aio=enabled ;;
> > diff --git a/subprojects/libvduse/include/atomic.h
> > b/subprojects/libvduse/include/atomic.h
> > new file mode 12
> > index 00..8c2be64f7b
> > --- /dev/null
> > +++ b/subprojects/libvduse/include/atomic.h
> > @@ -0,0 +1 @@
> > +../../../include/qemu/atomic.h
> > \ No newline at end of file
> > diff --git a/subprojects/libvduse/libvduse.c
> > b/subprojects/libvduse/libvduse.c
> > new file mode 100644
> > index 00..7671864bca
> > --- /dev/null
> > +++ b/subprojects/libvduse/libvduse.c
> > @@ -0,0 +1,1025 @@
> > +/*
> > + * VDUSE (vDPA Device in Userspace)
Re: [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library
On Tue, Jan 25, 2022 at 09:17:57PM +0800, Xie Yongji wrote:
> VDUSE [1] is a linux framework that makes it possible to implement
> software-emulated vDPA devices in userspace. This adds a library
> as a subproject to help implementing VDUSE backends in QEMU.
>
> [1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
This library assumes that the program is allowed to access the control
device (/dev/vduse/control). Is that always the case or should the
library also support access to /dev/vduse/ only (maybe even with
file descriptor passing) so a privileged process can create/destroy
VDUSE devices?
I didn't review the vring code in detail.
>
> Signed-off-by: Xie Yongji
> ---
> meson.build | 15 +
> meson_options.txt |2 +
> scripts/meson-buildoptions.sh |3 +
> subprojects/libvduse/include/atomic.h |1 +
> subprojects/libvduse/libvduse.c | 1025 +++
> subprojects/libvduse/libvduse.h | 193
> subprojects/libvduse/meson.build| 10 +
> subprojects/libvduse/standard-headers/linux |1 +
> 8 files changed, 1250 insertions(+)
> create mode 12 subprojects/libvduse/include/atomic.h
> create mode 100644 subprojects/libvduse/libvduse.c
> create mode 100644 subprojects/libvduse/libvduse.h
> create mode 100644 subprojects/libvduse/meson.build
> create mode 12 subprojects/libvduse/standard-headers/linux
>
> diff --git a/meson.build b/meson.build
> index 333c61deba..864fb50ade 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1305,6 +1305,21 @@ if not get_option('fuse_lseek').disabled()
>endif
> endif
>
> +have_libvduse = (targetos == 'linux')
> +if get_option('libvduse').enabled()
> +if targetos != 'linux'
> +error('libvduse requires linux')
> +endif
> +elif get_option('libvduse').disabled()
> +have_libvduse = false
> +endif
> +
> +libvduse = not_found
> +if have_libvduse
> + libvduse_proj = subproject('libvduse')
> + libvduse = libvduse_proj.get_variable('libvduse_dep')
> +endif
> +
> # libbpf
> libbpf = dependency('libbpf', required: get_option('bpf'), method:
> 'pkg-config')
> if libbpf.found() and not cc.links('''
> diff --git a/meson_options.txt b/meson_options.txt
> index 921967eddb..16790d1814 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -195,6 +195,8 @@ option('virtfs', type: 'feature', value: 'auto',
> description: 'virtio-9p support')
> option('virtiofsd', type: 'feature', value: 'auto',
> description: 'build virtiofs daemon (virtiofsd)')
> +option('libvduse', type: 'feature', value: 'auto',
> + description: 'build VDUSE Library')
>
> option('capstone', type: 'combo', value: 'auto',
> choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
> diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
> index a4af02c527..af5c75d758 100644
> --- a/scripts/meson-buildoptions.sh
> +++ b/scripts/meson-buildoptions.sh
> @@ -58,6 +58,7 @@ meson_options_help() {
>printf "%s\n" ' libssh ssh block device support'
>printf "%s\n" ' libudev Use libudev to enumerate host devices'
>printf "%s\n" ' libusb libusb support for USB passthrough'
> + printf "%s\n" ' libvdusebuild VDUSE Library'
>printf "%s\n" ' libxml2 libxml2 support for Parallels image
> format'
>printf "%s\n" ' linux-aio Linux AIO support'
>printf "%s\n" ' linux-io-uring Linux io_uring support'
> @@ -188,6 +189,8 @@ _meson_option_parse() {
> --disable-libudev) printf "%s" -Dlibudev=disabled ;;
> --enable-libusb) printf "%s" -Dlibusb=enabled ;;
> --disable-libusb) printf "%s" -Dlibusb=disabled ;;
> +--enable-libvduse) printf "%s" -Dlibvduse=enabled ;;
> +--disable-libvduse) printf "%s" -Dlibvduse=disabled ;;
> --enable-libxml2) printf "%s" -Dlibxml2=enabled ;;
> --disable-libxml2) printf "%s" -Dlibxml2=disabled ;;
> --enable-linux-aio) printf "%s" -Dlinux_aio=enabled ;;
> diff --git a/subprojects/libvduse/include/atomic.h
> b/subprojects/libvduse/include/atomic.h
> new file mode 12
> index 00..8c2be64f7b
> --- /dev/null
> +++ b/subprojects/libvduse/include/atomic.h
> @@ -0,0 +1 @@
> +../../../include/qemu/atomic.h
> \ No newline at end of file
> diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c
> new file mode 100644
> index 00..7671864bca
> --- /dev/null
> +++ b/subprojects/libvduse/libvduse.c
> @@ -0,0 +1,1025 @@
> +/*
> + * VDUSE (vDPA Device in Userspace) library
> + *
> + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights
> reserved.
> + * Portions of codes and concepts borrowed from libvhost-user.c, so:
> + * Copyright IBM, Corp. 2007
> + * Copyright (c) 2016 Red Hat, Inc.
> + *
> + * Author:
> + * Xie Yongji
> + * Anthony Liguori
> + * Marc-André Lureau
>
[PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library
VDUSE [1] is a linux framework that makes it possible to implement
software-emulated vDPA devices in userspace. This adds a library
as a subproject to help implementing VDUSE backends in QEMU.
[1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
Signed-off-by: Xie Yongji
---
meson.build | 15 +
meson_options.txt |2 +
scripts/meson-buildoptions.sh |3 +
subprojects/libvduse/include/atomic.h |1 +
subprojects/libvduse/libvduse.c | 1025 +++
subprojects/libvduse/libvduse.h | 193
subprojects/libvduse/meson.build| 10 +
subprojects/libvduse/standard-headers/linux |1 +
8 files changed, 1250 insertions(+)
create mode 12 subprojects/libvduse/include/atomic.h
create mode 100644 subprojects/libvduse/libvduse.c
create mode 100644 subprojects/libvduse/libvduse.h
create mode 100644 subprojects/libvduse/meson.build
create mode 12 subprojects/libvduse/standard-headers/linux
diff --git a/meson.build b/meson.build
index 333c61deba..864fb50ade 100644
--- a/meson.build
+++ b/meson.build
@@ -1305,6 +1305,21 @@ if not get_option('fuse_lseek').disabled()
endif
endif
+have_libvduse = (targetos == 'linux')
+if get_option('libvduse').enabled()
+if targetos != 'linux'
+error('libvduse requires linux')
+endif
+elif get_option('libvduse').disabled()
+have_libvduse = false
+endif
+
+libvduse = not_found
+if have_libvduse
+ libvduse_proj = subproject('libvduse')
+ libvduse = libvduse_proj.get_variable('libvduse_dep')
+endif
+
# libbpf
libbpf = dependency('libbpf', required: get_option('bpf'), method:
'pkg-config')
if libbpf.found() and not cc.links('''
diff --git a/meson_options.txt b/meson_options.txt
index 921967eddb..16790d1814 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -195,6 +195,8 @@ option('virtfs', type: 'feature', value: 'auto',
description: 'virtio-9p support')
option('virtiofsd', type: 'feature', value: 'auto',
description: 'build virtiofs daemon (virtiofsd)')
+option('libvduse', type: 'feature', value: 'auto',
+ description: 'build VDUSE Library')
option('capstone', type: 'combo', value: 'auto',
choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index a4af02c527..af5c75d758 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -58,6 +58,7 @@ meson_options_help() {
printf "%s\n" ' libssh ssh block device support'
printf "%s\n" ' libudev Use libudev to enumerate host devices'
printf "%s\n" ' libusb libusb support for USB passthrough'
+ printf "%s\n" ' libvdusebuild VDUSE Library'
printf "%s\n" ' libxml2 libxml2 support for Parallels image format'
printf "%s\n" ' linux-aio Linux AIO support'
printf "%s\n" ' linux-io-uring Linux io_uring support'
@@ -188,6 +189,8 @@ _meson_option_parse() {
--disable-libudev) printf "%s" -Dlibudev=disabled ;;
--enable-libusb) printf "%s" -Dlibusb=enabled ;;
--disable-libusb) printf "%s" -Dlibusb=disabled ;;
+--enable-libvduse) printf "%s" -Dlibvduse=enabled ;;
+--disable-libvduse) printf "%s" -Dlibvduse=disabled ;;
--enable-libxml2) printf "%s" -Dlibxml2=enabled ;;
--disable-libxml2) printf "%s" -Dlibxml2=disabled ;;
--enable-linux-aio) printf "%s" -Dlinux_aio=enabled ;;
diff --git a/subprojects/libvduse/include/atomic.h
b/subprojects/libvduse/include/atomic.h
new file mode 12
index 00..8c2be64f7b
--- /dev/null
+++ b/subprojects/libvduse/include/atomic.h
@@ -0,0 +1 @@
+../../../include/qemu/atomic.h
\ No newline at end of file
diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c
new file mode 100644
index 00..7671864bca
--- /dev/null
+++ b/subprojects/libvduse/libvduse.c
@@ -0,0 +1,1025 @@
+/*
+ * VDUSE (vDPA Device in Userspace) library
+ *
+ * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights
reserved.
+ * Portions of codes and concepts borrowed from libvhost-user.c, so:
+ * Copyright IBM, Corp. 2007
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * Author:
+ * Xie Yongji
+ * Anthony Liguori
+ * Marc-André Lureau
+ * Victor Kaplansky
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+
+#include "include/atomic.h"
+#include "standard-headers/linux/vhost_types.h"
+#include "standard-headers/linux/vduse.h"
+#include "libvduse.h"
+
+#define VIRTQUEUE_MAX_SIZE 1024
+#define VDUSE_VQ_ALIGN 4096
+#define MAX_IOVA_REGIONS 256
+
+/* Round number down to multiple */
+#define
