[PATCH v2 2/4] scripts/qapi: allow optional experimental enum values

2023-04-19 Thread Vladimir Sementsov-Ogievskiy
To be used in the next commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- scripts/qapi/types.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py index c39d054d2c..18f8734047 100644 --- a/scripts/qapi/types.py +++ b/scripts/qapi/types.py @@

[PATCH v2 1/4] block/meson.build: prefer positive condition for replication

2023-04-19 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/meson.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/meson.build b/block/meson.build index 382bec0e7d..b9a72e219b 100644 --- a/block/meson.build +++ b/block/meson.build @@ -84,7 +84,7 @@ block_ss.add(when:

[PATCH v2 4/4] configure: add --disable-colo-filters option

2023-04-19 Thread Vladimir Sementsov-Ogievskiy
Add option to not build COLO Proxy subsystem if it is not needed. Signed-off-by: Vladimir Sementsov-Ogievskiy --- meson.build | 1 + meson_options.txt | 2 ++ net/meson.build | 11 --- scripts/meson-buildoptions.sh | 3 +++ 4 files changed,

[PATCH v2 3/4] build: move COLO under CONFIG_REPLICATION

2023-04-19 Thread Vladimir Sementsov-Ogievskiy
We don't allow to use x-colo capability when replication is not configured. So, no reason to build COLO when replication is disabled, it's unusable in this case. Signed-off-by: Vladimir Sementsov-Ogievskiy --- hmp-commands.hx| 2 ++ migration/colo.c | 6 +

[PATCH v2 0/4] COLO: improve build options

2023-04-19 Thread Vladimir Sementsov-Ogievskiy
Hi all! COLO substem seems to be useless when CONFIG_REPLICATION is unset, as we simply don't allow to set x-colo capability in this case. So, let's not compile in unreachable code and interface we cannot use when CONFIG_REPLICATION is unset. Also, provide personal configure option for COLO

Re: [PATCH] replication: compile out some staff when replication is not configured

2023-04-19 Thread Vladimir Sementsov-Ogievskiy
On 11.04.23 17:51, Vladimir Sementsov-Ogievskiy wrote: Don't compile-in replication-related files when replication is disabled in config. Signed-off-by: Vladimir Sementsov-Ogievskiy v2 will be called "[PATCH v2 0/4] COLO: improve build options" -- Best regards, Vladimir

Re: [PATCH] replication: compile out some staff when replication is not configured

2023-04-19 Thread Vladimir Sementsov-Ogievskiy
On 17.04.23 20:19, Vladimir Sementsov-Ogievskiy wrote: On 16.04.23 21:44, Zhang, Chen wrote: -Original Message- From: Vladimir Sementsov-Ogievskiy Sent: Friday, April 14, 2023 5:51 PM To: Zhang, Chen ; qemu-de...@nongnu.org Cc: qemu-block@nongnu.org; pbonz...@redhat.com;

Re: [PATCH v2 16/16] virtio: make it possible to detach host notifier from any thread

2023-04-19 Thread Eric Blake
On Wed, Apr 19, 2023 at 01:28:17PM -0400, Stefan Hajnoczi wrote: > virtio_queue_aio_detach_host_notifier() does two things: > 1. It removes the fd handler from the event loop. > 2. It processes the virtqueue one last time. > > The first step can be peformed by any thread and without taking the >

[PATCH v2 05/16] util/vhost-user-server: rename refcount to in_flight counter

2023-04-19 Thread Stefan Hajnoczi
The VuServer object has a refcount field and ref/unref APIs. The name is confusing because it's actually an in-flight request counter instead of a refcount. Normally a refcount destroys the object upon reaching zero. The VuServer counter is used to wake up the vhost-user coroutine when there are

[PATCH v2 13/16] block/export: rewrite vduse-blk drain code

2023-04-19 Thread Stefan Hajnoczi
vduse_blk_detach_ctx() waits for in-flight requests using AIO_WAIT_WHILE(). This is not allowed according to a comment in bdrv_set_aio_context_commit(): /* * Take the old AioContex when detaching it from bs. * At this point, new_context lock is already acquired, and we are now * also

[PATCH v2 02/16] virtio-scsi: avoid race between unplug and transport event

2023-04-19 Thread Stefan Hajnoczi
Only report a transport reset event to the guest after the SCSIDevice has been unrealized by qdev_simple_device_unplug_cb(). qdev_simple_device_unplug_cb() sets the SCSIDevice's qdev.realized field to false so that scsi_device_find/get() no longer see it. scsi_target_emulate_report_luns() also

[PATCH v2 08/16] hw/xen: do not use aio_set_fd_handler(is_external=true) in xen_xenstore

2023-04-19 Thread Stefan Hajnoczi
There is no need to suspend activity between aio_disable_external() and aio_enable_external(), which is mainly used for the block layer's drain operation. This is part of ongoing work to remove the aio_disable_external() API. Reviewed-by: David Woodhouse Signed-off-by: Stefan Hajnoczi ---

[PATCH v2 03/16] virtio-scsi: stop using aio_disable_external() during unplug

2023-04-19 Thread Stefan Hajnoczi
This patch is part of an effort to remove the aio_disable_external() API because it does not fit in a multi-queue block layer world where many AioContexts may be submitting requests to the same disk. The SCSI emulation code is already in good shape to stop using aio_disable_external(). It was

[PATCH v2 09/16] block: add blk_in_drain() API

2023-04-19 Thread Stefan Hajnoczi
The BlockBackend quiesce_counter is greater than zero during drained sections. Add an API to check whether the BlockBackend is in a drained section. The next patch will use this API. Signed-off-by: Stefan Hajnoczi --- include/sysemu/block-backend-global-state.h | 1 + block/block-backend.c

[PATCH v2 06/16] block/export: wait for vhost-user-blk requests when draining

2023-04-19 Thread Stefan Hajnoczi
Each vhost-user-blk request runs in a coroutine. When the BlockBackend enters a drained section we need to enter a quiescent state. Currently any in-flight requests race with bdrv_drained_begin() because it is unaware of vhost-user-blk requests. When blk_co_preadv/pwritev()/etc returns it wakes

[PATCH v2 12/16] hw/xen: do not set is_external=true on evtchn fds

2023-04-19 Thread Stefan Hajnoczi
is_external=true suspends fd handlers between aio_disable_external() and aio_enable_external(). The block layer's drain operation uses this mechanism to prevent new I/O from sneaking in between bdrv_drained_begin() and bdrv_drained_end(). The previous commit converted the xen-block device to use

[PATCH v2 15/16] block/fuse: do not set is_external=true on FUSE fd

2023-04-19 Thread Stefan Hajnoczi
This is part of ongoing work to remove the aio_disable_external() API. Use BlockDevOps .drained_begin/end/poll() instead of aio_set_fd_handler(is_external=true). As a side-effect the FUSE export now follows AioContext changes like the other export types. Signed-off-by: Stefan Hajnoczi ---

[PATCH v2 16/16] virtio: make it possible to detach host notifier from any thread

2023-04-19 Thread Stefan Hajnoczi
virtio_queue_aio_detach_host_notifier() does two things: 1. It removes the fd handler from the event loop. 2. It processes the virtqueue one last time. The first step can be peformed by any thread and without taking the AioContext lock. The second step may need the AioContext lock (depending on

[PATCH v2 11/16] xen-block: implement BlockDevOps->drained_begin()

2023-04-19 Thread Stefan Hajnoczi
Detach event channels during drained sections to stop I/O submission from the ring. xen-block is no longer reliant on aio_disable_external() after this patch. This will allow us to remove the aio_disable_external() API once all other code that relies on it is converted. Extend

[PATCH v2 07/16] block/export: stop using is_external in vhost-user-blk server

2023-04-19 Thread Stefan Hajnoczi
vhost-user activity must be suspended during bdrv_drained_begin/end(). This prevents new requests from interfering with whatever is happening in the drained section. Previously this was done using aio_set_fd_handler()'s is_external argument. In a multi-queue block layer world the

[PATCH v2 00/16] block: remove aio_disable_external() API

2023-04-19 Thread Stefan Hajnoczi
v2: - Do not rely on BlockBackend request queuing, implement .drained_begin/end() instead in xen-block, virtio-blk, and virtio-scsi [Paolo] - Add qdev_is_realized() API [Philippe] - Add patch to avoid AioContext lock around blk_exp_ref/unref() [Paolo] - Add patch to call .drained_begin/end()

[PATCH v2 14/16] block/export: don't require AioContext lock around blk_exp_ref/unref()

2023-04-19 Thread Stefan Hajnoczi
The FUSE export calls blk_exp_ref/unref() without the AioContext lock. Instead of fixing the FUSE export, adjust blk_exp_ref/unref() so they work without the AioContext lock. This way it's less error-prone. Suggested-by: Paolo Bonzini Signed-off-by: Stefan Hajnoczi --- include/block/export.h

[PATCH v2 10/16] block: drain from main loop thread in bdrv_co_yield_to_drain()

2023-04-19 Thread Stefan Hajnoczi
For simplicity, always run BlockDevOps .drained_begin/end/poll() callbacks in the main loop thread. This makes it easier to implement the callbacks and avoids extra locks. Move the function pointer declarations from the I/O Code section to the Global State section in block-backend-common.h.

[PATCH v2 04/16] block/export: only acquire AioContext once for vhost_user_server_stop()

2023-04-19 Thread Stefan Hajnoczi
vhost_user_server_stop() uses AIO_WAIT_WHILE(). AIO_WAIT_WHILE() requires that AioContext is only acquired once. Since blk_exp_request_shutdown() already acquires the AioContext it shouldn't be acquired again in vhost_user_server_stop(). Signed-off-by: Stefan Hajnoczi ---

[PATCH v2 01/16] hw/qdev: introduce qdev_is_realized() helper

2023-04-19 Thread Stefan Hajnoczi
Add a helper function to check whether the device is realized without requiring the Big QEMU Lock. The next patch adds a second caller. The goal is to avoid spreading DeviceState field accesses throughout the code. Suggested-by: Philippe Mathieu-Daudé Signed-off-by: Stefan Hajnoczi ---

Re: QMP (without OOB) function running in thread different from the main thread as part of aio_poll

2023-04-19 Thread Paolo Bonzini
Il mer 19 apr 2023, 16:11 Fiona Ebner ha scritto: Hi, while debugging a completely different issue, I was surprised to see do_qmp_dispatch_bh being run in a vCPU thread. I was under the impression that QMP functions are supposed to be executed in the main thread. Is that wrong While monitor

QMP (without OOB) function running in thread different from the main thread as part of aio_poll

2023-04-19 Thread Fiona Ebner
Hi, while debugging a completely different issue, I was surprised to see do_qmp_dispatch_bh being run in a vCPU thread. I was under the impression that QMP functions are supposed to be executed in the main thread. Is that wrong? I managed to reproduced the scenario with a build of upstream QEMU