[PATCH v6 20/20] aio: remove aio_disable_external() API

2023-05-16 Thread Stefan Hajnoczi
, is_external, io_read, io_poll, io_poll_ready; @@ - aio_set_event_notifier(ctx, notifier, is_external, io_read, io_poll, io_poll_ready) + aio_set_event_notifier(ctx, notifier, io_read, io_poll, io_poll_ready) Reviewed-by: Juan Quintela Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefan

[PATCH v6 07/20] block/export: stop using is_external in vhost-user-blk server

2023-05-16 Thread Stefan Hajnoczi
the aio_disable_external() API cannot be used since multiple AioContext may be processing I/O, not just one. Switch to BlockDevOps->drained_begin/end() callbacks. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- block/export/vhost-user-blk-server.c | 28 ++-- util/vhost-u

[PATCH v6 03/20] virtio-scsi: avoid race between unplug and transport event

2023-05-16 Thread Stefan Hajnoczi
. These changes ensure that the guest driver does not see the SCSIDevice that's being unplugged if it responds very quickly to the transport reset event. Reviewed-by: Paolo Bonzini Reviewed-by: Michael S. Tsirkin Reviewed-by: Daniil Tatianin Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf

[PATCH v6 04/20] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-16 Thread Stefan Hajnoczi
equests remain once qdev_simple_device_unplug_cb() returns. Thanks to these two conditions we don't need aio_disable_external() anymore. Cc: Zhengui Li Reviewed-by: Paolo Bonzini Reviewed-by: Daniil Tatianin Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- hw/scsi/virtio-scsi.c | 3 --- 1 file c

[PATCH v6 06/20] block/export: wait for vhost-user-blk requests when draining

2023-05-16 Thread Stefan Hajnoczi
there are requests in flight. The in-flight counter needs to be atomic. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- v5: - Use atomic accesses for in_flight counter in vhost-user-server.c [Kevin] --- include/qemu/vhost-user-server.h | 4 +++- block/export/vhost-user-blk-server.c

[PATCH v6 02/20] hw/qdev: introduce qdev_is_realized() helper

2023-05-16 Thread Stefan Hajnoczi
-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- include/hw/qdev-core.h | 17 ++--- hw/scsi/scsi-bus.c | 3 +-- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h index 7623703943..f1070d6dc7 100644 --- a/include/hw

[PATCH v6 13/20] block/export: rewrite vduse-blk drain code

2023-05-16 Thread Stefan Hajnoczi
in .drained_end(). This eliminates the need for the aio_set_fd_handler(is_external=true) flag, which is being removed from QEMU. This is a long list but splitting it into individual commits would probably lead to git bisect failures - the changes are all related. Signed-off-by: Stefan Hajnoczi

[PATCH v6 05/20] util/vhost-user-server: rename refcount to in_flight counter

2023-05-16 Thread Stefan Hajnoczi
there are no more requests. Avoid confusing by renaming refcount and ref/unref to in_flight and inc/dec. Reviewed-by: Paolo Bonzini Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- include/qemu/vhost-user-server.h | 6 +++--- block/export/vhost-user

Re: [PATCH] hw/scsi/lsi53c895a: Fix reentrancy issues in the LSI controller (CVE-2023-0330)

2023-05-16 Thread Stefan Hajnoczi
On Tue, 16 May 2023 at 10:50, Thomas Huth wrote: > > On 16/05/2023 15.40, Stefan Hajnoczi wrote: > > On Tue, 16 May 2023 at 06:20, Thomas Huth wrote: > >> > >> We cannot use the generic reentrancy guard in the LSI code, so > >> we have to manually prevent e

Re: [PATCH] hw/scsi/lsi53c895a: Fix reentrancy issues in the LSI controller (CVE-2023-0330)

2023-05-16 Thread Stefan Hajnoczi
On Tue, 16 May 2023 at 06:20, Thomas Huth wrote: > > We cannot use the generic reentrancy guard in the LSI code, so > we have to manually prevent endless reentrancy here. The problematic > lsi_execute_script() function has already a way to detect whether > too many instructions have been executed

Re: Performance improvement with 58a2e3f5c37be02dac3086b81bdda9414b931edf

2023-05-16 Thread Stefan Hajnoczi
Thanks! Stefan

Re: Performance improvement with 58a2e3f5c37be02dac3086b81bdda9414b931edf

2023-05-16 Thread Stefan Hajnoczi
On Tue, 16 May 2023 at 08:56, Lukáš Doktor wrote: > > Hello Stefan, folks, > > the perf-ci detected and bisected the 58a2e3f5c37 - block: compile out > assert_bdrv_graph_readable() by default - as a performance improvement > especially with 4K reads with multiple jobs (but minor improvements

[PULL v2 06/16] iotests: test new zone operations

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li The new block layer APIs of zoned block devices can be tested by: $ tests/qemu-iotests/check zoned Run each zone operation on a newly created null_blk device and see whether it outputs the same zone information. Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi Acked-by: Kevin

[PULL v2 07/16] block: add some trace events for new block layer APIs

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi Reviewed-by: Dmitry Fomichev Acked-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-id: 20230508045533.175575-8-faithilike...@gmail.com Message-id: 20230324090605.28361-8-faithilike...@gmail.com Signed-off-by: Stefan

[PULL v2 11/16] qemu-iotests: test zone append operation

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li The patch tests zone append writes by reporting the zone wp after the completion of the call. "zap -p" option can print the sector offset value after completion, which should be the start sector where the append write begins. Signed-off-by: Sam Li Reviewed-by: Stefa

[PULL v2 09/16] file-posix: add tracking of the zone write pointers

2023-05-15 Thread Stefan Hajnoczi
- write to a zone - zone append Signed-off-by: Sam Li Message-id: 20230508051510.177850-2-faithilike...@gmail.com [Fix errno propagation from handle_aiocb_zone_mgmt() --Stefan] Signed-off-by: Stefan Hajnoczi --- include/block/block-common.h | 14 +++ include/block/block_int-common.h | 5

[PULL v2 08/16] docs/zoned-storage: add zoned device documentation

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li Add the documentation about the zoned device support to virtio-blk emulation. Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi Reviewed-by: Damien Le Moal Reviewed-by: Dmitry Fomichev Acked-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-id: 20230508045533.175575-9

[PULL v2 00/16] Block patches

2023-05-15 Thread Stefan Hajnoczi
The following changes since commit 8844bb8d896595ee1d25d21c770e6e6f29803097: Merge tag 'or1k-pull-request-20230513' of https://github.com/stffrdhrn/qemu into staging (2023-05-13 11:23:14 +0100) are available in the Git repository at: https://gitlab.com/stefanha/qemu.git

[PULL v2 04/16] block/raw-format: add zone operations to pass through requests

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li raw-format driver usually sits on top of file-posix driver. It needs to pass through requests of zone commands. Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi Reviewed-by: Damien Le Moal Reviewed-by: Hannes Reinecke Reviewed-by: Dmitry Fomichev Acked-by: Kevin Wolf Signed

[PULL v2 10/16] block: introduce zone append write for zoned devices

2023-05-15 Thread Stefan Hajnoczi
completion the device will respond with the position where the data has been written in the zone. Signed-off-by: Sam Li Reviewed-by: Dmitry Fomichev Reviewed-by: Stefan Hajnoczi Message-id: 20230508051510.177850-3-faithilike...@gmail.com Signed-off-by: Stefan Hajnoczi --- include/block/block

[PULL v2 15/16] virtio-blk: add some trace events for zoned emulation

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi Message-id: 20230508051916.178322-4-faithilike...@gmail.com Signed-off-by: Stefan Hajnoczi --- hw/block/virtio-blk.c | 12 hw/block/trace-events | 7 +++ 2 files changed, 19 insertions(+) diff --git a/hw/block

[PULL v2 14/16] block: add accounting for zone append operation

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li Taking account of the new zone append write operation for zoned devices, BLOCK_ACCT_ZONE_APPEND enum is introduced as other I/O request type (read, write, flush). Signed-off-by: Sam Li Message-id: 20230508051916.178322-3-faithilike...@gmail.com Signed-off-by: Stefan Hajnoczi

[PULL v2 16/16] docs/zoned-storage:add zoned emulation use case

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li Add the documentation about the example of using virtio-blk driver to pass the zoned block devices through to the guest. Signed-off-by: Sam Li Message-id: 20230508051916.178322-5-faithilike...@gmail.com [Fix pre-formatted code syntax --Stefan] Signed-off-by: Stefan Hajnoczi

[PULL v2 05/16] block: add zoned BlockDriver check to block layer

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li Putting zoned/non-zoned BlockDrivers on top of each other is not allowed. Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi Reviewed-by: Hannes Reinecke Reviewed-by: Dmitry Fomichev Acked-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-id: 20230508045533.175575-6

[PULL v2 13/16] virtio-blk: add zoned storage emulation for zoned devices

2023-05-15 Thread Stefan Hajnoczi
Message-id: 20230508051916.178322-2-faithilike...@gmail.com Signed-off-by: Stefan Hajnoczi --- hw/block/virtio-blk-common.c | 2 + hw/block/virtio-blk.c| 389 +++ hw/virtio/virtio-qmp.c | 2 + 3 files changed, 393 insertions(+) diff --git a/hw

[PULL v2 03/16] block/block-backend: add block layer APIs resembling Linux ZonedBlockDevice ioctls

2023-05-15 Thread Stefan Hajnoczi
=host_device, filename=/dev/nullb0 -c "zrp offset nr_zones" Signed-off-by: Sam Li Reviewed-by: Hannes Reinecke Reviewed-by: Stefan Hajnoczi Reviewed-by: Dmitry Fomichev Acked-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-id: 20230508045533.175575-4-faithilike...@gmail.com

[PULL v2 12/16] block: add some trace events for zone append

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li Signed-off-by: Sam Li Reviewed-by: Dmitry Fomichev Reviewed-by: Stefan Hajnoczi Message-id: 20230508051510.177850-5-faithilike...@gmail.com Signed-off-by: Stefan Hajnoczi --- block/file-posix.c | 3 +++ block/trace-events | 2 ++ 2 files changed, 5 insertions(+) diff --git

[PULL v2 02/16] block/file-posix: introduce helper functions for sysfs attributes

2023-05-15 Thread Stefan Hajnoczi
Reviewed-by: Stefan Hajnoczi Reviewed-by: Damien Le Moal Reviewed-by: Dmitry Fomichev Acked-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-id: 20230508045533.175575-3-faithilike...@gmail.com Message-id: 20230324090605.28361-3-faithilike...@gmail.com [Adjust commit message prefix as suggested

[PULL v2 01/16] block/block-common: add zoned device structs

2023-05-15 Thread Stefan Hajnoczi
From: Sam Li Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi Reviewed-by: Damien Le Moal Reviewed-by: Hannes Reinecke Reviewed-by: Dmitry Fomichev Acked-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-id: 20230508045533.175575-2-faithilike...@gmail.com Message-id

Re: [PATCH v4 20/20] aio: remove aio_disable_external() API

2023-05-11 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:34:17PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > All callers now pass is_external=false to aio_set_fd_handler() and > > aio_set_event_notifier(). The aio_disable_external() API that > > temporarily

Re: [PATCH v4 17/20] virtio-blk: implement BlockDevOps->drained_begin()

2023-05-11 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:13:42PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > Detach ioeventfds during drained sections to stop I/O submission from > > the guest. virtio-blk is no longer reliant on aio_disable_external() >

Re: [PATCH v4 17/20] virtio-blk: implement BlockDevOps->drained_begin()

2023-05-11 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:13:42PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > Detach ioeventfds during drained sections to stop I/O submission from > > the guest. virtio-blk is no longer reliant on aio_disable_external() >

Re: [PATCH v4 16/20] virtio: make it possible to detach host notifier from any thread

2023-05-11 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:00:35PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > virtio_queue_aio_detach_host_notifier() does two things: > > 1. It removes the fd handler from the event loop. > > 2. It processes the vi

Re: [PATCH v5 05/21] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-09 Thread Stefan Hajnoczi
On Tue, May 09, 2023 at 08:55:14PM +0200, Kevin Wolf wrote: > Am 04.05.2023 um 21:53 hat Stefan Hajnoczi geschrieben: > > This patch is part of an effort to remove the aio_disable_external() > > API because it does not fit in a multi-queue block layer world where > >

Re: [PATCH v5 00/21] block: remove aio_disable_external() API

2023-05-09 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:44:42PM +0200, Kevin Wolf wrote: > Am 04.05.2023 um 21:53 hat Stefan Hajnoczi geschrieben: > > v5: > > - Use atomic accesses for in_flight counter in vhost-user-server.c [Kevin] > > - Stash SCSIDevice id/lun values for VIRTIO_SCSI_T_TRANSPORT_RES

Re: [PATCH v2 00/20] Graph locking, part 3 (more block drivers)

2023-05-09 Thread Stefan Hajnoczi
don't want it. That's fine, we can leave them for now. Reviewed-by: Stefan Hajnoczi signature.asc Description: PGP signature

Re: [PATCH 2/4] vhost-user: Interface for migration state transfer

2023-05-09 Thread Stefan Hajnoczi
On Tue, 9 May 2023 at 11:35, Eugenio Perez Martin wrote: > > On Tue, May 9, 2023 at 5:09 PM Stefan Hajnoczi wrote: > > > > On Tue, May 09, 2023 at 08:45:33AM +0200, Eugenio Perez Martin wrote: > > > On Mon, May 8, 2023 at 10:10 PM Stefan Hajnoczi > > > w

Re: [PATCH 0/4] vhost-user-fs: Internal migration

2023-05-09 Thread Stefan Hajnoczi
On Fri, May 05, 2023 at 11:03:16AM +0200, Hanna Czenczek wrote: > On 04.05.23 23:14, Stefan Hajnoczi wrote: > > On Thu, 4 May 2023 at 13:39, Hanna Czenczek wrote: > > > On 11.04.23 17:05, Hanna Czenczek wrote: > > > > > > [...] > > > > > &

Re: [PATCH 0/4] vhost-user-fs: Internal migration

2023-05-09 Thread Stefan Hajnoczi
On Fri, May 05, 2023 at 04:26:08PM +0200, Eugenio Perez Martin wrote: > On Fri, May 5, 2023 at 11:51 AM Hanna Czenczek wrote: > > > > (By the way, thanks for the explanations :)) > > > > On 05.05.23 11:03, Hanna Czenczek wrote: > > > On 0

Re: [PATCH 2/4] vhost-user: Interface for migration state transfer

2023-05-09 Thread Stefan Hajnoczi
On Tue, May 09, 2023 at 08:45:33AM +0200, Eugenio Perez Martin wrote: > On Mon, May 8, 2023 at 10:10 PM Stefan Hajnoczi wrote: > > > > On Thu, Apr 20, 2023 at 03:29:44PM +0200, Eugenio Pérez wrote: > > > On Wed, 2023-04-19 at 07:21 -0400, Stefan Hajnoczi wrote: > >

Re: [PATCH 0/4] vhost-user-fs: Internal migration

2023-05-09 Thread Stefan Hajnoczi
On Tue, May 09, 2023 at 10:53:35AM +0200, Hanna Czenczek wrote: > On 08.05.23 23:10, Stefan Hajnoczi wrote: > > On Fri, May 05, 2023 at 02:51:55PM +0200, Hanna Czenczek wrote: > > > On 05.05.23 11:53, Eugenio Perez Martin wrote: > > > > On Fri, May 5, 2

Re: [PATCH v2 06/12] simpletrace: Simplify construction of tracing methods

2023-05-09 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 11:23:33AM +0200, Mads Ynddal wrote: > From: Mads Ynddal > > By moving the dynamic argument construction to keyword-arguments, > we can remove all of the specialized handling, and streamline it. > If a tracing method wants to access these, they can define the > kwargs, or

Re: [PATCH v2 05/12] simpletrace: Changed Analyzer class to become context-manager

2023-05-09 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 11:23:32AM +0200, Mads Ynddal wrote: > From: Mads Ynddal > > Instead of explicitly calling `begin` and `end`, we can change the class > to use the context-manager paradigm. This is mostly a styling choice, > used in modern Python code. But it also allows for more advanced

Re: [PATCH v2 04/12] simpletrace: update code for Python 3.11

2023-05-09 Thread Stefan Hajnoczi
ion required by QEMU. That makes it clear that its safe to merge this patch. Otherwise: Reviewed-by: Stefan Hajnoczi signature.asc Description: PGP signature

Re: [PATCH v2 03/12] simpletrace: changed naming of edict and idtoname to improve readability

2023-05-09 Thread Stefan Hajnoczi
; Signed-off-by: Mads Ynddal > --- > scripts/simpletrace.py | 34 +- > 1 file changed, 17 insertions(+), 17 deletions(-) Reviewed-by: Stefan Hajnoczi signature.asc Description: PGP signature

Re: [PATCH v2 02/12] simpletrace: Annotate magic constants from QEMU code

2023-05-09 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 11:23:29AM +0200, Mads Ynddal wrote: > From: Mads Ynddal > > It wasn't clear where the constants and structs came from, so I added > comments to help. > > Signed-off-by: Mads Ynddal > --- > scripts/simpletrace.py | 14 +++--- > 1 file changed, 7 insertions(+),

Re: [PATCH v2 00/12] simpletrace: refactor and general improvements

2023-05-09 Thread Stefan Hajnoczi
On Mon, May 08, 2023 at 06:50:58PM +0200, Mads Ynddal wrote: > > > > > I was curious how Mads is using simpletrace for an internal (to > > Samsung?) project. > > > > I was just tracing the NVMe emulation to get some metrics. The code is all > upstream or a part of this patchset. The rest is

Re: [PATCH] libvhost-user: Fix update of signalled_used

2023-05-09 Thread Stefan Hajnoczi
On Tue, 9 May 2023 at 06:28, Nir Soffer wrote: > > When we check if a driver needs a signal, we compare: > > - used_event: written by the driver each time it consumes an item In practice drivers tend to update it as you described, but devices cannot make that assumption. used_event is simply

Re: Your clang TSA patches

2023-05-09 Thread Stefan Hajnoczi
On Tue, 9 May 2023 at 03:01, Marc-André Lureau wrote: > > Hi Stefan > > On Mon, May 8, 2023 at 6:54 PM Stefan Hajnoczi wrote: >> >> Hi Marc-André, >> clang's Thread Safety Analysis is being used more in the QEMU block >> layer and a few limitations have bee

Re: [PATCH 0/4] vhost-user-fs: Internal migration

2023-05-08 Thread Stefan Hajnoczi
On Fri, May 05, 2023 at 02:51:55PM +0200, Hanna Czenczek wrote: > On 05.05.23 11:53, Eugenio Perez Martin wrote: > > On Fri, May 5, 2023 at 11:03 AM Hanna Czenczek wrote: > > > On 04.05.23 23:14, Stefan Hajnoczi wrote: > > > > On Thu, 4 May 2023 a

Re: [PATCH 2/4] vhost-user: Interface for migration state transfer

2023-05-08 Thread Stefan Hajnoczi
On Thu, Apr 20, 2023 at 03:29:44PM +0200, Eugenio Pérez wrote: > On Wed, 2023-04-19 at 07:21 -0400, Stefan Hajnoczi wrote: > > On Wed, 19 Apr 2023 at 07:10, Hanna Czenczek wrote: > > > On 18.04.23 09:54, Eugenio Perez Martin wrote: > > > > On Mon, Apr 17, 2

Re: [PATCH 2/4] vhost-user: Interface for migration state transfer

2023-05-08 Thread Stefan Hajnoczi
On Thu, Apr 20, 2023 at 03:27:51PM +0200, Eugenio Pérez wrote: > On Tue, 2023-04-18 at 16:40 -0400, Stefan Hajnoczi wrote: > > On Tue, 18 Apr 2023 at 14:31, Eugenio Perez Martin > > wrote: > > > On Tue, Apr 18, 2023 at 7:59 PM Stefan Hajnoczi > > > wrote: >

Re: [PATCH v2 00/12] simpletrace: refactor and general improvements

2023-05-08 Thread Stefan Hajnoczi
On Mon, May 08, 2023 at 03:28:04PM +0200, Mads Ynddal wrote: > > > A question for you: do you think it's possible to move simpletrace into > > qemu/python/utils? This requires cleaning up the code to some fairly > > pedantic standards, but helps protect it against rot as we change target > >

Re: [PATCH v2 01/12] simpletrace: Improve parsing of sys.argv; fix files never closed.

2023-05-08 Thread Stefan Hajnoczi
On Mon, May 08, 2023 at 01:18:40PM +, Mads Ynddal wrote: > > simpletrace.py is both a command-line tool and a Python module. The > > Python module has a public API that people's scripts may rely on. Let's > > avoid breaking API changes unless necessary so that existing scripts > > that import

Re: [PATCH v2 00/12] simpletrace: refactor and general improvements

2023-05-08 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 01:53:38PM -0400, John Snow wrote: > On Thu, May 4, 2023, 1:48 PM Stefan Hajnoczi wrote: > > > On Tue, May 02, 2023 at 11:23:27AM +0200, Mads Ynddal wrote: > > > From: Mads Ynddal > > > > > > I wanted to use simpletrace

Your clang TSA patches

2023-05-08 Thread Stefan Hajnoczi
Hi Marc-André, clang's Thread Safety Analysis is being used more in the QEMU block layer and a few limitations have been hit: 1. Mutexes that are struct fields are not supported (only global mutexes work). 2. Analysis does not extend across function pointers. See GRAPH_RDLOCK_PTR for a workaround

Re: [PATCH 0/4] vhost-user-fs: Internal migration

2023-05-04 Thread Stefan Hajnoczi
On Thu, 4 May 2023 at 13:39, Hanna Czenczek wrote: > > On 11.04.23 17:05, Hanna Czenczek wrote: > > [...] > > > Hanna Czenczek (4): > >vhost: Re-enable vrings after setting features > >vhost-user: Interface for migration state transfer > >vhost: Add high-level state save/load

[PATCH v5 06/21] util/vhost-user-server: rename refcount to in_flight counter

2023-05-04 Thread Stefan Hajnoczi
there are no more requests. Avoid confusing by renaming refcount and ref/unref to in_flight and inc/dec. Reviewed-by: Paolo Bonzini Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefan Hajnoczi --- include/qemu/vhost-user-server.h | 6 +++--- block/export/vhost-user-blk-server.c | 11

[PATCH v5 03/21] hw/qdev: introduce qdev_is_realized() helper

2023-05-04 Thread Stefan Hajnoczi
-off-by: Stefan Hajnoczi --- include/hw/qdev-core.h | 17 ++--- hw/scsi/scsi-bus.c | 3 +-- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h index 7623703943..f1070d6dc7 100644 --- a/include/hw/qdev-core.h +++ b/include

[PATCH v5 04/21] virtio-scsi: avoid race between unplug and transport event

2023-05-04 Thread Stefan Hajnoczi
. These changes ensure that the guest driver does not see the SCSIDevice that's being unplugged if it responds very quickly to the transport reset event. Reviewed-by: Paolo Bonzini Reviewed-by: Michael S. Tsirkin Reviewed-by: Daniil Tatianin Signed-off-by: Stefan Hajnoczi --- v5: - Stash

[PATCH v5 21/21] aio: remove aio_disable_external() API

2023-05-04 Thread Stefan Hajnoczi
, is_external, io_read, io_poll, io_poll_ready; @@ - aio_set_event_notifier(ctx, notifier, is_external, io_read, io_poll, io_poll_ready) + aio_set_event_notifier(ctx, notifier, io_read, io_poll, io_poll_ready) Reviewed-by: Juan Quintela Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefan

[PATCH v5 16/21] block/fuse: do not set is_external=true on FUSE fd

2023-05-04 Thread Stefan Hajnoczi
This is part of ongoing work to remove the aio_disable_external() API. Use BlockDevOps .drained_begin/end/poll() instead of aio_set_fd_handler(is_external=true). As a side-effect the FUSE export now follows AioContext changes like the other export types. Signed-off-by: Stefan Hajnoczi

[PATCH v5 20/21] virtio: do not set is_external=true on host notifiers

2023-05-04 Thread Stefan Hajnoczi
Host notifiers can now use is_external=false since virtio-blk and virtio-scsi no longer rely on is_external=true for drained sections. Signed-off-by: Stefan Hajnoczi --- hw/virtio/virtio.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio

[PATCH v5 08/21] block/export: stop using is_external in vhost-user-blk server

2023-05-04 Thread Stefan Hajnoczi
the aio_disable_external() API cannot be used since multiple AioContext may be processing I/O, not just one. Switch to BlockDevOps->drained_begin/end() callbacks. Signed-off-by: Stefan Hajnoczi --- block/export/vhost-user-blk-server.c | 28 ++-- util/vhost-user-server.c |

[PATCH v5 17/21] virtio: make it possible to detach host notifier from any thread

2023-05-04 Thread Stefan Hajnoczi
-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 2 ++ hw/scsi/virtio-scsi-dataplane.c | 9 + 2 files changed, 11 insertions(+) diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c index a6202997ee..27eafa6c92 100644 --- a/hw/block/dataplane/virtio-blk.c +++ b

[PATCH v5 15/21] block/export: don't require AioContext lock around blk_exp_ref/unref()

2023-05-04 Thread Stefan Hajnoczi
The FUSE export calls blk_exp_ref/unref() without the AioContext lock. Instead of fixing the FUSE export, adjust blk_exp_ref/unref() so they work without the AioContext lock. This way it's less error-prone. Suggested-by: Paolo Bonzini Signed-off-by: Stefan Hajnoczi --- include/block/export.h

[PATCH v5 07/21] block/export: wait for vhost-user-blk requests when draining

2023-05-04 Thread Stefan Hajnoczi
there are requests in flight. The in-flight counter needs to be atomic. Signed-off-by: Stefan Hajnoczi --- v5: - Use atomic accesses for in_flight counter in vhost-user-server.c [Kevin] --- include/qemu/vhost-user-server.h | 4 +++- block/export/vhost-user-blk-server.c | 13 + util

[PATCH v5 10/21] block: add blk_in_drain() API

2023-05-04 Thread Stefan Hajnoczi
The BlockBackend quiesce_counter is greater than zero during drained sections. Add an API to check whether the BlockBackend is in a drained section. The next patch will use this API. Signed-off-by: Stefan Hajnoczi --- include/sysemu/block-backend-global-state.h | 1 + block/block-backend.c

[PATCH v5 19/21] virtio-scsi: implement BlockDevOps->drained_begin()

2023-05-04 Thread Stefan Hajnoczi
sees one pair of .drained_begin/end() calls. After this commit, virtio-scsi no longer depends on hw/virtio's ioeventfd aio_set_event_notifier(is_external=true). This commit is a step towards removing the aio_disable_external() API. Signed-off-by: Stefan Hajnoczi --- include/hw/scsi/scsi.h

[PATCH v5 14/21] block/export: rewrite vduse-blk drain code

2023-05-04 Thread Stefan Hajnoczi
in .drained_end(). This eliminates the need for the aio_set_fd_handler(is_external=true) flag, which is being removed from QEMU. This is a long list but splitting it into individual commits would probably lead to git bisect failures - the changes are all related. Signed-off-by: Stefan Hajnoczi

[PATCH v5 11/21] block: drain from main loop thread in bdrv_co_yield_to_drain()

2023-05-04 Thread Stefan Hajnoczi
, and BlockDriver. Narrow IO_OR_GS_CODE() to GLOBAL_STATE_CODE() where appropriate. The test-bdrv-drain test case calls bdrv_drain() from an IOThread. This is now only allowed from coroutine context, so update the test case to run in a coroutine. Signed-off-by: Stefan Hajnoczi --- include/block

[PATCH v5 02/21] block-backend: split blk_do_set_aio_context()

2023-05-04 Thread Stefan Hajnoczi
o_ctx_commit() being invoked as part of the AioContext change propagation. This can be solved by temporarily setting blk->allow_aio_context_change to true. Future patches call blk_get_aio_context() from BlockDevOps->drained_end(), so this patch will become necessary. Signed-off-by: Stefan Hajnoczi

[PATCH v5 05/21] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-04 Thread Stefan Hajnoczi
equests remain once qdev_simple_device_unplug_cb() returns. Thanks to these two conditions we don't need aio_disable_external() anymore. Cc: Zhengui Li Reviewed-by: Paolo Bonzini Reviewed-by: Daniil Tatianin Signed-off-by: Stefan Hajnoczi --- hw/scsi/virtio-scsi.c | 3 --- 1 file changed, 3 deletions(-) d

[PATCH v5 13/21] hw/xen: do not set is_external=true on evtchn fds

2023-05-04 Thread Stefan Hajnoczi
BlockDevOps .drained_begin/end() callbacks. It no longer relies on is_external=true so it is safe to pass is_external=false. This is part of ongoing work to remove the aio_disable_external() API. Signed-off-by: Stefan Hajnoczi --- hw/xen/xen-bus.c | 8 1 file changed, 4 insertions(+), 4

[PATCH v5 12/21] xen-block: implement BlockDevOps->drained_begin()

2023-05-04 Thread Stefan Hajnoczi
on the dataplane instance, so move the blk_set_dev_ops() call after xen_block_dataplane_create(). Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/xen-block.h | 2 ++ hw/block/dataplane/xen-block.c | 42 +- hw/block/xen-block.c | 24

[PATCH v5 09/21] hw/xen: do not use aio_set_fd_handler(is_external=true) in xen_xenstore

2023-05-04 Thread Stefan Hajnoczi
-by: Stefan Hajnoczi --- hw/i386/kvm/xen_xenstore.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/i386/kvm/xen_xenstore.c b/hw/i386/kvm/xen_xenstore.c index 900679af8a..6e81bc8791 100644 --- a/hw/i386/kvm/xen_xenstore.c +++ b/hw/i386/kvm/xen_xenstore.c @@ -133,7 +133,7

[PATCH v5 18/21] virtio-blk: implement BlockDevOps->drained_begin()

2023-05-04 Thread Stefan Hajnoczi
attaching/detaching ioeventfds if the data plane is started/stopped during a drained section. This should be rare, but maybe the mirror block job can trigger it. Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 17 +-- hw/block/virtio-blk.c | 38

[PATCH v5 01/21] block: Fix use after free in blockdev_mark_auto_del()

2023-05-04 Thread Stefan Hajnoczi
. For both reasons, we can't just continue at block_job_next_locked(job). Instead, start at the head of the list again after job_cancel_locked() and skip those jobs that we already cancelled (or that are completing anyway). Signed-off-by: Kevin Wolf Reviewed-by: Stefan Hajnoczi Signed-off-by: Stefan

[PATCH v5 00/21] block: remove aio_disable_external() API

2023-05-04 Thread Stefan Hajnoczi
I/O requests are not submitted in drained sections. Kevin Wolf (1): block: Fix use after free in blockdev_mark_auto_del() Stefan Hajnoczi (20): block-backend: split blk_do_set_aio_context() hw/qdev: introduce qdev_is_realized() helper virtio-scsi: avoid race between unplug and transport event

Re: [PATCH v2 01/12] simpletrace: Improve parsing of sys.argv; fix files never closed.

2023-05-04 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 11:23:28AM +0200, Mads Ynddal wrote: > From: Mads Ynddal > > The arguments extracted from `sys.argv` named and unpacked to make it > clear what the arguments are and what they're used for. > > The two input files were opened, but never explicitly closed. File usage >

Re: [PATCH v2 00/12] simpletrace: refactor and general improvements

2023-05-04 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 11:23:27AM +0200, Mads Ynddal wrote: > From: Mads Ynddal > > I wanted to use simpletrace.py for an internal project, so I tried to update > and polish the code. Some of the commits resolve specific issues, while some > are more subjective. An internal project based on

Re: [PATCH] block: Fix use after free in blockdev_mark_auto_del()

2023-05-03 Thread Stefan Hajnoczi
t; anyway). > > Signed-off-by: Kevin Wolf > --- > blockdev.c | 18 ++ > 1 file changed, 14 insertions(+), 4 deletions(-) Reviewed-by: Stefan Hajnoczi signature.asc Description: PGP signature

Re: [PATCH] block/export: call blk_set_dev_ops(blk, NULL, NULL)

2023-05-03 Thread Stefan Hajnoczi
On Wed, May 03, 2023 at 10:43:16AM -0500, Eric Blake wrote: > On Tue, May 02, 2023 at 05:11:19PM -0400, Stefan Hajnoczi wrote: > > Most export types install BlockDeviceOps pointers. It is easy to forget > > to remove them because that happens automatically via the "drive"

Re: [PATCH] block/blkio: add 'fd' option to virtio-blk-vhost-vdpa driver

2023-05-03 Thread Stefan Hajnoczi
On Wed, May 03, 2023 at 11:15:56AM +0200, Stefano Garzarella wrote: > On Tue, May 02, 2023 at 03:02:32PM -0400, Stefan Hajnoczi wrote: > > On Tue, May 02, 2023 at 04:50:50PM +0200, Stefano Garzarella wrote: > > > The virtio-blk-vhost-vdpa driver in libblkio 1.3.0 supports

Re: [PATCH v4 03/20] virtio-scsi: avoid race between unplug and transport event

2023-05-03 Thread Stefan Hajnoczi
On Wed, May 03, 2023 at 10:00:57AM +0200, Kevin Wolf wrote: > Am 02.05.2023 um 20:56 hat Stefan Hajnoczi geschrieben: > > On Tue, May 02, 2023 at 05:19:46PM +0200, Kevin Wolf wrote: > > > Am 25.04.2023 um 19:26 hat Stefan Hajnoczi geschrieben: > > > > Only

Re: [PATCH v4 07/20] block/export: stop using is_external in vhost-user-blk server

2023-05-03 Thread Stefan Hajnoczi
On Wed, May 03, 2023 at 10:08:46AM +0200, Kevin Wolf wrote: > Am 02.05.2023 um 22:06 hat Stefan Hajnoczi geschrieben: > > On Tue, May 02, 2023 at 06:04:24PM +0200, Kevin Wolf wrote: > > > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > > > vhost-user act

Re: [PATCH v4 04/20] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-03 Thread Stefan Hajnoczi
On Wed, May 03, 2023 at 01:40:49PM +0200, Kevin Wolf wrote: > Am 02.05.2023 um 22:02 hat Stefan Hajnoczi geschrieben: > > On Tue, May 02, 2023 at 03:19:52PM +0200, Kevin Wolf wrote: > > > Am 01.05.2023 um 17:09 hat Stefan Hajnoczi geschrieben: > > > > On Fri, Ap

[PATCH] block/export: call blk_set_dev_ops(blk, NULL, NULL)

2023-05-02 Thread Stefan Hajnoczi
o remember. This fixes the nbd and vhost-user-blk export types. Fixes: fd6afc501a01 ("nbd/server: Use drained block ops to quiesce the server") Fixes: ca858a5fe94c ("vhost-user-blk-server: notify client about disk resize") Signed-off-by: Stefan Hajnoczi --- block/export/export.c

Re: [PATCH v4 10/20] block: drain from main loop thread in bdrv_co_yield_to_drain()

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 06:21:20PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > For simplicity, always run BlockDevOps .drained_begin/end/poll() > > callbacks in the main loop thread. This makes it easier to implement the > > callb

Re: [PATCH v4 07/20] block/export: stop using is_external in vhost-user-blk server

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 06:04:24PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > vhost-user activity must be suspended during bdrv_drained_begin/end(). > > This prevents new requests from interfering with whatever is happening > > i

Re: [PATCH v4 04/20] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 03:19:52PM +0200, Kevin Wolf wrote: > Am 01.05.2023 um 17:09 hat Stefan Hajnoczi geschrieben: > > On Fri, Apr 28, 2023 at 04:22:55PM +0200, Kevin Wolf wrote: > > > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > > > This patch

Re: [PATCH v4 06/20] block/export: wait for vhost-user-blk requests when draining

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 05:42:51PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > Each vhost-user-blk request runs in a coroutine. When the BlockBackend > > enters a drained section we need to enter a quiescent state. Currently > > a

Re: [PATCH] block/blkio: add 'fd' option to virtio-blk-vhost-vdpa driver

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 04:50:50PM +0200, Stefano Garzarella wrote: > The virtio-blk-vhost-vdpa driver in libblkio 1.3.0 supports the new > 'fd' property. Let's expose this to the user, so the management layer > can pass the file descriptor of an already opened vhost-vdpa character > device. This

Re: [PATCH v4 03/20] virtio-scsi: avoid race between unplug and transport event

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 05:19:46PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:26 hat Stefan Hajnoczi geschrieben: > > Only report a transport reset event to the guest after the SCSIDevice > > has been unrealized by qdev_simple_device_unplug_cb(). > > > > qdev_sim

[PATCH 0/2] aio-posix: do not nest poll handlers

2023-05-02 Thread Stefan Hajnoczi
, but I have included a test case that demonstrates the problem. Stefan Hajnoczi (2): aio-posix: do not nest poll handlers tested: add test for nested aio_poll() in poll handlers tests/unit/test-nested-aio-poll.c | 130 ++ util/aio-posix.c | 11

[PATCH 2/2] tested: add test for nested aio_poll() in poll handlers

2023-05-02 Thread Stefan Hajnoczi
Signed-off-by: Stefan Hajnoczi --- tests/unit/test-nested-aio-poll.c | 130 ++ tests/unit/meson.build| 1 + 2 files changed, 131 insertions(+) create mode 100644 tests/unit/test-nested-aio-poll.c diff --git a/tests/unit/test-nested-aio-poll.c b/tests

[PATCH 1/2] aio-posix: do not nest poll handlers

2023-05-02 Thread Stefan Hajnoczi
c: Paolo Bonzini Signed-off-by: Stefan Hajnoczi --- util/aio-posix.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/util/aio-posix.c b/util/aio-posix.c index a8be940f76..34bc2a64d8 100644 --- a/util/aio-posix.c +++ b/util/aio-posix.c @@ -353,8 +353,19 @@ static bool aio_dispatch_ha

Re: [PATCH] async: avoid use-after-free on re-entrancy guard

2023-05-02 Thread Stefan Hajnoczi
8513 > Fixes: 9c86c97f12 ("async: Add an optional reentrancy guard to the BH API") > Signed-off-by: Alexander Bulekov > --- > util/async.c | 14 -- > 1 file changed, 8 insertions(+), 6 deletions(-) Reviewed-by: Stefan Hajnoczi signature.asc Description: PGP signature

[PATCH v2] vhost-user: send SET_STATUS 0 after GET_VRING_BASE

2023-05-02 Thread Stefan Hajnoczi
in Cc: Cindy Lu Cc: Yajun Wu Signed-off-by: Stefan Hajnoczi --- v2: - Added VHOST_USER_PROTOCOL_F_STATUS check [Yajun Wu] - Added "Fixes:" tag [Michael] --- hw/virtio/vhost-user.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-user.c

Re: [PATCH] vhost-user: send SET_STATUS 0 after GET_VRING_BASE

2023-05-01 Thread Stefan Hajnoczi
On Fri, Apr 21, 2023 at 01:30:48PM +0800, Yajun Wu wrote: > > On 4/20/2023 9:07 PM, Stefan Hajnoczi wrote: > > > > Setting the VIRTIO Device Status Field to 0 resets the device. The > > device's state is lost, including the vring configuration. > > > > vhost

<    9   10   11   12   13   14   15   16   17   18   >