[PATCH v2 1/3] qapi: rename BlockDirtyBitmapMergeSource to BlockDirtyBitmapOrStr

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
From: Vladimir Sementsov-Ogievskiy Rename the type to be reused. Old name is "what is it for". To be natively reused for other needs, let's name it exactly "what is it". Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/monitor/bitmap-qmp-cmds.c| 6 +++--- include/block/block_int-g

[PATCH v2 for-7.1 0/3] qapi: nbd-export: select bitmap by node/name pair

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
From: Vladimir Sementsov-Ogievskiy Hi all! Here is small improvement for bitmap exporting interface. v2: Sorry for the noise, me trying to find a email service, that don't consider sending patch series by git-send-email as as spam :/ Vladimir Sementsov-Ogievskiy (3): qapi: rename BlockDirtyB

[PATCH v2 2/3] qapi: nbd-export: allow select bitmaps by node/name pair

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
From: Vladimir Sementsov-Ogievskiy Hi all! Current logic of relying on search through backing chain is not safe neither convenient. Sometimes it leads to necessity of extra bitmap copying. Also, we are going to add "snapshot-access" driver, to access some snapshot state through NBD. And this dri

[PATCH v2 3/3] iotests/223: check new possibility of exporting bitmaps by node/name

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
From: Vladimir Sementsov-Ogievskiy Add simple test that new interface introduced in previous commit works. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/223 | 16 + tests/qemu-iotests/223.out | 47 -- 2 files changed, 61

Re: [PATCH 3/3] Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Alex Bennée
Markus Armbruster writes: > g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, > for two reasons. One, it catches multiplication overflowing size_t. > Two, it returns T * rather than void *, which lets the compiler catch > more type errors. > > diff --git a/semihosting/conf

Re: [PATCH v2 for-7.1 0/3] qapi: nbd-export: select bitmap by node/name pair

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
15.03.2022 00:32, Vladimir Sementsov-Ogievskiy wrote: From: Vladimir Sementsov-Ogievskiy Hi all! Here is small improvement for bitmap exporting interface. v2: Sorry for the noise, me trying to find a email service, that don't consider sending patch series by git-send-email as as spam :/ Aha

[PATCH v3 3/3] nbd/server: Allow MULTI_CONN for shared writable exports

2022-03-14 Thread Eric Blake
According to the NBD spec, a server that advertises NBD_FLAG_CAN_MULTI_CONN promises that multiple client connections will not see any cache inconsistencies: when properly separated by a single flush, actions performed by one client will be visible to another client, regardless of which client did

[PATCH v3 2/3] qemu-nbd: Pass max connections to blockdev layer

2022-03-14 Thread Eric Blake
The next patch wants to adjust whether the NBD server code advertises MULTI_CONN based on whether it is known if the server limits to exactly one client. For a server started by QMP, this information is obtained through nbd_server_start (which can support more than one export); but for qemu-nbd (w

[PATCH v3 1/3] docs: Consistent typography for options of qemu-nbd

2022-03-14 Thread Eric Blake
Prefer the :option:`--name` form when cross-referencing other options from the qemu-nbd documentation. Signed-off-by: Eric Blake --- docs/tools/qemu-nbd.rst | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/tools/qemu-nbd.rst b/docs/tools/qemu-nbd.rst index 2b

[PATCH v3 0/3] nbd: MULTI_CONN for shared writable exports

2022-03-14 Thread Eric Blake
v2 was here: https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg03314.html Since then: - expose the knob through 'qemu-nbd -m on|off|auto' - reuse QAPI OnOffAuto type instead of rolling my own - rewrite the iotest from bash into python (thanks: Nir, Vladimir) - split out easy preliminary fix

Re: [PATCH 3/3] Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Christian Schoenebeck
On Montag, 14. März 2022 20:48:47 CET Alex Bennée wrote: > Markus Armbruster writes: > > g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, > > for two reasons. One, it catches multiplication overflowing size_t. > > Two, it returns T * rather than void *, which lets the compile

Re: [PATCH 3/3] Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Alex Bennée
Markus Armbruster writes: > g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, > for two reasons. One, it catches multiplication overflowing size_t. > Two, it returns T * rather than void *, which lets the compiler catch > more type errors. > > This commit only touches alloc

[PATCH for-7.1 0/3] qapi: nbd-export: select bitmap by node/name pair

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
Hi all! Here is small improvement for bitmap exporting interface. Vladimir Sementsov-Ogievskiy (3): qapi: rename BlockDirtyBitmapMergeSource to BlockDirtyBitmapOrStr qapi: nbd-export: allow select bitmaps by node/name pair iotests/223: check new possibility of exporting bitmaps by node/name

[PATCH 1/3] qapi: rename BlockDirtyBitmapMergeSource to BlockDirtyBitmapOrStr

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
From: Vladimir Sementsov-Ogievskiy Rename the type to be reused. Old name is "what is it for". To be natively reused for other needs, let's name it exactly "what is it". Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/monitor/bitmap-qmp-cmds.c| 6 +++--- include/block/block_int-g

Re: [PATCH experiment 00/16] C++20 coroutine backend

2022-03-14 Thread Richard Henderson
On 3/14/22 09:21, Paolo Bonzini wrote: But perhaps someone will try to use templates to replace repeated inclusion (which is common in hw/display) and others will follow suit. The code in fpu/ desperately calls out for overloading and templates. At present it is a tangle of _Generic and multi

Re: [PATCH experiment 14/16] util: introduce C++ stackless coroutine backend

2022-03-14 Thread Paolo Bonzini
Il lun 14 mar 2022, 15:37 Stefan Hajnoczi ha scritto: > On Mon, Mar 14, 2022 at 10:32:01AM +0100, Paolo Bonzini wrote: > > +// > > + > > +// CoroutineFn does not even need anything more than what > > +// BaseCoroutine provides, so it's just a type alias. The magic > > +/

Re: [PATCH 2/3] 9pfs: Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Alex Bennée
Markus Armbruster writes: > g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, > for two reasons. One, it catches multiplication overflowing size_t. > Two, it returns T * rather than void *, which lets the compiler catch > more type errors. > > This commit only touches alloc

Re: [PATCH 1/3] scripts/coccinelle: New use-g_new-etc.cocci

2022-03-14 Thread Alex Bennée
Markus Armbruster writes: > This is the semantic patch from commit b45c03f585 "arm: Use g_new() & > friends where that makes obvious sense". > > Signed-off-by: Markus Armbruster Reviewed-by: Alex Bennée -- Alex Bennée

Re: [PATCH v2 1/6] block: Support passing NULL ops to blk_set_dev_ops()

2022-03-14 Thread John Snow
On Mon, Mar 14, 2022 at 1:23 PM Stefan Hajnoczi wrote: > > On Tue, Feb 15, 2022 at 06:59:38PM +0800, Xie Yongji wrote: > > This supports passing NULL ops to blk_set_dev_ops() > > so that we can remove stale ops in some cases. > > > > Signed-off-by: Xie Yongji > > --- > > block/block-backend.c |

Re: [PATCH 2/3] 9pfs: Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Philippe Mathieu-Daudé
Hi Christian, On 14/3/22 17:42, Christian Schoenebeck wrote: On Montag, 14. März 2022 17:01:07 CET Markus Armbruster wrote: g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, for two reasons. One, it catches multiplication overflowing size_t. Two, it returns T * rather than

Re: [PATCH] MAINTAINERS: change Vladimir's email address

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
14.03.2022 19:40, Vladimir Sementsov-Ogievskiy wrote: 14.03.2022 18:16, Vladimir Sementsov-Ogievskiy wrote: 14.03.2022 17:47, Eric Blake wrote: On Mon, Mar 14, 2022 at 02:04:15PM +0300, Vladimir Sementsov-Ogievskiy wrote: Old vsement...@virtuozzo.com is not accessible anymore. Signed-off-by:

Re: [PATCH 3/3] Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Daniel P . Berrangé
On Mon, Mar 14, 2022 at 05:52:32PM +0100, Markus Armbruster wrote: > Peter Maydell writes: > > > On Mon, 14 Mar 2022 at 16:01, Markus Armbruster wrote: > >> > >> g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, > >> for two reasons. One, it catches multiplication overflowin

Re: [PATCH v2] nbd/server: Allow MULTI_CONN for shared writable exports

2022-03-14 Thread Eric Blake
On Wed, Feb 16, 2022 at 07:14:58PM +0200, Nir Soffer wrote: > > > I'm not the best at writing python iotests; I welcome a language > > > translation of this aspect. > > > > > > > > Let me try:) > > Thanks! This is much nicer and will be easier to maintain. > > > > > > > #!/usr/bin/env python3 > >

Re: [PATCH v2 1/6] block: Support passing NULL ops to blk_set_dev_ops()

2022-03-14 Thread Stefan Hajnoczi
On Tue, Feb 15, 2022 at 06:59:38PM +0800, Xie Yongji wrote: > This supports passing NULL ops to blk_set_dev_ops() > so that we can remove stale ops in some cases. > > Signed-off-by: Xie Yongji > --- > block/block-backend.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a

Re: [PATCH v2 4/4] util/event-loop-base: Introduce options to set the thread pool size

2022-03-14 Thread Nicolas Saenz Julienne
On Mon, 2022-03-14 at 13:35 +, Stefan Hajnoczi wrote: > On Fri, Mar 11, 2022 at 11:40:30AM +0100, Nicolas Saenz Julienne wrote: > > On Thu, 2022-03-10 at 10:45 +, Stefan Hajnoczi wrote: > > > On Thu, Mar 03, 2022 at 04:13:07PM +0100, Nicolas Saenz Julienne wrote: > > > > @@ -537,10 +546,19

Re: [PATCH 3/3] Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Cédric Le Goater
On 3/14/22 17:01, Markus Armbruster wrote: g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, for two reasons. One, it catches multiplication overflowing size_t. Two, it returns T * rather than void *, which lets the compiler catch more type errors. This commit only touches a

Re: [PATCH experiment 00/16] C++20 coroutine backend

2022-03-14 Thread Daniel P . Berrangé
On Mon, Mar 14, 2022 at 10:31:47AM +0100, Paolo Bonzini wrote: > This was compiled with GCC 11 only. Coroutine support was added in > GCC 10, released in 2020, which IIRC is much newer than the most recent > release we support. Currrently we target 7.4: commit 2a85a08c998e418a46a308095893f2236

Re: [PATCH 3/3] Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Markus Armbruster
Peter Maydell writes: > On Mon, 14 Mar 2022 at 16:01, Markus Armbruster wrote: >> >> g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, >> for two reasons. One, it catches multiplication overflowing size_t. >> Two, it returns T * rather than void *, which lets the compiler ca

Re: [PATCH 2/3] 9pfs: Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Christian Schoenebeck
On Montag, 14. März 2022 17:01:07 CET Markus Armbruster wrote: > g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, > for two reasons. One, it catches multiplication overflowing size_t. > Two, it returns T * rather than void *, which lets the compiler catch > more type errors. >

Re: [PATCH 0/3] Use g_new() & friends where that makes obvious

2022-03-14 Thread Philippe Mathieu-Daudé
On 14/3/22 17:01, Markus Armbruster wrote: g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, for two reasons. One, it catches multiplication overflowing size_t. Two, it returns T * rather than void *, which lets the compiler catch more type errors. This series only touches a

Re: [PATCH] MAINTAINERS: change Vladimir's email address

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
14.03.2022 18:16, Vladimir Sementsov-Ogievskiy wrote: 14.03.2022 17:47, Eric Blake wrote: On Mon, Mar 14, 2022 at 02:04:15PM +0300, Vladimir Sementsov-Ogievskiy wrote: Old vsement...@virtuozzo.com is not accessible anymore. Signed-off-by: Vladimir Sementsov-Ogievskiy --- Hi all! That's my n

[PATCH for-7.0 1/2] block/vmdk: Fix reopening bs->file

2022-03-14 Thread Hanna Reitz
VMDK disk data is stored in extents, which may or may not be separate from bs->file. VmdkExtent.file points to where they are stored. Each that is stored in bs->file will simply reuse the exact pointer value of bs->file. (That is why vmdk_free_extents() will unref VmdkExtent.file (e->file) only

[PATCH 1/3] scripts/coccinelle: New use-g_new-etc.cocci

2022-03-14 Thread Markus Armbruster
This is the semantic patch from commit b45c03f585 "arm: Use g_new() & friends where that makes obvious sense". Signed-off-by: Markus Armbruster --- scripts/coccinelle/use-g_new-etc.cocci | 75 ++ 1 file changed, 75 insertions(+) create mode 100644 scripts/coccinelle/use-

Re: [PATCH 3/3] Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Peter Maydell
On Mon, 14 Mar 2022 at 16:01, Markus Armbruster wrote: > > g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, > for two reasons. One, it catches multiplication overflowing size_t. > Two, it returns T * rather than void *, which lets the compiler catch > more type errors. > > Th

[PATCH for-7.0 2/2] iotests/reopen-file: Test reopening file child

2022-03-14 Thread Hanna Reitz
This should work for all format drivers that support reopening, so test it. (This serves as a regression test for HEAD^: This test used to fail for VMDK before HEAD^.) Signed-off-by: Hanna Reitz --- tests/qemu-iotests/tests/reopen-file | 88 tests/qemu-iotests/tests

[PATCH for-7.0 0/2] block/vmdk: Fix reopening bs->file

2022-03-14 Thread Hanna Reitz
Hi, A couple of months ago I noticed that changing a vmdk node’s file child through blockdev-reopen would crash qemu. I started writing a fix at some point, got distracted, but now here it is. Hanna Reitz (2): block/vmdk: Fix reopening bs->file iotests/reopen-file: Test reopening file child

Re: [PATCH v2] block/nbd.c: Fixed IO request coroutine not being wakeup when kill NBD server

2022-03-14 Thread Eric Blake
On Wed, Mar 09, 2022 at 04:21:01PM +0300, Vladimir Sementsov-Ogievskiy wrote: > 09.03.2022 10:48, Rao Lei wrote: > > During the IO stress test, the IO request coroutine has a probability that > > is > > can't be awakened when the NBD server is killed. > > > > The GDB stack is as follows: > > Whe

Re: [PATCH experiment 00/16] C++20 coroutine backend

2022-03-14 Thread Paolo Bonzini
On 3/14/22 15:07, Stefan Hajnoczi wrote: If we can reach a consensus about C++ language usage in QEMU then I'm in favor of using C++ coroutines. It's probably not realistic to think we can limit C++ language usage to just coroutines forever. Someone finds another C++ feature they absolutely need

[PATCH 2/3] 9pfs: Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Markus Armbruster
g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, for two reasons. One, it catches multiplication overflowing size_t. Two, it returns T * rather than void *, which lets the compiler catch more type errors. This commit only touches allocations with size arguments of the form si

[PATCH 3/3] Use g_new() & friends where that makes obvious sense

2022-03-14 Thread Markus Armbruster
g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, for two reasons. One, it catches multiplication overflowing size_t. Two, it returns T * rather than void *, which lets the compiler catch more type errors. This commit only touches allocations with size arguments of the form si

[PATCH 0/3] Use g_new() & friends where that makes obvious

2022-03-14 Thread Markus Armbruster
g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, for two reasons. One, it catches multiplication overflowing size_t. Two, it returns T * rather than void *, which lets the compiler catch more type errors. This series only touches allocations with size arguments of the form si

Re: [PATCH v2] nbd/server: Allow MULTI_CONN for shared writable exports

2022-03-14 Thread Eric Blake
On Wed, Feb 16, 2022 at 11:08:06AM +0300, Vladimir Sementsov-Ogievskiy wrote: > 16.02.2022 02:24, Eric Blake wrote: > > > > +++ b/tests/qemu-iotests/tests/nbd-multiconn > > > > @@ -0,0 +1,188 @@ > > > > +#!/usr/bin/env bash > > > > +# group: rw auto quick > > > > +# > > > > +# Test that qemu-nbd M

Re: [PATCH] MAINTAINERS: change Vladimir's email address

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
14.03.2022 17:47, Eric Blake wrote: On Mon, Mar 14, 2022 at 02:04:15PM +0300, Vladimir Sementsov-Ogievskiy wrote: Old vsement...@virtuozzo.com is not accessible anymore. Signed-off-by: Vladimir Sementsov-Ogievskiy --- Hi all! That's my new address: v.sementsov...@ya.ru , the old one is not a

Re: [PATCH v13 0/8] hmp,qmp: Add commands to introspect virtio devices

2022-03-14 Thread Jonah Palmer
My apologies for not notifying earlier but I have been looking into these errors. Hopefully should be able to get a fix sent out within the week. Thanks Jonah On 3/7/22 17:46, Michael S. Tsirkin wrote: On Mon, Mar 07, 2022 at 08:08:33AM -0500, Jonah Palmer wrote: This series introduces new Q

Re: [PATCH] MAINTAINERS: change Vladimir's email address

2022-03-14 Thread Eric Blake
On Mon, Mar 14, 2022 at 02:04:15PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Old vsement...@virtuozzo.com is not accessible anymore. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > > Hi all! > > That's my new address: v.sementsov...@ya.ru , the old one is not > available anymore. I

Re: [PATCH experiment 14/16] util: introduce C++ stackless coroutine backend

2022-03-14 Thread Stefan Hajnoczi
On Mon, Mar 14, 2022 at 10:32:01AM +0100, Paolo Bonzini wrote: > +// > + > +// CoroutineFn does not even need anything more than what > +// BaseCoroutine provides, so it's just a type alias. The magic > +// is all in ValuePromise. > +// > +// Suspended CoroutineFns are cha

Re: [PATCH experiment 00/35] stackless coroutine backend

2022-03-14 Thread Stefan Hajnoczi
On Sun, Mar 13, 2022 at 04:18:40PM +0100, Paolo Bonzini wrote: > On 3/11/22 13:17, Daniel P. Berrangé wrote: > The question is what is easier to maintain, stack switching code that is > becoming less and less portable (status quo with SafeStack, CET and the TLS > issues that Stefan has worked on),

[PATCH v6 11/18] job.h: rename job API functions called with job_mutex held

2022-03-14 Thread Emanuele Giuseppe Esposito
With the *nop* job_lock/unlock placed, rename the job functions of the job API that are always under job_mutex, adding "_locked" suffix. List of functions that get this suffix: job_txn_unref job_txn_add_job job_ref job_unref job_enter_cond job_finish_sync job_is_

[PATCH v6 09/18] block/mirror.c: use of job helpers in drivers to avoid TOC/TOU

2022-03-14 Thread Emanuele Giuseppe Esposito
Once job lock is used and aiocontext is removed, mirror has to perform job operations under the same critical section, using the helpers prepared in previous commit. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by:

[PATCH v6 05/18] job.h: add _locked duplicates for job API functions called with and without job_mutex

2022-03-14 Thread Emanuele Giuseppe Esposito
In preparation to the job_lock/unlock usage, create _locked duplicates of some functions, since they will be sometimes called with job_mutex held (mostly within job.c), and sometimes without (mostly from JobDrivers using the job API). Therefore create a _locked version of such function, so that it

Re: [PATCH experiment 00/16] C++20 coroutine backend

2022-03-14 Thread Stefan Hajnoczi
On Mon, Mar 14, 2022 at 10:31:47AM +0100, Paolo Bonzini wrote: > However, there are no ramifications to actual coroutine code, except > for the template syntax "CoroutineFn" for the function and > the mandatory co_await/co_return keywords... both of which are an > improvement, really: the fact tha

[PATCH] MAINTAINERS: change Vladimir's email address

2022-03-14 Thread Vladimir Sementsov-Ogievskiy
Old vsement...@virtuozzo.com is not accessible anymore. Signed-off-by: Vladimir Sementsov-Ogievskiy --- Hi all! That's my new address: v.sementsov...@ya.ru , the old one is not available anymore. I've also subscribed this new address for qemu-devel mailing list, but not yet get any message fro

Re: [PATCH experiment 12/16] remove "new" keyword from trace-events

2022-03-14 Thread Philippe Mathieu-Daudé
On 14/3/22 10:31, Paolo Bonzini wrote: This is invalid in C++. Signed-off-by: Paolo Bonzini --- util/trace-events | 16 1 file changed, 8 insertions(+), 8 deletions(-) Reviewed-by: Philippe Mathieu-Daudé

[PATCH v6 16/18] jobs: protect job.aio_context with BQL and job_mutex

2022-03-14 Thread Emanuele Giuseppe Esposito
In order to make it thread safe, implement a "fake rwlock", where we allow reads under BQL *or* job_mutex held, but writes only under BQL *and* job_mutex. The only write we have is in child_job_set_aio_ctx, which always happens under drain (so the job is paused). For this reason, introduce job_set

[PATCH v6 13/18] job.h: define unlocked functions

2022-03-14 Thread Emanuele Giuseppe Esposito
All these functions assume that the lock is not held, and acquire it internally. These functions will be useful when job_lock is globally applied, as they will allow callers to access the job struct fields without worrying about the job lock. Update also the comments in blockjob.c (and move them

Re: [PATCH v2] block-qdict: Fix -Werror=maybe-uninitialized build failure

2022-03-14 Thread Philippe Mathieu-Daudé
On 11/3/22 23:16, Murilo Opsfelder Araujo wrote: Building QEMU on Fedora 37 (Rawhide Prerelease) ppc64le failed with the following error: $ ../configure --prefix=/usr/local/qemu-disabletcg --target-list=ppc-softmmu,ppc64-softmmu --disable-tcg --disable-linux-user ... $ make -j$(n

Re: [PATCH experiment 08/16] tracetool: add extern "C" around generated headers

2022-03-14 Thread Paolo Bonzini
On 3/14/22 14:33, Philippe Mathieu-Daudé wrote: On 14/3/22 10:31, Paolo Bonzini wrote: Signed-off-by: Paolo Bonzini ---   scripts/tracetool/format/h.py | 8 +++-   1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/scripts/tracetool/format/h.py b/scripts/tracetool/format/h.py ind

[PATCH v2 09/10] child_job_drained_poll: override polling condition only when in home thread

2022-03-14 Thread Emanuele Giuseppe Esposito
drv->drained_poll() is only implemented in mirror, and allows it to drain from within the coroutine. The mirror implementation uses in_drain flag to recognize when it is draining from coroutine, and consequently avoid deadlocking (wait the poll condition in child_job_drained_poll to wait for itself

[PATCH v6 10/18] jobs: rename static functions called with job_mutex held

2022-03-14 Thread Emanuele Giuseppe Esposito
With the *nop* job_lock/unlock placed, rename the static functions that are always under job_mutex, adding "_locked" suffix. List of functions that get this suffix: job_txn_refjob_txn_del_job job_txn_apply job_state_transition job_should_pause job_event_cance

[PATCH v6 04/18] aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED

2022-03-14 Thread Emanuele Giuseppe Esposito
Same as AIO_WAIT_WHILE macro, but if we are in the Main loop do not release and then acquire ctx_ 's aiocontext. Once all Aiocontext locks go away, this macro will replace AIO_WAIT_WHILE. Reviewed-by: Stefan Hajnoczi Signed-off-by: Emanuele Giuseppe Esposito --- include/block/aio-wait.h | 17 +

[PATCH v2 08/10] tests/unit/test-bdrv-drain.c: graph setup functions can't run in coroutines

2022-03-14 Thread Emanuele Giuseppe Esposito
Graph initialization functions like blk_new(), bdrv_new() and so on should not run in a coroutine. In fact, they might invoke a drain (for example blk_insert_bs eventually calls bdrv_replace_child_noperm) that in turn can invoke callbacks like bdrv_do_drained_begin_quiesce(), that asserts exactly t

[PATCH v6 18/18] block_job_query: remove atomic read

2022-03-14 Thread Emanuele Giuseppe Esposito
Not sure what the atomic here was supposed to do, since job.busy is protected by the job lock. Since the whole function is called under job_mutex, just remove the atomic. Reviewed-by: Stefan Hajnoczi Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 2 +- 1 file changed, 1 insertion(+)

[PATCH v6 15/18] job: detect change of aiocontext within job coroutine

2022-03-14 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini We want to make sure access of job->aio_context is always done under either BQL or job_mutex. The problem is that using aio_co_enter(job->aiocontext, job->co) in job_start and job_enter_cond makes the coroutine immediately resume, so we can't hold the job lock. And caching it

[PATCH v6 06/18] jobs: protect jobs with job_lock/unlock

2022-03-14 Thread Emanuele Giuseppe Esposito
Introduce the job locking mechanism through the whole job API, following the comments in job.h and requirements of job-monitor (like the functions in job-qmp.c, assume lock is held) and job-driver (like in mirror.c and all other JobDriver, lock is not held). Use the _locked helpers introduced bef

[PATCH v6 01/18] job.c: make job_mutex and job_lock/unlock() public

2022-03-14 Thread Emanuele Giuseppe Esposito
job mutex will be used to protect the job struct elements and list, replacing AioContext locks. Right now use a shared lock for all jobs, in order to keep things simple. Once the AioContext lock is gone, we can introduce per-job locks. To simplify the switch from aiocontext to job lock, introduce

[PATCH v6 17/18] job.c: enable job lock/unlock and remove Aiocontext locks

2022-03-14 Thread Emanuele Giuseppe Esposito
Change the job_{lock/unlock} and macros to use job_mutex. Now that they are not nop anymore, remove the aiocontext to avoid deadlocks. Therefore: - when possible, remove completely the aiocontext lock/unlock pair - if it is used by some other function too, reduce the locking section as much as po

[PATCH v6 03/18] job.c: API functions not used outside should be static

2022-03-14 Thread Emanuele Giuseppe Esposito
job_event_* functions can all be static, as they are not used outside job.c. Same applies for job_txn_add_job(). Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 18 -- job.c | 22 +++--- 2 files changed, 19 insertions(+), 21 deleti

Re: [PATCH v2 4/4] util/event-loop-base: Introduce options to set the thread pool size

2022-03-14 Thread Stefan Hajnoczi
On Fri, Mar 11, 2022 at 11:40:30AM +0100, Nicolas Saenz Julienne wrote: > On Thu, 2022-03-10 at 10:45 +, Stefan Hajnoczi wrote: > > On Thu, Mar 03, 2022 at 04:13:07PM +0100, Nicolas Saenz Julienne wrote: > > > @@ -537,10 +546,19 @@ > > > # 0 means that the engine will use its d

[PATCH v6 12/18] block_job: rename block_job functions called with job_mutex held

2022-03-14 Thread Emanuele Giuseppe Esposito
Just as for the job API, rename block_job functions that are always called under job lock. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 3 ++- block/backup.c | 4 ++-- blockdev.c | 12 +++- blockjob.c

[PATCH v6 14/18] commit and mirror: create new nodes using bdrv_get_aio_context, and not the job aiocontext

2022-03-14 Thread Emanuele Giuseppe Esposito
We are always using the given bs AioContext, so there is no need to take the job ones (which is identical anyways). This also reduces the point we need to check when protecting job.aio_context field. Reviewed-by: Stefan Hajnoczi Signed-off-by: Emanuele Giuseppe Esposito --- block/commit.c | 4 +

Re: [PATCH experiment 08/16] tracetool: add extern "C" around generated headers

2022-03-14 Thread Philippe Mathieu-Daudé
On 14/3/22 10:31, Paolo Bonzini wrote: Signed-off-by: Paolo Bonzini --- scripts/tracetool/format/h.py | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/scripts/tracetool/format/h.py b/scripts/tracetool/format/h.py index e94f0be7da..2d92fa8bd2 100644 --- a/scripts/trac

[PATCH v6 08/18] jobs: use job locks also in the unit tests

2022-03-14 Thread Emanuele Giuseppe Esposito
Add missing job synchronization in the unit tests, with explicit locks. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit/test-bdrv-drain.c | 76 - tests/unit/test-block-iothread.c |

[PATCH v6 07/18] jobs: add job lock in find_* functions

2022-03-14 Thread Emanuele Giuseppe Esposito
Both blockdev.c and job-qmp.c have TOC/TOU conditions, because they first search for the job and then perform an action on it. Therefore, we need to do the search + action under the same job mutex critical section. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed

[PATCH v6 02/18] job.h: categorize fields in struct Job

2022-03-14 Thread Emanuele Giuseppe Esposito
Categorize the fields in struct Job to understand which ones need to be protected by the job mutex and which don't. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 59 ++ 1 file changed, 34 insertions(+), 25 deletions(-) diff --git

[PATCH v6 00/18] job: replace AioContext lock with job_mutex

2022-03-14 Thread Emanuele Giuseppe Esposito
In this series, we want to remove the AioContext lock and instead use the already existent job_mutex to protect the job structures and list. This is part of the work to get rid of AioContext lock usage in favour of smaller granularity locks. In order to simplify reviewer's job, job lock/unlock fun

Re: [PATCH v2 2/4] Introduce event-loop-base abstract class

2022-03-14 Thread Stefan Hajnoczi
On Fri, Mar 11, 2022 at 11:17:22AM +0100, Nicolas Saenz Julienne wrote: > On Thu, 2022-03-10 at 10:25 +, Stefan Hajnoczi wrote: > > On Thu, Mar 03, 2022 at 03:58:20PM +0100, Nicolas Saenz Julienne wrote: > > > @@ -2935,13 +2947,6 @@ qemu_syms = custom_target('qemu.syms', output: > > > 'qemu.sy

Re: [PATCH experiment 03/16] coroutine: small code cleanup in qemu_co_rwlock_wrlock

2022-03-14 Thread Philippe Mathieu-Daudé
On 14/3/22 10:31, Paolo Bonzini wrote: qemu_co_rwlock_wrlock stores the current coroutine in a loc variable, use it instead of calling qemu_coroutine_self() again. Signed-off-by: Paolo Bonzini --- util/qemu-coroutine-lock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by:

Re: [PATCH v2 2/4] Introduce event-loop-base abstract class

2022-03-14 Thread Nicolas Saenz Julienne
On Mon, 2022-03-14 at 13:33 +, Stefan Hajnoczi wrote: > On Fri, Mar 11, 2022 at 11:17:22AM +0100, Nicolas Saenz Julienne wrote: > > On Thu, 2022-03-10 at 10:25 +, Stefan Hajnoczi wrote: > > > On Thu, Mar 03, 2022 at 03:58:20PM +0100, Nicolas Saenz Julienne wrote: > > > > @@ -2935,13 +2947,6

[PATCH v2 06/10] test-bdrv-drain.c: adapt test to support additional subtree drains

2022-03-14 Thread Emanuele Giuseppe Esposito
There will be a problem in this test when we will add subtree drains in bdrv_replace_child_noperm: test_detach_indirect is only interested in observing the first call to .drained_begin. In the original test, there was only a single subtree drain; however, with additional drains introduced in bdrv_

[PATCH v2 07/10] test-bdrv-drain.c: remove test_detach_by_parent_cb()

2022-03-14 Thread Emanuele Giuseppe Esposito
This test uses a callback of an I/O function (blk_aio_preadv) to modify the graph, using bdrv_attach_child. This is simply not allowed anymore. I/O cannot change the graph. Before "block/io.c: make bdrv_do_drained_begin_quiesce static and introduce bdrv_drained_begin_no_poll", the test would simpl

[PATCH v2 02/10] bdrv_parent_drained_begin_single: handle calls from coroutine context

2022-03-14 Thread Emanuele Giuseppe Esposito
bdrv_parent_drained_begin_single() is also called by bdrv_replace_child_noperm(). The latter is often called from coroutines, for example in bdrv_co_create_opts() callbacks. This can potentially create deadlocks, because if the drain_saldo in bdrv_replace_child_noperm is > 0, the coroutine will st

[PATCH v2 03/10] block/io.c: fix bdrv_child_cb_drained_begin invocations from a coroutine

2022-03-14 Thread Emanuele Giuseppe Esposito
Using bdrv_do_drained_begin_quiesce() in bdrv_child_cb_drained_begin() is not a good idea: the callback might be called when running a drain in a coroutine, and bdrv_drained_begin_poll() does not handle that case, resulting in assertion failure. Instead, bdrv_do_drained_begin with no recursion and

[PATCH v2 04/10] block.c: bdrv_replace_child_noperm: first remove the child, and then call ->detach()

2022-03-14 Thread Emanuele Giuseppe Esposito
Doing the opposite can make ->detach() (more precisely bdrv_unapply_subtree_drain() in bdrv_child_cb_detach) undo the subtree_drain just performed to protect the removal of the child from the graph, thus making the fully-enabled assert_bdrv_graph_writable fail. Note that assert_bdrv_graph_writable

[PATCH v2 05/10] block.c: bdrv_replace_child_noperm: first call ->attach(), and then add child

2022-03-14 Thread Emanuele Giuseppe Esposito
Doing the opposite can make adding the child node to a non-drained node, as apply_subtree_drain is only done in ->attach() and thus make assert_bdrv_graph_writable fail. This can happen for example during a transaction rollback (test 245, test_io_with_graph_changes): 1. a node is removed from the

[PATCH v2 10/10] tests/qemu-iotests/030: test_stream_parallel should use auto_finalize=False

2022-03-14 Thread Emanuele Giuseppe Esposito
First, use run_job() instead of the current logic to run the stream job. Then, use auto_finalize=False to be sure that the job is not automatically deleted once it is done. In this way, if the job finishes before we want, it is not finalized yet so the other commands can still execute without fail

[PATCH v2 01/10] drains: create bh only when polling

2022-03-14 Thread Emanuele Giuseppe Esposito
We need to prevent coroutines from calling BDRV_POLL_WHILE, because it can create deadlocks. This is done by firstly creating a bottom half and then yielding. The bh is then scheduled in the main loop, performs the drain and polling, and then resumes the coroutine. The problem is that currently we

[PATCH v2 00/10] block: bug fixes in preparation of AioContext removal

2022-03-14 Thread Emanuele Giuseppe Esposito
This serie aims to remove and clean up some bugs that came up when trying to replace the AioContext lock and still protect BlockDriverState fields. They were part of the serie "Removal of Aiocontext lock through drains: protect bdrv_replace_child_noperm", but since that serie is still a work in pr

Re: [PATCH experiment 06/16] use g_new0 instead of g_malloc0

2022-03-14 Thread Markus Armbruster
Paolo Bonzini writes: > Casting to/from void* must be explicit in C++. g_new0 takes care of that. > > Signed-off-by: Paolo Bonzini > --- > include/qemu/timer.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/include/qemu/timer.h b/include/qemu/timer.h > index 88ef11468

Re: [PATCH v2] block-qdict: Fix -Werror=maybe-uninitialized build failure

2022-03-14 Thread Markus Armbruster
Murilo Opsfelder Araujo writes: > Building QEMU on Fedora 37 (Rawhide Prerelease) ppc64le failed with the > following error: > > $ ../configure --prefix=/usr/local/qemu-disabletcg > --target-list=ppc-softmmu,ppc64-softmmu --disable-tcg --disable-linux-user > ... > $ make -j$(nproc) >

[PATCH experiment 15/16] port QemuCoLockable to C++ coroutines

2022-03-14 Thread Paolo Bonzini
Convert "T coroutine_fn" annotations to the new type CoroutineFn, and add co_await as needed. _Generic is replaced by an overloaded constructor. C++ also does not like & on a temporary, so that is replaced by a function qemu_make_co_lockable_nonnull that hides it from the compiler. Signed-off-by

[PATCH experiment 14/16] util: introduce C++ stackless coroutine backend

2022-03-14 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini --- configure | 44 +- include/qemu/coroutine.h | 445 +- include/qemu/coroutine_int.h | 8 + util/coroutine-stackless.cc | 145 ++ util/meson.build

[PATCH experiment 10/16] add space between liter and string macro

2022-03-14 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini --- util/trace-events | 28 ++-- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/util/trace-events b/util/trace-events index c8f53d7d9f..5bc718eff7 100644 --- a/util/trace-events +++ b/util/trace-events @@ -1,10 +1,10 @@ # See

[PATCH experiment 13/16] disable some code

2022-03-14 Thread Paolo Bonzini
Disable a lot of code that I can't be bothered to convert right now. Signed-off-by: Paolo Bonzini --- include/qemu/coroutine.h| 2 ++ qapi/qmp-dispatch.c | 2 ++ tests/unit/meson.build | 2 +- tests/unit/test-coroutine.c | 6 ++ util/async.c| 2 ++ util/

[PATCH experiment 16/16] port test-coroutine to C++ coroutines

2022-03-14 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini --- tests/unit/meson.build| 6 +- .../{test-coroutine.c => test-coroutine.cc} | 140 +- 2 files changed, 76 insertions(+), 70 deletions(-) rename tests/unit/{test-coroutine.c => test-coroutine.cc} (81%) diff --git a/tests/

[PATCH experiment 09/16] start adding extern "C" markers

2022-03-14 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini --- include/block/aio.h | 5 + include/fpu/softfloat-types.h | 4 include/qemu/bitops.h | 3 +++ include/qemu/bswap.h | 10 +++--- include/qemu/coroutine.h | 4 include/qemu/host-utils.h | 4 include/qemu

[PATCH experiment 11/16] bump to C++20

2022-03-14 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini --- configure | 4 ++-- meson.build | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/configure b/configure index 886000346a..091710ec03 100755 --- a/configure +++ b/configure @@ -157,8 +157,8 @@ update_cxxflags() { # Set QEMU_CXXFLAGS from

[PATCH experiment 12/16] remove "new" keyword from trace-events

2022-03-14 Thread Paolo Bonzini
This is invalid in C++. Signed-off-by: Paolo Bonzini --- util/trace-events | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/util/trace-events b/util/trace-events index 5bc718eff7..9e23c11d11 100644 --- a/util/trace-events +++ b/util/trace-events @@ -3,8 +3,8 @

[PATCH experiment 02/16] coroutine: qemu_coroutine_get_aio_context is not a coroutine_fn

2022-03-14 Thread Paolo Bonzini
Since it operates on a given coroutine, qemu_coroutine_get_aio_context can be called from outside coroutine context. This is for example how qio_channel_restart_read uses it. Signed-off-by: Paolo Bonzini --- include/qemu/coroutine.h | 2 +- util/qemu-coroutine.c| 2 +- 2 files changed, 2 in

[PATCH experiment 08/16] tracetool: add extern "C" around generated headers

2022-03-14 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini --- scripts/tracetool/format/h.py | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/scripts/tracetool/format/h.py b/scripts/tracetool/format/h.py index e94f0be7da..2d92fa8bd2 100644 --- a/scripts/tracetool/format/h.py +++ b/scripts/tracetool/f

[PATCH experiment 07/16] start porting compiler.h to C++

2022-03-14 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini --- include/qemu/compiler.h | 4 1 file changed, 4 insertions(+) diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h index 3baa5e3790..18848f0d49 100644 --- a/include/qemu/compiler.h +++ b/include/qemu/compiler.h @@ -72,6 +72,10 @@ int:(x) ?

  1   2   >