[PATCH v5 18/20] jobs: protect job.aio_context with BQL and job_mutex

2022-02-08 Thread Emanuele Giuseppe Esposito
introduce job_set_aio_context and make sure that the context is set under BQL, job_mutex and drain. Also make sure all other places where the aiocontext is read are protected. Suggested-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/replication.c | 2 +- blockjob.c

[PATCH v5 17/20] job: detect change of aiocontext within job coroutine

2022-02-08 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini We want to make sure access of job->aio_context is always done under either BQL or job_mutex. The problem is that using aio_co_enter(job->aiocontext, job->co) in job_start and job_enter_cond makes the coroutine immediately resume, so we can't hold the job lock. And caching it

[PATCH v2 2/3] jobs: add job-monitor.h

2022-02-08 Thread Emanuele Giuseppe Esposito
job-monitor.h contains all functions of job.h that are used by the monitor and essentially all functions that do not define a JobDriver/Blockdriver. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job-monitor.h | 249

[PATCH 5/6] test-bdrv-drain.c: remove test_detach_by_parent_cb()

2022-02-08 Thread Emanuele Giuseppe Esposito
s://lists.nongnu.org/archive/html/qemu-block/2018-05/msg01132.html but as explained above I believe that it is not valid anymore, and can be discarded. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit/test-bdrv-drain.c | 46 +--- 1 file changed, 11 inserti

[PATCH 2/6] block.c: bdrv_replace_child_noperm: first remove the child, and then call ->detach()

2022-02-08 Thread Emanuele Giuseppe Esposito
ote that assert_bdrv_graph_writable is not yet fully enabled. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/block.c b/block.c index 4551eba2aa..ec346a7e2e 100644 --- a/block.c +++ b/block.c @@ -2854,14 +2854,16 @@ static v

[PATCH v2 3/3] jobs: add job-driver.h

2022-02-08 Thread Emanuele Giuseppe Esposito
job-driver.h contains all functions of job.h that are used by the drivers (JobDriver, BlockJobDriver). These functions are unaware of the job_mutex, so they all take and release the lock internally. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job

[PATCH 1/6] block/io.c: fix bdrv_child_cb_drained_begin invocations from a coroutine

2022-02-08 Thread Emanuele Giuseppe Esposito
and poll will accomplish the same thing (invoking bdrv_do_drained_begin_quiesce) but will firstly check if we are already in a coroutine, and exit from that via bdrv_co_yield_to_drain(). Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 2 +- block/io.c | 7

[PATCH 5/5] block: move BQL logic of bdrv_co_invalidate_cache in bdrv_activate

2022-02-09 Thread Emanuele Giuseppe Esposito
ore, and so is every direct call to bdrv_invalidate_cache(). Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 37 +++-- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/block.c b/block.c index 5e65f134f8..df353d55e8 100644 --- a/block.c ++

[PATCH 1/5] crypto: perform permission checks under BQL

2022-02-09 Thread Emanuele Giuseppe Esposito
-by: Emanuele Giuseppe Esposito --- block/amend.c | 24 block/crypto.c| 27 +++ include/block/block_int.h | 14 ++ 3 files changed, 65 insertions(+) diff --git a/block/amend.c b/block/amend.c index 392df9ef83

[PATCH 0/5] block layer: permission API refactoring in preparation

2022-02-09 Thread Emanuele Giuseppe Esposito
take care of bdrv_invalidate_cache and callers, since this function checks too for permisisons while being called by an iothread. Emanuele Giuseppe Esposito (5): crypto: perform permission checks under BQL crypto: distinguish between main loop and I/O in block_crypto_amend_options_generic_luks block:

[PATCH 3/5] block: introduce bdrv_activate

2022-02-09 Thread Emanuele Giuseppe Esposito
-off-by: Emanuele Giuseppe Esposito Reviewed-by: Hanna Reitz --- block.c | 7 ++- block/block-backend.c| 2 +- block/export/export.c| 2 +- block/parallels.c| 2 +- include/block/block.h| 1 + tests/unit/test-block

[PATCH 4/5] block: rename bdrv_invalidate_cache_all, blk_invalidate_cache and test_sync_op_invalidate_cache

2022-02-09 Thread Emanuele Giuseppe Esposito
Following the bdrv_activate renaming, change also the name of the respective callers. bdrv_invalidate_cache_all -> bdrv_activate_all blk_invalidate_cache -> blk_activate test_sync_op_invalidate_cache -> test_sync_op_activate No functional change intended. Signed-off-by: Emanuele

[PATCH 2/5] crypto: distinguish between main loop and I/O in block_crypto_amend_options_generic_luks

2022-02-09 Thread Emanuele Giuseppe Esposito
for permissions but delegate .bdrv_amend_pre_run() and .bdrv_amend_clean() to do it, performing these checks before and after the job runs in its aiocontext. Signed-off-by: Emanuele Giuseppe Esposito --- block/crypto.c | 35 +++ 1 file changed, 15 insertions(+), 20

Re: [PATCH 4/6] test-bdrv-drain.c: adapt test to the coming subtree drains

2022-02-10 Thread Emanuele Giuseppe Esposito
On 10/02/2022 15:32, Stefan Hajnoczi wrote: > On Tue, Feb 08, 2022 at 10:36:53AM -0500, Emanuele Giuseppe Esposito wrote: >> There will be 2 problems in this test when we will add >> subtree drains in bdrv_replace_child_noperm: >> >> - First, the test is inconsistent

Re: [PATCH v5 02/20] job.h: categorize fields in struct Job

2022-02-10 Thread Emanuele Giuseppe Esposito
On 10/02/2022 16:40, Stefan Hajnoczi wrote: > On Tue, Feb 08, 2022 at 09:34:55AM -0500, Emanuele Giuseppe Esposito wrote: >> Categorize the fields in struct Job to understand which ones >> need to be protected by the job mutex and which don't. >> >> Signed-off-by:

Re: [PATCH v6 00/33] block layer: split block APIs in global state and I/O

2022-02-10 Thread Emanuele Giuseppe Esposito
On 08/02/2022 14:08, Kevin Wolf wrote: > Am 08.02.2022 um 12:42 hat Emanuele Giuseppe Esposito geschrieben: >> >> >> On 07/02/2022 19:30, Kevin Wolf wrote: >>> Am 21.01.2022 um 18:05 hat Emanuele Giuseppe Esposito geschrieben: >>>> Each function in t

Re: [PATCH v5 02/20] job.h: categorize fields in struct Job

2022-02-11 Thread Emanuele Giuseppe Esposito
On 10/02/2022 18:35, Stefan Hajnoczi wrote: > On Thu, Feb 10, 2022 at 05:26:52PM +0100, Emanuele Giuseppe Esposito wrote: >> >> >> On 10/02/2022 16:40, Stefan Hajnoczi wrote: >>> On Tue, Feb 08, 2022 at 09:34:55AM -0500, Emanuele Giuseppe Esposito wrote: >>&g

[PATCH v7 00/31] block layer: split block APIs in global state and I/O

2022-02-11 Thread Emanuele Giuseppe Esposito
ing in preparation to the API split" Based-on: <20220209105452.1694545-1-eespo...@redhat.com> Signed-off-by: Emanuele Giuseppe Esposito --- v7: * crypto permissions and bdrv-activate patches sent in another serie * (*bdrv_probe) and (*get_name) are I/O * add missing licens

[PATCH v7 01/31] main-loop.h: introduce qemu_in_main_thread()

2022-02-11 Thread Emanuele Giuseppe Esposito
in next patches fail despite the AioContext is still the main loop. See the comment in the function header for more information. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/main-loop.h | 24 softmmu/cpus.c | 5 + stubs/iothread-lock.c| 5

[PATCH v7 12/31] assertions for block_int global state API

2022-02-11 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 15 +++ block/backup.c | 1 + block/block-backend.c | 3 +++ block/commit.c | 2 ++ block/dirty-bitmap.c| 1 + block/io.c

[PATCH v7 15/31] include/block/blockjob_int.h: split header into I/O and GS API

2022-02-11 Thread Emanuele Giuseppe Esposito
Since the I/O functions are not many, keep a single file. Also split the function pointers in BlockJobDriver. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/block/blockjob_int.h | 28 1 file changed, 28 insertions(+) diff --git

[PATCH v7 04/31] assertions for block global state API

2022-02-11 Thread Emanuele Giuseppe Esposito
All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 125

[PATCH v7 08/31] block/block-backend.c: assertions for block-backend

2022-02-11 Thread Emanuele Giuseppe Esposito
All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 79

[PATCH v7 11/31] include/block/block_int: split header into I/O and global state API

2022-02-11 Thread Emanuele Giuseppe Esposito
-by: Emanuele Giuseppe Esposito --- blockdev.c |5 + include/block/block_int-common.h | 1180 +++ include/block/block_int-global-state.h | 312 + include/block/block_int-io.h | 179 +++ include/block/block_int.h | 1489

[PATCH v7 02/31] main loop: macros to mark GS and I/O functions

2022-02-11 Thread Emanuele Giuseppe Esposito
Righ now, IO_CODE and IO_OR_GS_CODE are nop, as there isn't really a way to check that a function is only called in I/O. On the other side, we can use qemu_in_main_thread to check if we are in the main loop. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/main-loop.h | 9 + 1

[PATCH v7 14/31] block: introduce assert_bdrv_graph_writable

2022-02-11 Thread Emanuele Giuseppe Esposito
. Because adding drains requires additional discussions, they will be added in future series. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 4 include/block/block_int-global-state.h | 17 + 2 files changed, 21 insertions(+) diff --git

Re: [PATCH v6 01/33] main-loop.h: introduce qemu_in_main_thread()

2022-01-31 Thread Emanuele Giuseppe Esposito
On 27/01/2022 11:56, Kevin Wolf wrote: > Am 21.01.2022 um 18:05 hat Emanuele Giuseppe Esposito geschrieben: >> When invoked from the main loop, this function is the same >> as qemu_mutex_iothread_locked, and returns true if the BQL is held. > > So its name is misleadi

[PATCH] block.h: remove outdated comment

2022-01-31 Thread Emanuele Giuseppe Esposito
: Emanuele Giuseppe Esposito --- include/block/block.h | 1 - 1 file changed, 1 deletion(-) diff --git a/include/block/block.h b/include/block/block.h index 9d4050220b..e1713ee306 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -344,7 +344,6 @@ typedef unsigned int BdrvChildR

Re: [PATCH v6 02/33] include/block/block: split header into I/O and global state API

2022-01-31 Thread Emanuele Giuseppe Esposito
On 27/01/2022 12:01, Kevin Wolf wrote: >> +/* Common functions that are neither I/O nor Global State */ >> + >> +int bdrv_parse_aio(const char *mode, int *flags); > Makes sense to me to have this here, it is just a helper function that > parses stuff and doesn't touch any state. However, what is

[PATCH 02/12] block/io.c: make bdrv_do_drained_begin_quiesce static and introduce bdrv_drained_begin_no_poll

2022-01-18 Thread Emanuele Giuseppe Esposito
and poll will accomplish the same thing (invoking bdrv_do_drained_begin_quiesce) but will firstly check if we are already in a coroutine, and exit from that via bdrv_co_yield_to_drain(). Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 2 +- block/io.c | 7

[PATCH 00/12] Removal of Aiocontext lock through drains: protect bdrv_replace_child_noperm.

2022-01-18 Thread Emanuele Giuseppe Esposito
able(), checking also for the drains, in patch 11. This series is based on "job: replace AioContext lock with job_mutex" that in turns is based on the block API split ("block layer: split block APIs in global state and I/O"). Based-on: <20220105140208.365608-1-eespo...@redh

[PATCH 01/12] introduce BDRV_POLL_WHILE_UNLOCKED

2022-01-18 Thread Emanuele Giuseppe Esposito
Same as BDRV_POLL_WHILE, but uses AIO_WAIT_WHILE_UNLOCKED. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block-global-state.h | 5 + 1 file changed, 5 insertions(+) diff --git a/include/block/block-global-state.h b/include/block/block-global-state.h index 419fe8427f

[PATCH 09/12] jobs: ensure sleep in job_sleep_ns is fully performed

2022-01-18 Thread Emanuele Giuseppe Esposito
sserts. Signed-off-by: Emanuele Giuseppe Esposito --- job.c | 28 +--- tests/qemu-iotests/030 | 2 +- tests/qemu-iotests/151 | 4 ++-- tests/unit/test-blockjob.c | 2 +- 4 files changed, 21 insertions(+), 15 deletions(-) diff --git a/

[PATCH 05/12] test-bdrv-drain.c: adapt test to the coming subtree drains

2022-01-18 Thread Emanuele Giuseppe Esposito
.drained_begin was not a problem. Now that we want to have additional subtree drains, we risk to call the test callback to early, or multiple times. We do not want that, so override the callback only when we actually want to use it. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit/test-bdrv-drain.c

[PATCH 03/12] block.c: bdrv_replace_child_noperm: first remove the child, and then call ->detach()

2022-01-18 Thread Emanuele Giuseppe Esposito
ote that assert_bdrv_graph_writable is not yet fully enabled. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/block.c b/block.c index 08fde585f4..29de2b62b5 100644 --- a/block.c +++ b/block.c @@ -2861,14 +2861,16 @@ static v

[PATCH 06/12] test-bdrv-drain.c: remove test_detach_by_parent_cb()

2022-01-18 Thread Emanuele Giuseppe Esposito
s://lists.nongnu.org/archive/html/qemu-block/2018-05/msg01132.html but as explained above I believe that it is not valid anymore, and can be discarded. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit/test-bdrv-drain.c | 46 +--- 1 file changed, 11 inserti

[PATCH 11/12] block/io.c: fully enable assert_bdrv_graph_writable

2022-01-18 Thread Emanuele Giuseppe Esposito
As explained in the TODO, complete the function by checking that the node is also drained. In this way, we can ensure that modify the bs is thread safe, as the drain makes sure that no I/O concurrently reads the field, and all writes are under BQL. Signed-off-by: Emanuele Giuseppe Esposito

[PATCH 04/12] block.c: bdrv_replace_child_noperm: first call ->attach(), and then add child

2022-01-18 Thread Emanuele Giuseppe Esposito
toring the drain with apply_subtree_drain(), leaving the node undrained between the two operations. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/block.c b/block.c index 29de2b62b5..fb5bc3077a 100644 --- a/block.c

[PATCH 08/12] reopen: add a transaction to drain_end nodes picked in bdrv_reopen_parse_file_or_backing

2022-01-18 Thread Emanuele Giuseppe Esposito
that just executes subtree_drained_end to match the drained_begin done in this function. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 25 - 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/block.c b/block.c index fb5bc3077a..fcc44a49a0 100644

[PATCH 10/12] block.c: add subtree_drains where needed

2022-01-18 Thread Emanuele Giuseppe Esposito
() releasing and then holding the AioContext lock, since it later invokes bdrv_try_set_aio_context() that is not safe yet. Once all is cleaned up, we can also remove the acquire/release locks in job_unref, artificially added because of this. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 50

[PATCH 12/12] block.c: additional assert qemu in main tread

2022-01-18 Thread Emanuele Giuseppe Esposito
Add some missing assertion in static functions of block.c Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 2 ++ block/block-backend.c | 3 +++ 2 files changed, 5 insertions(+) diff --git a/block.c b/block.c index 6196c95aae..7961f5a984 100644 --- a/block.c +++ b/block.c

[PATCH 07/12] block/io.c: introduce bdrv_subtree_drained_{begin/end}_unlocked

2022-01-18 Thread Emanuele Giuseppe Esposito
Same as the locked version, but use BDRV_POLL_UNLOCKED Signed-off-by: Emanuele Giuseppe Esposito --- block/io.c | 50 +--- include/block/block-io.h | 2 ++ 2 files changed, 39 insertions(+), 13 deletions(-) diff --git a/block/io.c b/block/io.c

Re: [PATCH v5 28/31] block.c: assert BQL lock held in bdrv_co_invalidate_cache

2022-01-21 Thread Emanuele Giuseppe Esposito
On 20/01/2022 14:48, Kevin Wolf wrote: Am 20.01.2022 um 14:22 hat Paolo Bonzini geschrieben: On 1/19/22 19:34, Kevin Wolf wrote: So if we go back to a bdrv_invalidate_cache() that does all the graph manipulations (and asserts that we're in the main loop) and then have a much smaller

Re: [PATCH v3 03/16] job.h: define locked functions

2022-01-21 Thread Emanuele Giuseppe Esposito
On 19/01/2022 11:44, Paolo Bonzini wrote: On 1/5/22 15:01, Emanuele Giuseppe Esposito wrote: These functions assume that the job lock is held by the caller, to avoid TOC/TOU conditions. Therefore, their name must end with _locked. Introduce also additional helpers that define _locked

Re: [PATCH v3 14/16] job.c: use job_get_aio_context()

2022-01-21 Thread Emanuele Giuseppe Esposito
On 19/01/2022 11:31, Paolo Bonzini wrote: diff --git a/job.c b/job.c index f16a4ef542..8a5b710d9b 100644 --- a/job.c +++ b/job.c @@ -566,7 +566,7 @@ void job_enter_cond_locked(Job *job, bool(*fn)(Job *job))   job->busy = true;   real_job_unlock();   job_unlock(); -   

Re: [PATCH v3 14/16] job.c: use job_get_aio_context()

2022-01-21 Thread Emanuele Giuseppe Esposito
On 19/01/2022 11:31, Paolo Bonzini wrote: diff --git a/blockjob.c b/blockjob.c index cf1f49f6c2..468ba735c5 100644 --- a/blockjob.c +++ b/blockjob.c @@ -155,14 +155,16 @@ static void child_job_set_aio_ctx(BdrvChild *c, AioContext *ctx,   bdrv_set_aio_context_ignore(sibling->bs, ctx,

[PATCH v6 07/33] include/block/block_int: split header into I/O and global state API

2022-01-21 Thread Emanuele Giuseppe Esposito
-by: Emanuele Giuseppe Esposito --- blockdev.c |5 + include/block/block_int-common.h | 1161 +++ include/block/block_int-global-state.h | 312 + include/block/block_int-io.h | 170 +++ include/block/block_int.h | 1475

[PATCH v6 05/33] include/sysemu/block-backend: split header into I/O and global state (GS) API

2022-01-21 Thread Emanuele Giuseppe Esposito
ared between the two headers, and the functions that can't be categorized as I/O or global state. Assertions are added in the next patch. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 9 +- include/sysemu/block-backend-common.h | 84 ++

[PATCH v6 08/33] assertions for block_int global state API

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 15 +++ block/backup.c | 1 + block/block-backend.c | 3 +++ block/commit.c | 2 ++ block/dirty-bitmap.c| 1 + block/io.c

[PATCH v6 15/33] include/sysemu/blockdev.h: global state API

2022-01-21 Thread Emanuele Giuseppe Esposito
blockdev functions run always under the BQL lock. Signed-off-by: Emanuele Giuseppe Esposito --- include/sysemu/blockdev.h | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index f9fb54d437..a420bbc630 100644

[PATCH v6 10/33] include/block/blockjob_int.h: split header into I/O and GS API

2022-01-21 Thread Emanuele Giuseppe Esposito
Since the I/O functions are not many, keep a single file. Also split the function pointers in BlockJobDriver. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/block/blockjob_int.h | 28 1 file changed, 28 insertions(+) diff --git

[PATCH v6 18/33] block/copy-before-write.h: global state API + assertions

2022-01-21 Thread Emanuele Giuseppe Esposito
copy-before-write functions always run under BQL lock. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/copy-before-write.c | 2 ++ block/copy-before-write.h | 7 +++ 2 files changed, 9 insertions(+) diff --git a/block/copy

Re: [PATCH v3 14/16] job.c: use job_get_aio_context()

2022-01-21 Thread Emanuele Giuseppe Esposito
On 21/01/2022 13:33, Emanuele Giuseppe Esposito wrote: On 19/01/2022 11:31, Paolo Bonzini wrote: diff --git a/blockjob.c b/blockjob.c index cf1f49f6c2..468ba735c5 100644 --- a/blockjob.c +++ b/blockjob.c @@ -155,14 +155,16 @@ static void child_job_set_aio_ctx(BdrvChild *c, AioContext *ctx

[PATCH v6 16/33] assertions for blockdev.h global state API

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 3 +++ blockdev.c| 16 2 files changed, 19 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index fe56cc1cf4

[PATCH v6 32/33] crypto: delegate permission functions to JobDriver .pre_run

2022-01-21 Thread Emanuele Giuseppe Esposito
rt(), now it is the moment to use it. Signed-off-by: Emanuele Giuseppe Esposito --- block/crypto.c | 57 -- job.c | 13 2 files changed, 50 insertions(+), 20 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index f

[PATCH v6 06/33] block/block-backend.c: assertions for block-backend

2022-01-21 Thread Emanuele Giuseppe Esposito
All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 79

[PATCH v6 11/33] assertions for blockjob_int.h

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockjob.c | 4 1 file changed, 4 insertions(+) diff --git a/blockjob.c b/blockjob.c index 10815a89fe..32fe27a6d4 100644 --- a/blockjob.c +++ b/blockjob.c @@ -84,6 +84,7 @@ BlockJob *block_job_get(const char *id

[PATCH v6 17/33] include/block/snapshot: global state API + assertions

2022-01-21 Thread Emanuele Giuseppe Esposito
Snapshots run also under the BQL lock, so they all are in the global state API. The aiocontext lock that they hold is currently an overkill and in future could be removed. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/snapshot.c

[PATCH v6 14/33] assertions for blockjob.h global state API

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockjob.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/blockjob.c b/blockjob.c index 32fe27a6d4..cce8146cb6 100644 --- a/blockjob.c +++ b/blockjob.c @@ -62,6 +62,7 @@ static bool is_block_job(Job *job

[PATCH v6 24/33] block_int-common.h: assertions in the callers of BlockDriver function pointers

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 18 ++ block/create.c | 2 ++ 2 files changed, 20 insertions(+) diff --git a/block.c b/block.c index bad834c86e..448fb9d76f 100644 --- a/block.c +++ b/block.c @@ -526,6 +526,7 @@ static void coroutine_fn

[PATCH v6 26/33] block_int-common.h: assertions in the callers of BdrvChildClass function pointers

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 9 + 1 file changed, 9 insertions(+) diff --git a/block.c b/block.c index 448fb9d76f..ca16d90627 100644 --- a/block.c +++ b/block.c @@ -1491,6 +1491,7 @@ const BdrvChildClass child_of_bds = { AioContext

[PATCH v6 27/33] block-backend-common.h: split function pointers in BlockDevOps

2022-01-21 Thread Emanuele Giuseppe Esposito
Assertions in the callers of the function pointrs are already added by previous patches. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé --- include/sysemu/block-backend-common.h | 28 ++- 1 file changed, 23

[PATCH v6 20/33] block: rename bdrv_invalidate_cache_all, blk_invalidate_cache and test_sync_op_invalidate_cache

2022-01-21 Thread Emanuele Giuseppe Esposito
Following the bdrv_activate renaming, change also the name of the respective callers. bdrv_invalidate_cache_all -> bdrv_activate_all blk_invalidate_cache -> blk_activate test_sync_op_invalidate_cache -> test_sync_op_activate No functional change intended. Signed-off-by: Emanuele

[PATCH v6 29/33] job.h: assertions in the callers of JobDriver funcion pointers

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- job.c | 9 + 1 file changed, 9 insertions(+) diff --git a/job.c b/job.c index 54db80df66..39bf511949 100644 --- a/job.c +++ b/job.c @@ -381,6 +381,8 @@ void job_ref(Job *job) void job_unref(Job *job) { +assert(qemu_in_main_thread

[PATCH v6 09/33] block: introduce assert_bdrv_graph_writable

2022-01-21 Thread Emanuele Giuseppe Esposito
. Because adding drains requires additional discussions, they will be added in future series. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 4 block/io.c | 11 +++ include/block/block_int-global-state.h | 8

[PATCH v6 25/33] block_int-common.h: split function pointers in BdrvChildClass

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block_int-common.h | 67 +++- 1 file changed, 40 insertions(+), 27 deletions(-) diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h index e007dbf768..cc8c8835ba 100644

[PATCH v6 31/33] include/qemu/job.h: introduce job->pre_run() and use it in amend

2022-01-21 Thread Emanuele Giuseppe Esposito
lse anyways. Signed-off-by: Emanuele Giuseppe Esposito --- block/amend.c | 33 + include/qemu/job.h | 9 + 2 files changed, 42 insertions(+) diff --git a/block/amend.c b/block/amend.c index 392df9ef83..1618fd05a6 100644 --- a/block/amend.c +++ b/bl

[PATCH v6 03/33] assertions for block global state API

2022-01-21 Thread Emanuele Giuseppe Esposito
All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 133

[PATCH v6 01/33] main-loop.h: introduce qemu_in_main_thread()

2022-01-21 Thread Emanuele Giuseppe Esposito
in next patches fail despite the AioContext is still the main loop. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/qemu/main-loop.h | 13 + softmmu/cpus.c | 5 + stubs/iothread-lock.c| 5 + 3 files changed, 23 insertions(+) diff

[PATCH v6 04/33] block/export/fuse.c: allow writable exports to take RESIZE permission

2022-01-21 Thread Emanuele Giuseppe Esposito
to set RESIZE, it will be blocked. Also assert in fuse_do_truncate that if we give the RESIZE permission we can then restore the original ones, since we don't check the return value of blk_set_perm. Signed-off-by: Emanuele Giuseppe Esposito --- block/export/fuse.c | 25 ++--- 1

[PATCH v6 23/33] block_int-common.h: split function pointers in BlockDriver

2022-01-21 Thread Emanuele Giuseppe Esposito
Similar to the header split, also the function pointers in BlockDriver can be split in I/O and global state. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block_int-common.h | 434 --- 1 file changed, 231 insertions(+), 203 deletions(-) diff --git

[PATCH v6 22/33] block/coroutines: I/O API

2022-01-21 Thread Emanuele Giuseppe Esposito
block coroutines functions run in different aiocontext, and are not protected by the BQL. Therefore are I/O. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/coroutines.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/block

[PATCH v6 28/33] job.h: split function pointers in JobDriver

2022-01-21 Thread Emanuele Giuseppe Esposito
The job API will be handled separately in another serie. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 22 ++ 1 file changed, 22 insertions(+) diff --git a/include/qemu/job.h b/include/qemu/job.h index 6e67b6977f..4ea7a4a0cd 100644 --- a/include/qemu

[PATCH v6 30/33] include/block/block_int-common.h: introduce bdrv_amend_pre_run and bdrv_amend_clean

2022-01-21 Thread Emanuele Giuseppe Esposito
ons are restored. Therefore bdrv_amend_pre_run() and bdrv_amend_clean() will take care of just temporarly setting the crypto-specific updating_keys flag. Note that at this stage, they are not yet invoked. Signed-off-by: Emanuele Giuseppe Esposito --- block/crypto.c |

[PATCH v6 12/33] block.c: add assertions to static functions

2022-01-21 Thread Emanuele Giuseppe Esposito
Following the assertion derived from the API split, propagate the assertion also in the static functions. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 47 ++- block/block-backend.c | 3 +++ 2 files changed, 49 insertions(+), 1

[PATCH v6 19/33] block: introduce bdrv_activate

2022-01-21 Thread Emanuele Giuseppe Esposito
-off-by: Emanuele Giuseppe Esposito --- block.c| 7 ++- block/block-backend.c | 2 +- block/export/export.c | 2 +- block/parallels.c | 2 +- include/block/block-global-state.h | 2 +- tests/unit/test-block-iothread.c | 2

[PATCH v6 00/33] block layer: split block APIs in global state and I/O

2022-01-21 Thread Emanuele Giuseppe Esposito
ged, "block: minor refactoring in preparation to the block layer API split". Signed-off-by: Emanuele Giuseppe Esposito --- v6: * Additional assertions in "block.c: add assertions to static functions" * bdrv_co_invalidate_cache: create a new GS function bdrv_activate, a

[PATCH v6 13/33] include/block/blockjob.h: global state API

2022-01-21 Thread Emanuele Giuseppe Esposito
blockjob functions run always under the BQL lock. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/block/blockjob.h | 9 + 1 file changed, 9 insertions(+) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 87fbb3985f..2373dfeb07

[PATCH v6 33/33] block.c: assertions to the block layer permissions API

2022-01-21 Thread Emanuele Giuseppe Esposito
Now that we "covered" the three main cases where the permission API was being used under BQL (fuse, amend and invalidate_cache), we can safely assert for the permission functions implemented in block.c Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 12 1 file c

[PATCH v6 02/33] include/block/block: split header into I/O and global state API

2022-01-21 Thread Emanuele Giuseppe Esposito
earer view on what needs what kind of protection. block-common.h contains common structures shared by both headers. block.h is left there for legacy and to avoid changing all includes in all c files that use the block APIs. Assertions are added in the next patch. Signed-off-by: Emanuele Giusepp

[PATCH v6 21/33] block: move BQL logic of bdrv_co_invalidate_cache in bdrv_activate

2022-01-21 Thread Emanuele Giuseppe Esposito
ect call to bdrv_invalidate_cache(). Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 36 +--- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/block.c b/block.c index 7ab5031027..bad834c86e 100644 --- a/block.c +++ b/block.c @@ -6550,23 +6550

Re: [PATCH v3 09/16] jobs: remove aiocontext locks since the functions are under BQL

2022-01-26 Thread Emanuele Giuseppe Esposito
On 19/01/2022 12:09, Paolo Bonzini wrote: >> @@ -3707,15 +3707,11 @@ BlockJobInfoList *qmp_query_block_jobs(Error >> **errp) >>     for (job = block_job_next(NULL); job; job = >> block_job_next(job)) { >>   BlockJobInfo *value; >> -    AioContext *aio_context; >>     if

Re: [PATCH v3 03/16] job.h: define locked functions

2022-01-26 Thread Emanuele Giuseppe Esposito
On 24/01/2022 15:26, Paolo Bonzini wrote: > On 1/21/22 17:04, Vladimir Sementsov-Ogievskiy wrote: >>> >>> The split was proposed in previous versions, but Vladimir did not >>> really like it and suggested to send it as a separate series: >> >> I didn't really like it as it seemed unusual and

Re: [PATCH v3 14/16] job.c: use job_get_aio_context()

2022-01-26 Thread Emanuele Giuseppe Esposito
On 24/01/2022 15:22, Paolo Bonzini wrote: > On 1/21/22 16:18, Emanuele Giuseppe Esposito wrote: >>>> >>> >>> Better to use aio_co_schedule here, too, and move it under the >>> previous WITH_JOB_LOCK_GUARD. >> >> Unfortunately th

Re: [PATCH] block/stream: Drain subtree around graph change

2022-04-05 Thread Emanuele Giuseppe Esposito
Am 05/04/2022 um 19:53 schrieb Emanuele Giuseppe Esposito: > > > Am 05/04/2022 um 17:04 schrieb Kevin Wolf: >> Am 05.04.2022 um 15:09 hat Emanuele Giuseppe Esposito geschrieben: >>> Am 05/04/2022 um 12:14 schrieb Kevin Wolf: >>>> I think all of this is

Re: [PATCH] block/stream: Drain subtree around graph change

2022-04-05 Thread Emanuele Giuseppe Esposito
Am 05/04/2022 um 17:04 schrieb Kevin Wolf: > Am 05.04.2022 um 15:09 hat Emanuele Giuseppe Esposito geschrieben: >> Am 05/04/2022 um 12:14 schrieb Kevin Wolf: >>> I think all of this is really relevant for Emanuele's work, which >>> involves adding AIO_WAIT_WHILE

Re: [RFC PATCH 0/5] Removal of AioContext lock, bs->parents and ->children: proof of concept

2022-04-13 Thread Emanuele Giuseppe Esposito
So this is a more concrete and up-to-date header. Few things to notice: - we have a list of AioContext. They are registered once an aiocontext is created, and deleted when it is destroyed. This list is helpful because each aiocontext can only modify its own number of readers, avoiding unnecessary

Re: [RFC PATCH 0/5] Removal of AioContext lock, bs->parents and ->children: proof of concept

2022-04-13 Thread Emanuele Giuseppe Esposito
Am 13/04/2022 um 16:51 schrieb Kevin Wolf: > Am 13.04.2022 um 15:43 hat Emanuele Giuseppe Esposito geschrieben: >> So this is a more concrete and up-to-date header. >> >> Few things to notice: >> - we have a list of AioContext. They are registered once an aiocontext

Re: [RFC PATCH 0/5] Removal of AioContext lock, bs->parents and ->children: proof of concept

2022-04-13 Thread Emanuele Giuseppe Esposito
Am 13/04/2022 um 17:14 schrieb Emanuele Giuseppe Esposito: > > > Am 13/04/2022 um 16:51 schrieb Kevin Wolf: >> Am 13.04.2022 um 15:43 hat Emanuele Giuseppe Esposito geschrieben: >>> So this is a more concrete and up-to-date header. >>> >>>

Re: [PATCH 02/21] preallocate: Factor out preallocate_truncate_to_real_size()

2023-08-21 Thread Emanuele Giuseppe Esposito
Am 17/08/2023 um 14:50 schrieb Kevin Wolf: > It's essentially the same code in preallocate_check_perm() and > preallocate_close(), except that the latter ignores errors. > > Signed-off-by: Kevin Wolf > --- Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 12/21] block: Mark bdrv_attach_child() GRAPH_WRLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
t they can't call functions that take it > internally. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 04/21] block: Take AioContext lock for bdrv_append() more consistently

2023-08-21 Thread Emanuele Giuseppe Esposito
Am 17/08/2023 um 14:50 schrieb Kevin Wolf: > The documentation for bdrv_append() says that the caller must hold the > AioContext lock for bs_top. Change all callers to actually adhere to the > contract. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 08/21] block: Mark bdrv_replace_child_noperm() GRAPH_WRLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
t they can't call functions that take it > internally. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 14/21] block: Mark bdrv_get_cumulative_perm() and callers GRAPH_RDLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
river callbacks for updating permissions and annotate the function > pointers with GRAPH_RDLOCK_PTR. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 06/21] block-coroutine-wrapper: Add no_co_wrapper_bdrv_wrlock functions

2023-08-21 Thread Emanuele Giuseppe Esposito
Am 17/08/2023 um 14:50 schrieb Kevin Wolf: > Add a new wrapper type for GRAPH_WRLOCK functions that should be called > from coroutine context. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 18/21] block: Take graph rdlock in bdrv_change_aio_context()

2023-08-21 Thread Emanuele Giuseppe Esposito
Am 17/08/2023 um 14:50 schrieb Kevin Wolf: > The function reads the parents list, so it needs to hold the graph lock. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 15/21] block: Mark bdrv_child_perm() GRAPH_RDLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
niently already hold the lock. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 01/21] block: Remove unused BlockReopenQueueEntry.perms_checked

2023-08-21 Thread Emanuele Giuseppe Esposito
Am 17/08/2023 um 14:50 schrieb Kevin Wolf: > This field has been unused since commit 72373e40fbc ('block: > bdrv_reopen_multiple: refresh permissions on updated graph'). > Remove it. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 13/21] block: Mark bdrv_parent_perms_conflict() and callers GRAPH_RDLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
Am 17/08/2023 um 14:50 schrieb Kevin Wolf: > The function reads the parents list, so it needs to hold the graph lock. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 16/21] block: Mark bdrv_parent_cb_change_media() GRAPH_RDLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
Am 17/08/2023 um 14:50 schrieb Kevin Wolf: > The function reads the parents list, so it needs to hold the graph lock. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

<    4   5   6   7   8   9   10   11   12   13   >