Re: [PATCH 09/21] block: Mark bdrv_replace_child_tran() GRAPH_WRLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
e transaction > callbacks can be changed, too. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 07/21] block-coroutine-wrapper: Allow arbitrary parameter names

2023-08-21 Thread Emanuele Giuseppe Esposito
Am 17/08/2023 um 14:50 schrieb Kevin Wolf: > Don't assume specific parameter names like 'bs' or 'blk' in the > generated code, but use the actual name. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 17/21] block: Take graph rdlock in bdrv_drop_intermediate()

2023-08-21 Thread Emanuele Giuseppe Esposito
Am 17/08/2023 um 14:50 schrieb Kevin Wolf: > The function reads the parents list, so it needs to hold the graph lock. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 21/21] block: Mark bdrv_add/del_child() and caller GRAPH_WRLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
in Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 19/21] block: Mark bdrv_root_unref_child() GRAPH_WRLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
y can't call functions that take it > internally. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

Re: [PATCH 20/21] block: Mark bdrv_unref_child() GRAPH_WRLOCK

2023-08-21 Thread Emanuele Giuseppe Esposito
y can't call functions that take it internally. > > Signed-off-by: Kevin Wolf Reviewed-by: Emanuele Giuseppe Esposito

[RFC PATCH v2 2/8] coroutine-lock: release lock when restarting all coroutines

2022-04-26 Thread Emanuele Giuseppe Esposito
the same functionality that we want. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/coroutine.h | 10 ++ util/qemu-coroutine-lock.c | 26 ++ 2 files changed, 20 insertions(+), 16 deletions(-) diff --git a/include/qemu/coroutine.h b/include/qemu

[RFC PATCH v2 7/8] graph-lock: implement WITH_GRAPH_RDLOCK_GUARD and GRAPH_RDLOCK_GUARD macros

2022-04-26 Thread Emanuele Giuseppe Esposito
Similar to the implementation in lockable.h, implement macros to automatically take and release the rdlock. Create the empty GraphLockable struct only to use it as a type for G_DEFINE_AUTOPTR_CLEANUP_FUNC. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/graph-lock.h | 30

[RFC PATCH v2 3/8] block: introduce a lock to protect graph operations

2022-04-26 Thread Emanuele Giuseppe Esposito
re is the reader, but only if there is one. If instead the original AioContext gets deleted, we need to transfer the current amount of readers in a global shared counter, so that the writer is always aware of all readers. Signed-off-by: Emanuele Giuseppe Esposito --- block/graph-lock

[RFC PATCH v2 8/8] mirror: protect drains in coroutine with rdlock

2022-04-26 Thread Emanuele Giuseppe Esposito
Drain performed in coroutine schedules a bh in the main loop. However, drain itself still is a read, and we need to signal the writer that we are going through the graph. Signed-off-by: Emanuele Giuseppe Esposito --- block/mirror.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/block

[RFC PATCH v2 1/8] aio_wait_kick: add missing memory barrier

2022-04-26 Thread Emanuele Giuseppe Esposito
It seems that aio_wait_kick always required a memory barrier or atomic operation in the caller, but almost nobody actually took care of doing it. Let's put the barrier in the function instead. Signed-off-by: Emanuele Giuseppe Esposito --- util/aio-wait.c | 3 ++- 1 file changed, 2 insertions

[RFC PATCH v2 6/8] block: assert that graph read and writes are performed correctly

2022-04-26 Thread Emanuele Giuseppe Esposito
Remove the old assert_bdrv_graph_writable, and replace it with the new version using graph-lock API. See the function documentation for more information. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 8 block/graph-lock.c

[RFC PATCH v2 0/8] Removal of AioContext lock, bs->parents and ->children: new rwlock

2022-04-26 Thread Emanuele Giuseppe Esposito
omit all functions protected by the added lock to avoid having duplicates when querying for new callbacks. Emanuele Giuseppe Esposito (8): aio_wait_kick: add missing memory barrier coroutine-lock: release lock when restarting all coroutines block: introduce a lock to protect graph operations

[RFC PATCH v2 4/8] async: register/unregister aiocontext in graph lock list

2022-04-26 Thread Emanuele Giuseppe Esposito
Add/remove the AioContext in aio_context_list in graph-lock.c only when it is being effectively created/destroyed. Signed-off-by: Emanuele Giuseppe Esposito --- util/async.c | 4 util/meson.build | 1 + 2 files changed, 5 insertions(+) diff --git a/util/async.c b/util/async.c index

[RFC PATCH v2 5/8] block.c: wrlock in bdrv_replace_child_noperm

2022-04-26 Thread Emanuele Giuseppe Esposito
ection. Therefore change ->attach and ->detach to return true if they are modifying the lock, so that we don't take it twice or release temporarly. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 31 +++ block/block-backend.c

Re: [RFC PATCH v2 0/8] Removal of AioContext lock, bs->parents and ->children: new rwlock

2022-04-27 Thread Emanuele Giuseppe Esposito
Am 26/04/2022 um 10:51 schrieb Emanuele Giuseppe Esposito: > Luckly, most of the cases where we recursively go through a graph are > the BlockDriverState callback functions in block_int-common.h > In order to understand what to protect, I categorized the callbacks in > block_

Re: [RFC PATCH 0/5] Removal of AioContext lock, bs->parents and ->children: proof of concept

2022-03-17 Thread Emanuele Giuseppe Esposito
Am 09/03/2022 um 14:26 schrieb Emanuele Giuseppe Esposito: >>> * Drains allow the caller (either main loop or iothread running >>> the context) to wait all in_flights requests and operations >>> of a BDS: normal drains target a given node and is parents, while &g

Re: [PATCH 06/18] block: Implement blk_{pread, pwrite}() using generated_co_wrapper

2022-05-18 Thread Emanuele Giuseppe Esposito
n the next series? So that I can follow too. Thank you, Emanuele >From 84fcea52c09024adcfe24bb0d6d2ec6842c6826b Mon Sep 17 00:00:00 2001 From: Emanuele Giuseppe Esposito Date: Tue, 17 May 2022 13:35:54 -0400 Subject: [PATCH] block-coroutine-wrapper: remove includes from coroutines.h These incl

Re: [RFC PATCH v2 0/8] Removal of AioContext lock, bs->parents and ->children: new rwlock

2022-05-18 Thread Emanuele Giuseppe Esposito
Am 17/05/2022 um 12:59 schrieb Stefan Hajnoczi: > On Wed, May 04, 2022 at 02:39:05PM +0100, Stefan Hajnoczi wrote: >> On Tue, Apr 26, 2022 at 04:51:06AM -0400, Emanuele Giuseppe Esposito wrote: >>> This is a new attempt to replace the need to take the AioContext lock to

Re: [PATCH] block: drop unused bdrv_co_drain() API

2022-05-23 Thread Emanuele Giuseppe Esposito
drv_drained_end() instead. They are "mixed" > functions that can be called from coroutine context. Unlike > bdrv_co_drain(), these functions provide control of the length of the > drained section, which is usually the right thing. > > Signed-off-by: Stefan Hajnoczi R

Re: [RFC PATCH v2 0/8] Removal of AioContext lock, bs->parents and ->children: new rwlock

2022-05-23 Thread Emanuele Giuseppe Esposito
Am 23/05/2022 um 15:15 schrieb Stefan Hajnoczi: > On Mon, May 23, 2022 at 10:48:48AM +0200, Emanuele Giuseppe Esposito wrote: >> >> >> Am 22/05/2022 um 17:06 schrieb Stefan Hajnoczi: >>> On Wed, May 18, 2022 at 06:14:17PM +0200, Kevin Wolf wrote: >>>>

Re: [RFC PATCH v2 0/8] Removal of AioContext lock, bs->parents and ->children: new rwlock

2022-05-23 Thread Emanuele Giuseppe Esposito
Am 22/05/2022 um 17:06 schrieb Stefan Hajnoczi: > On Wed, May 18, 2022 at 06:14:17PM +0200, Kevin Wolf wrote: >> Am 18.05.2022 um 14:43 hat Paolo Bonzini geschrieben: >>> On 5/18/22 14:28, Emanuele Giuseppe Esposito wrote: >>>> For example, all callers of bdrv_ope

[PATCH] aio_wait_kick: add missing memory barrier

2022-05-24 Thread Emanuele Giuseppe Esposito
. Suggested-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- include/block/aio-wait.h | 2 ++ util/aio-wait.c | 16 +++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h index b39eefb38d..54840f8622

Re: aio_wait_bh_oneshot() thread-safety question

2022-05-24 Thread Emanuele Giuseppe Esposito
Am 24/05/2022 um 09:08 schrieb Paolo Bonzini: > On 5/23/22 18:04, Vladimir Sementsov-Ogievskiy wrote: >> >> I have a doubt about how aio_wait_bh_oneshot() works. Exactly, I see >> that data->done is not accessed atomically, and doesn't have any >> barrier protecting it.. >> >> Is following

Re: [RFC PATCH v2 0/8] Removal of AioContext lock, bs->parents and ->children: new rwlock

2022-05-25 Thread Emanuele Giuseppe Esposito
Am 24/05/2022 um 14:10 schrieb Kevin Wolf: > Am 18.05.2022 um 14:28 hat Emanuele Giuseppe Esposito geschrieben: >> label: // read till the end to see why I wrote this here >> >> I was hoping someone from the "No" party would answer to your question, >> bec

Re: [PATCH v7 10/18] jobs: rename static functions called with job_mutex held

2022-06-24 Thread Emanuele Giuseppe Esposito
Am 24/06/2022 um 17:28 schrieb Paolo Bonzini: > On 6/24/22 16:29, Kevin Wolf wrote: >> Yes, I think Vladimir is having the same difficulties with reading the >> series as I had. And I believe his suggestion would make the >> intermediate states less impossible to review. The question is how

Re: [PATCH v7 10/18] jobs: rename static functions called with job_mutex held

2022-06-23 Thread Emanuele Giuseppe Esposito
Am 22/06/2022 um 20:38 schrieb Vladimir Sementsov-Ogievskiy: > On 6/22/22 17:26, Emanuele Giuseppe Esposito wrote: >> >> >> Am 21/06/2022 um 19:26 schrieb Vladimir Sementsov-Ogievskiy: >>> On 6/16/22 16:18, Emanuele Giuseppe Esposito wrote: >>>> Wit

Re: [PATCH v7 10/18] jobs: rename static functions called with job_mutex held

2022-06-23 Thread Emanuele Giuseppe Esposito
Am 23/06/2022 um 13:10 schrieb Vladimir Sementsov-Ogievskiy: > On 6/23/22 12:08, Emanuele Giuseppe Esposito wrote: >> >> >> Am 22/06/2022 um 20:38 schrieb Vladimir Sementsov-Ogievskiy: >>> On 6/22/22 17:26, Emanuele Giuseppe Esposito wrote: >>>> >>

Re: [PATCH v8 13/20] jobs: group together API calls under the same job lock

2022-07-05 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 10:17 schrieb Emanuele Giuseppe Esposito: > > > Am 05/07/2022 um 10:14 schrieb Stefan Hajnoczi: >> On Wed, Jun 29, 2022 at 10:15:31AM -0400, Emanuele Giuseppe Esposito wrote: >>> diff --git a/blockdev.c b/blockdev.c >>> index 71f793c4ab..5b790

Re: [PATCH 7/8] VirtIOBlock: protect rq with its own lock

2022-07-08 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 16:45 schrieb Stefan Hajnoczi: > On Thu, Jun 09, 2022 at 10:37:26AM -0400, Emanuele Giuseppe Esposito wrote: >> @@ -946,17 +955,20 @@ static void virtio_blk_reset(VirtIODevice *vdev) >> * stops all Iothreads. >> */ >

Re: [PATCH 6/8] virtio-blk: mark IO_CODE functions

2022-07-08 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 16:39 schrieb Stefan Hajnoczi: > On Thu, Jun 09, 2022 at 10:37:25AM -0400, Emanuele Giuseppe Esposito wrote: >> Just as done in the block API, mark functions in virtio-blk >> that are called also from iothread(s). >> >> We know such functions a

Re: AioContext lock removal: help needed

2022-07-08 Thread Emanuele Giuseppe Esposito
Am 08/07/2022 um 10:42 schrieb Emanuele Giuseppe Esposito: > Hello everyone, > > As you all know, I am trying to find a way to replace the well known > AioContext lock with something else that makes sense and provides the > same (or even better) guarantees than using this lock.

Re: [PATCH 1/8] virtio_queue_aio_attach_host_notifier: remove AioContext lock

2022-07-08 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 16:11 schrieb Stefan Hajnoczi: > On Thu, Jun 09, 2022 at 10:37:20AM -0400, Emanuele Giuseppe Esposito wrote: >> @@ -146,7 +147,6 @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev) >> >> s->dataplane_starting = false; >>

Re: [PATCH 7/8] VirtIOBlock: protect rq with its own lock

2022-07-08 Thread Emanuele Giuseppe Esposito
Am 08/07/2022 um 11:33 schrieb Emanuele Giuseppe Esposito: > > > Am 05/07/2022 um 16:45 schrieb Stefan Hajnoczi: >> On Thu, Jun 09, 2022 at 10:37:26AM -0400, Emanuele Giuseppe Esposito wrote: >>> @@ -946,17 +955,20 @@ static void virtio_blk_reset(VirtIODevice *vde

Re: [PATCH 3/8] virtio_blk_process_queued_requests: always run in a bh

2022-07-08 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 16:23 schrieb Stefan Hajnoczi: > On Thu, Jun 09, 2022 at 10:37:22AM -0400, Emanuele Giuseppe Esposito wrote: >> diff --git a/hw/block/dataplane/virtio-blk.c >> b/hw/block/dataplane/virtio-blk.c >> index f9224f23d2..03e10a36a4 100644 >> --- a/hw/b

AioContext lock removal: help needed

2022-07-08 Thread Emanuele Giuseppe Esposito
Hello everyone, As you all know, I am trying to find a way to replace the well known AioContext lock with something else that makes sense and provides the same (or even better) guarantees than using this lock. The reason for this change have been explained over and over and I don't really want

Re: [PATCH v9 05/21] job.c: add job_lock/unlock while keeping job.h intact

2022-07-11 Thread Emanuele Giuseppe Esposito
Am 08/07/2022 um 21:25 schrieb Vladimir Sementsov-Ogievskiy: >>   static bool job_started(Job *job) > > So we can call it both with mutex locked and without. Hope it never race > with job_start. Where exactly do you see it called with mutex not held? I don't see it anywhere, and if you agree

Re: [PATCH v7 05/18] job.h: add _locked duplicates for job API functions called with and without job_mutex

2022-06-22 Thread Emanuele Giuseppe Esposito
Am 21/06/2022 um 17:03 schrieb Vladimir Sementsov-Ogievskiy: > On 6/16/22 16:18, Emanuele Giuseppe Esposito wrote: >> In preparation to the job_lock/unlock usage, create _locked >> duplicates of some functions, since they will be sometimes called with >> job_mutex held

Re: [PATCH v7 10/18] jobs: rename static functions called with job_mutex held

2022-06-22 Thread Emanuele Giuseppe Esposito
Am 21/06/2022 um 19:26 schrieb Vladimir Sementsov-Ogievskiy: > On 6/16/22 16:18, Emanuele Giuseppe Esposito wrote: >> With the*nop*  job_lock/unlock placed, rename the static >> functions that are always under job_mutex, adding "_locked" suffix. >> >> L

Re: [PATCH v7 10/18] jobs: rename static functions called with job_mutex held

2022-06-28 Thread Emanuele Giuseppe Esposito
Am 22/06/2022 um 20:38 schrieb Vladimir Sementsov-Ogievskiy: > On 6/22/22 17:26, Emanuele Giuseppe Esposito wrote: >> >> >> Am 21/06/2022 um 19:26 schrieb Vladimir Sementsov-Ogievskiy: >>> On 6/16/22 16:18, Emanuele Giuseppe Esposito wrote: >>>> Wit

Re: [PATCH v7 10/18] jobs: rename static functions called with job_mutex held

2022-06-28 Thread Emanuele Giuseppe Esposito
Am 28/06/2022 um 12:47 schrieb Vladimir Sementsov-Ogievskiy: > On 6/28/22 10:40, Emanuele Giuseppe Esposito wrote: >> >> >> Am 22/06/2022 um 20:38 schrieb Vladimir Sementsov-Ogievskiy: >>> On 6/22/22 17:26, Emanuele Giuseppe Esposito wrote: >>>> >>

Re: [PATCH v7 01/18] job.c: make job_mutex and job_lock/unlock() public

2022-06-28 Thread Emanuele Giuseppe Esposito
Am 24/06/2022 um 20:22 schrieb Vladimir Sementsov-Ogievskiy: > I've already acked this (honestly, because Stefan do), but still, want > to clarify: > > On 6/16/22 16:18, Emanuele Giuseppe Esposito wrote: >> job mutex will be used to protect the job struct elements and

Re: [PATCH v9 07/21] blockjob: introduce block_job _locked() APIs

2022-07-11 Thread Emanuele Giuseppe Esposito
Am 11/07/2022 um 14:04 schrieb Vladimir Sementsov-Ogievskiy: > On 7/6/22 23:15, Emanuele Giuseppe Esposito wrote: >> Just as done with job.h, create _locked() functions in blockjob.h >> >> These functions will be later useful when caller has already taken >> the

[PATCH v8 15/20] job: detect change of aiocontext within job coroutine

2022-06-29 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini We want to make sure access of job->aio_context is always done under either BQL or job_mutex. The problem is that using aio_co_enter(job->aiocontext, job->co) in job_start and job_enter_cond makes the coroutine immediately resume, so we can't hold the job lock. And caching it

[PATCH v8 01/20] job.c: make job_mutex and job_lock/unlock() public

2022-06-29 Thread Emanuele Giuseppe Esposito
change the nop into an actual mutex and remove the aiocontext lock. Since job_mutex is already being used, add static real_job_{lock/unlock} for the existing usage. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Reviewed-by: Vladimir Sementsov-Ogievskiy --- include

[PATCH v8 03/20] job.c: API functions not used outside should be static

2022-06-29 Thread Emanuele Giuseppe Esposito
job_event_* functions can all be static, as they are not used outside job.c. Same applies for job_txn_add_job(). Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Reviewed-by: Vladimir Sementsov-Ogievskiy --- include/qemu/job.h | 18 -- job.c

[PATCH v8 00/20] job: replace AioContext lock with job_mutex

2022-06-29 Thread Emanuele Giuseppe Esposito
use JOB_LOCK_GUARD and WITH_JOB_LOCK_GUARD * mu(u)ltiple typos in commit messages * job API split patches are sent separately in another series * use of empty job_{lock/unlock} and JOB_LOCK_GUARD/WITH_JOB_LOCK_GUARD to avoid deadlocks and simplify the reviewer job * move patch 11 (block_job_query:

[PATCH v8 04/20] aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED

2022-06-29 Thread Emanuele Giuseppe Esposito
Same as AIO_WAIT_WHILE macro, but if we are in the Main loop do not release and then acquire ctx_ 's aiocontext. Once all Aiocontext locks go away, this macro will replace AIO_WAIT_WHILE. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Reviewed-by: Vladimir Sementsov

[PATCH v8 17/20] job.c: enable job lock/unlock and remove Aiocontext locks

2022-06-29 Thread Emanuele Giuseppe Esposito
ned-off-by: Emanuele Giuseppe Esposito --- blockdev.c | 74 +--- include/qemu/job.h | 22 - job-qmp.c| 44 - job.c| 82 ++-- tests/unit/t

[PATCH v8 10/20] jobs: add job lock in find_* functions

2022-06-29 Thread Emanuele Giuseppe Esposito
*. Signed-off-by: Emanuele Giuseppe Esposito --- blockdev.c | 67 +- job-qmp.c | 55 ++-- 2 files changed, 84 insertions(+), 38 deletions(-) diff --git a/blockdev.c b/blockdev.c index 9230888e34..71f793c4ab

[PATCH v8 16/20] jobs: protect job.aio_context with BQL and job_mutex

2022-06-29 Thread Emanuele Giuseppe Esposito
introduce job_set_aio_context and make sure that the context is set under BQL, job_mutex and drain. Also make sure all other places where the aiocontext is read are protected. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Suggested-by: Paolo Bonzini Signed-off-by: Emanuel

[PATCH v8 11/20] jobs: use job locks also in the unit tests

2022-06-29 Thread Emanuele Giuseppe Esposito
job_pause() is/will be only used in tests to avoid: WITH_JOB_LOCK_GUARD(){ job_pause_locked(); } then it is not worth keeping job_pause(), and just use the guard. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit

[PATCH v8 12/20] block/mirror.c: use of job helpers in drivers to avoid TOC/TOU

2022-06-29 Thread Emanuele Giuseppe Esposito
Once job lock is used and aiocontext is removed, mirror has to perform job operations under the same critical section, using the helpers prepared in previous commit. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito Reviewed

[PATCH v8 14/20] commit and mirror: create new nodes using bdrv_get_aio_context, and not the job aiocontext

2022-06-29 Thread Emanuele Giuseppe Esposito
We are always using the given bs AioContext, so there is no need to take the job ones (which is identical anyways). This also reduces the point we need to check when protecting job.aio_context field. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- block/commit.c | 4

[PATCH v8 08/20] blockjob.h: introduce block_job _locked() APIs

2022-06-29 Thread Emanuele Giuseppe Esposito
-by: Emanuele Giuseppe Esposito --- blockjob.c | 52 include/block/blockjob.h | 15 2 files changed, 57 insertions(+), 10 deletions(-) diff --git a/blockjob.c b/blockjob.c index 7da59a1f1c..0d59aba439 100644 --- a/blockjob.c +++ b

[PATCH v8 09/20] blockjob: rename notifier callbacks as _locked

2022-06-29 Thread Emanuele Giuseppe Esposito
They all are called with job_lock held, in job_event_*_locked() Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 25 +++-- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/blockjob.c b/blockjob.c index 0d59aba439..70952879d8 100644 --- a/blockjob.c

[PATCH v8 13/20] jobs: group together API calls under the same job lock

2022-06-29 Thread Emanuele Giuseppe Esposito
the lock internally. Instead we want JOB_LOCK_GUARD(); for(job = job_next_locked(); ...) Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 20 +++--- blockdev.c | 12

[PATCH v8 06/20] job.h: define functions called without job lock held

2022-06-29 Thread Emanuele Giuseppe Esposito
intended. Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 20 include/qemu/job.h | 37 ++--- job.c | 15 +++ 3 files changed, 49 insertions(+), 23 deletions(-) diff --git a/blockjob.c b/blockjob.c index

[PATCH v8 20/20] job: remove unused functions

2022-06-29 Thread Emanuele Giuseppe Esposito
These public functions are not used anywhere, thus can be dropped. Also, since this is the final job API that doesn't use AioContext lock and replaces it with job_lock, adjust all remaining function documentation to clearly specify if the job lock is taken or not. Signed-off-by: Emanuele Giuseppe

[PATCH v8 07/20] job.h: add _locked public functions

2022-06-29 Thread Emanuele Giuseppe Esposito
These functions will be used later when we use the job lock. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 15 --- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/include

[PATCH v8 19/20] blockjob: remove unused functions

2022-06-29 Thread Emanuele Giuseppe Esposito
These public functions are not used anywhere, thus can be dropped. Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 30 -- include/block/blockjob.h | 33 + 2 files changed, 13 insertions(+), 50 deletions

[PATCH v8 05/20] job.c: add job_lock/unlock while keeping job.h intact

2022-06-29 Thread Emanuele Giuseppe Esposito
_job_{lock/unlock}. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop* .Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 73 +- job.c | 607 +++-- 2 files changed, 499 insertions(+), 181 deletion

[PATCH v8 02/20] job.h: categorize fields in struct Job

2022-06-29 Thread Emanuele Giuseppe Esposito
Categorize the fields in struct Job to understand which ones need to be protected by the job mutex and which don't. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy --- include/qemu/job.h | 61 +++--- 1 file changed, 36

[PATCH v8 18/20] block_job_query: remove atomic read

2022-06-29 Thread Emanuele Giuseppe Esposito
Not sure what the atomic here was supposed to do, since job.busy is protected by the job lock. Since the whole function is called under job_mutex, just remove the atomic. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockjob.c | 2 +- 1 file changed, 1 insertion

Re: [PATCH v8 08/20] blockjob.h: introduce block_job _locked() APIs

2022-07-06 Thread Emanuele Giuseppe Esposito
Am 06/07/2022 um 14:36 schrieb Emanuele Giuseppe Esposito: > > > Am 06/07/2022 um 14:23 schrieb Vladimir Sementsov-Ogievskiy: >> On 7/6/22 15:05, Emanuele Giuseppe Esposito wrote: >>> >>> >>> Am 05/07/2022 um 17:01 schrieb Vladimir Sementsov-Ogievskiy

Re: [PATCH v8 08/20] blockjob.h: introduce block_job _locked() APIs

2022-07-06 Thread Emanuele Giuseppe Esposito
Am 06/07/2022 um 14:23 schrieb Vladimir Sementsov-Ogievskiy: > On 7/6/22 15:05, Emanuele Giuseppe Esposito wrote: >> >> >> Am 05/07/2022 um 17:01 schrieb Vladimir Sementsov-Ogievskiy: >>> On 6/29/22 17:15, Emanuele Giuseppe Esposito wrote: >>>&g

Re: [PATCH v8 08/20] blockjob.h: introduce block_job _locked() APIs

2022-07-06 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 17:01 schrieb Vladimir Sementsov-Ogievskiy: > On 6/29/22 17:15, Emanuele Giuseppe Esposito wrote: >> Just as done with job.h, create _locked() functions in blockjob.h > > We modify not only blockjob.h, I'd s/blockjob.h/blockjob/ in subject. > > Also,

Re: [PATCH v8 06/20] job.h: define functions called without job lock held

2022-07-06 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 12:53 schrieb Vladimir Sementsov-Ogievskiy: > On 6/29/22 17:15, Emanuele Giuseppe Esposito wrote: >> These functions don't need a _locked() counterpart, since >> they are all called outside job.c and take the lock only >> internally. >> >> Updat

Re: [PATCH v8 06/20] job.h: define functions called without job lock held

2022-07-06 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 12:54 schrieb Vladimir Sementsov-Ogievskiy: > To subject: hmm, the commit don't define any function.. > mark functions called without job lock held?

[PATCH v9 00/21] job: replace AioContext lock with job_mutex

2022-07-06 Thread Emanuele Giuseppe Esposito
job API split patches are sent separately in another series * use of empty job_{lock/unlock} and JOB_LOCK_GUARD/WITH_JOB_LOCK_GUARD to avoid deadlocks and simplify the reviewer job * move patch 11 (block_job_query: remove atomic read) as last Emanuele Giuseppe Esposito (20): job.c: make job_mutex and

[PATCH v9 06/21] job: move and update comments from blockjob.c

2022-07-06 Thread Emanuele Giuseppe Esposito
This comment applies more on job, it was left in blockjob as in the past the whole job logic was implemented there. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 20

[PATCH v9 11/21] jobs: group together API calls under the same job lock

2022-07-06 Thread Emanuele Giuseppe Esposito
the lock internally. Instead we want JOB_LOCK_GUARD(); for(job = job_next_locked(); ...) Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 20 +++--- blockdev.c | 12

[PATCH v9 01/21] job.c: make job_mutex and job_lock/unlock() public

2022-07-06 Thread Emanuele Giuseppe Esposito
change the nop into an actual mutex and remove the aiocontext lock. Since job_mutex is already being used, add static real_job_{lock/unlock} for the existing usage. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Reviewed-by: Vladimir Sementsov-Ogievskiy --- include

Re: [PATCH v8 17/20] job.c: enable job lock/unlock and remove Aiocontext locks

2022-07-06 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 15:07 schrieb Stefan Hajnoczi: > On Wed, Jun 29, 2022 at 10:15:35AM -0400, Emanuele Giuseppe Esposito wrote: >> Change the job_{lock/unlock} and macros to use job_mutex. >> >> Now that they are not nop anymore, remove the aiocontext >> to avoid

[PATCH v9 03/21] job.c: API functions not used outside should be static

2022-07-06 Thread Emanuele Giuseppe Esposito
job_event_* functions can all be static, as they are not used outside job.c. Same applies for job_txn_add_job(). Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Reviewed-by: Vladimir Sementsov-Ogievskiy --- include/qemu/job.h | 18 -- job.c

[PATCH v9 05/21] job.c: add job_lock/unlock while keeping job.h intact

2022-07-06 Thread Emanuele Giuseppe Esposito
lock guard macros are *nop* Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 138 ++- job.c | 605 - 2 files changed, 558 insertions(+), 185 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index

[PATCH v9 07/21] blockjob: introduce block_job _locked() APIs

2022-07-06 Thread Emanuele Giuseppe Esposito
-by: Emanuele Giuseppe Esposito --- blockjob.c | 52 include/block/blockjob.h | 18 ++ 2 files changed, 60 insertions(+), 10 deletions(-) diff --git a/blockjob.c b/blockjob.c index 7da59a1f1c..0d59aba439 100644 --- a/blockjob.c +++ b

[PATCH v9 02/21] job.h: categorize fields in struct Job

2022-07-06 Thread Emanuele Giuseppe Esposito
Categorize the fields in struct Job to understand which ones need to be protected by the job mutex and which don't. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy --- include/qemu/job.h | 61 +++--- 1 file changed, 36

[PATCH v9 17/21] blockjob.h: categorize fields in struct BlockJob

2022-07-06 Thread Emanuele Giuseppe Esposito
The same job lock is being used also to protect some of blockjob fields. Categorize them just as done in job.h. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/blockjob.h | 17 ++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/include/block/blockjob.h

[PATCH v9 08/21] jobs: add job lock in find_* functions

2022-07-06 Thread Emanuele Giuseppe Esposito
*. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockdev.c | 67 +- job-qmp.c | 57 -- 2 files changed, 86 insertions(+), 38 deletions(-) diff --git a/blockdev.c b/blockdev.c

[PATCH v9 18/21] blockjob: rename notifier callbacks as _locked

2022-07-06 Thread Emanuele Giuseppe Esposito
They all are called with job_lock held, in job_event_*_locked() Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockjob.c | 25 +++-- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/blockjob.c b/blockjob.c index bbd297b583

[PATCH v9 20/21] blockjob: remove unused functions

2022-07-06 Thread Emanuele Giuseppe Esposito
These public functions are not used anywhere, thus can be dropped. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockjob.c | 30 -- include/block/blockjob.h | 36 +--- 2 files changed, 13

[PATCH v9 15/21] job.c: enable job lock/unlock and remove Aiocontext locks

2022-07-06 Thread Emanuele Giuseppe Esposito
der aiocontext lock. Also remove real_job_{lock/unlock}, as they are replaced by the public functions. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockdev.c | 74 --- include/qemu/job.h | 22 job-qm

[PATCH v9 21/21] job: remove unused functions

2022-07-06 Thread Emanuele Giuseppe Esposito
These public functions are not used anywhere, thus can be dropped. Also, since this is the final job API that doesn't use AioContext lock and replaces it with job_lock, adjust all remaining function documentation to clearly specify if the job lock is taken or not. Signed-off-by: Emanuele Giuseppe

[PATCH v9 13/21] job: detect change of aiocontext within job coroutine

2022-07-06 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini We want to make sure access of job->aio_context is always done under either BQL or job_mutex. The problem is that using aio_co_enter(job->aiocontext, job->co) in job_start and job_enter_cond makes the coroutine immediately resume, so we can't hold the job lock. And caching it

[PATCH v9 09/21] jobs: use job locks also in the unit tests

2022-07-06 Thread Emanuele Giuseppe Esposito
job_pause() is/will be only used in tests to avoid: WITH_JOB_LOCK_GUARD(){ job_pause_locked(); } then it is not worth keeping job_pause(), and just use the guard. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito Reviewed

[PATCH v9 16/21] block_job_query: remove atomic read

2022-07-06 Thread Emanuele Giuseppe Esposito
Not sure what the atomic here was supposed to do, since job.busy is protected by the job lock. Since the whole function is called under job_mutex, just remove the atomic. Reviewed-by: Stefan Hajnoczi Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 2 +- 1 file changed, 1 insertion

[PATCH v9 04/21] aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED

2022-07-06 Thread Emanuele Giuseppe Esposito
Same as AIO_WAIT_WHILE macro, but if we are in the Main loop do not release and then acquire ctx_ 's aiocontext. Once all Aiocontext locks go away, this macro will replace AIO_WAIT_WHILE. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Reviewed-by: Vladimir Sementsov

[PATCH v9 12/21] commit and mirror: create new nodes using bdrv_get_aio_context, and not the job aiocontext

2022-07-06 Thread Emanuele Giuseppe Esposito
We are always using the given bs AioContext, so there is no need to take the job ones (which is identical anyways). This also reduces the point we need to check when protecting job.aio_context field. Reviewed-by: Stefan Hajnoczi Signed-off-by: Emanuele Giuseppe Esposito --- block/commit.c | 4

[PATCH v9 10/21] block/mirror.c: use of job helpers in drivers to avoid TOC/TOU

2022-07-06 Thread Emanuele Giuseppe Esposito
Once job lock is used and aiocontext is removed, mirror has to perform job operations under the same critical section, using the helpers prepared in previous commit. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito Reviewed

[PATCH v9 14/21] jobs: protect job.aio_context with BQL and job_mutex

2022-07-06 Thread Emanuele Giuseppe Esposito
introduce job_set_aio_context and make sure that the context is set under BQL, job_mutex and drain. Also make sure all other places where the aiocontext is read are protected. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Suggested-by: Paolo Bonzini Signed-off-by: Emanuel

[PATCH v9 19/21] blockjob: protect iostatus field in BlockJob struct

2022-07-06 Thread Emanuele Giuseppe Esposito
called under job lock. Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/blockjob.c b/blockjob.c index a2559b97a7..893c8ff08e 100644 --- a/blockjob.c +++ b/blockjob.c @@ -367,7 +367,8 @@ BlockJobInfo *block_job_query

Re: [PATCH v8 05/20] job.c: add job_lock/unlock while keeping job.h intact

2022-07-05 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 09:39 schrieb Stefan Hajnoczi: > On Wed, Jun 29, 2022 at 10:15:23AM -0400, Emanuele Giuseppe Esposito wrote: >> +void job_ref(Job *job) >> +{ >> +JOB_LOCK_GUARD(); >> +job_ref_locked(job); >> +} > > You don't need to fix thi

Re: [PATCH v8 08/20] blockjob.h: introduce block_job _locked() APIs

2022-07-05 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 09:58 schrieb Stefan Hajnoczi: > On Wed, Jun 29, 2022 at 10:15:26AM -0400, Emanuele Giuseppe Esposito wrote: >> +BlockJob *block_job_next(BlockJob *bjob) >> { >> -Job *job = job_get(id); >> +JOB_LOCK_GUARD(); >> +

Re: [PATCH v8 13/20] jobs: group together API calls under the same job lock

2022-07-05 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 10:14 schrieb Stefan Hajnoczi: > On Wed, Jun 29, 2022 at 10:15:31AM -0400, Emanuele Giuseppe Esposito wrote: >> diff --git a/blockdev.c b/blockdev.c >> index 71f793c4ab..5b79093155 100644 >> --- a/blockdev.c >> +++ b/blockdev.c &g

Re: [PATCH v6 02/18] job.h: categorize fields in struct Job

2022-06-08 Thread Emanuele Giuseppe Esposito
Am 07/06/2022 um 17:41 schrieb Paolo Bonzini: > On 6/7/22 15:20, Emanuele Giuseppe Esposito wrote: >> >> >> Am 03/06/2022 um 18:00 schrieb Kevin Wolf: >>> Am 14.03.2022 um 14:36 hat Emanuele Giuseppe Esposito geschrieben: >>>> Categorize the fie

Re: [PATCH v6 06/18] jobs: protect jobs with job_lock/unlock

2022-06-07 Thread Emanuele Giuseppe Esposito
Am 03/06/2022 um 18:40 schrieb Kevin Wolf: > Am 14.03.2022 um 14:36 hat Emanuele Giuseppe Esposito geschrieben: >> Introduce the job locking mechanism through the whole job API, >> following the comments in job.h and requirements of job-monitor >> (like the functions in j

Re: [PATCH v6 02/18] job.h: categorize fields in struct Job

2022-06-07 Thread Emanuele Giuseppe Esposito
Am 03/06/2022 um 18:00 schrieb Kevin Wolf: > Am 14.03.2022 um 14:36 hat Emanuele Giuseppe Esposito geschrieben: >> Categorize the fields in struct Job to understand which ones >> need to be protected by the job mutex and which don't. >> >> Signed-off-by: Emanuel

Re: [PATCH v6 15/18] job: detect change of aiocontext within job coroutine

2022-06-07 Thread Emanuele Giuseppe Esposito
Am 03/06/2022 um 18:59 schrieb Kevin Wolf: > Am 14.03.2022 um 14:37 hat Emanuele Giuseppe Esposito geschrieben: >> From: Paolo Bonzini >> >> We want to make sure access of job->aio_context is always done >> under either BQL or job_mutex. The problem is

Re: [PATCH v6 05/18] job.h: add _locked duplicates for job API functions called with and without job_mutex

2022-06-07 Thread Emanuele Giuseppe Esposito
Am 03/06/2022 um 18:17 schrieb Kevin Wolf: > Am 14.03.2022 um 14:36 hat Emanuele Giuseppe Esposito geschrieben: >> In preparation to the job_lock/unlock usage, create _locked >> duplicates of some functions, since they will be sometimes called with >> job_mutex held

<    5   6   7   8   9   10   11   12   13   14   >