Re: [PATCH maybe-7.2 1/3] hw/i2c: only schedule pending master when bus is idle

2022-11-16 Thread Klaus Jensen
On Nov 17 07:56, Cédric Le Goater wrote: > On 11/17/22 07:40, Klaus Jensen wrote: > > On Nov 16 16:58, Cédric Le Goater wrote: > > > On 11/16/22 09:43, Klaus Jensen wrote: > > > > From: Klaus Jensen > > > > > > > > It is not given that the current master will release the bus after a > > > >

Re: [PATCH v3 2/2] nvme: Add physical writes/reads from OCP log

2022-11-16 Thread Klaus Jensen
On Nov 16 18:14, Joel Granados wrote: > In order to evaluate write amplification factor (WAF) within the storage > stack it is important to know the number of bytes written to the > controller. The existing SMART log value of Data Units Written is too > coarse (given in units of 500 Kb) and so we

Re: [PATCH maybe-7.2 1/3] hw/i2c: only schedule pending master when bus is idle

2022-11-16 Thread Cédric Le Goater
On 11/17/22 07:40, Klaus Jensen wrote: On Nov 16 16:58, Cédric Le Goater wrote: On 11/16/22 09:43, Klaus Jensen wrote: From: Klaus Jensen It is not given that the current master will release the bus after a transfer ends. Only schedule a pending master if the bus is idle. Fixes:

Re: [PATCH RFC 2/3] hw/i2c: add mctp core

2022-11-16 Thread Klaus Jensen
On Nov 16 08:27, Corey Minyard wrote: > On Wed, Nov 16, 2022 at 09:43:11AM +0100, Klaus Jensen wrote: > > From: Klaus Jensen > > > > Add an abstract MCTP over I2C endpoint model. This implements MCTP > > control message handling as well as handling the actual I2C transport > > (packetization). >

Re: [PATCH maybe-7.2 1/3] hw/i2c: only schedule pending master when bus is idle

2022-11-16 Thread Klaus Jensen
On Nov 16 16:58, Cédric Le Goater wrote: > On 11/16/22 09:43, Klaus Jensen wrote: > > From: Klaus Jensen > > > > It is not given that the current master will release the bus after a > > transfer ends. Only schedule a pending master if the bus is idle. > > > > Fixes: 37fa5ca42623 ("hw/i2c:

Re: [PATCH v2 3/3] nvme: Add physical writes/reads from OCP log

2022-11-16 Thread Klaus Jensen
On Nov 16 17:19, Joel Granados wrote: > On Tue, Nov 15, 2022 at 12:26:17PM +0100, Klaus Jensen wrote: > > On Nov 14 14:50, Joel Granados wrote: > > > > > > +static uint16_t nvme_vendor_specific_log(uint8_t lid, NvmeCtrl *n, > > > uint8_t rae, > > > +

Re: [PATCH v3 14/17] vfio/migration: Reset device if setting recover state fails

2022-11-16 Thread Alex Williamson
On Thu, 3 Nov 2022 18:16:17 +0200 Avihai Horon wrote: > If vfio_migration_set_state() fails to set the device in the requested > state it tries to put it in a recover state. If setting the device in > the recover state fails as well, hw_error is triggered and the VM is > aborted. > > To improve

Re: [PATCH v3 12/17] vfio/migration: Implement VFIO migration protocol v2

2022-11-16 Thread Alex Williamson
On Thu, 3 Nov 2022 18:16:15 +0200 Avihai Horon wrote: > Add implementation of VFIO migration protocol v2. The two protocols, v1 > and v2, will co-exist and in next patch v1 protocol will be removed. > > There are several main differences between v1 and v2 protocols: > - VFIO device state is now

[PATCH v3 1/2] nvme: Move adjustment of data_units{read,written}

2022-11-16 Thread Joel Granados
In order to return the units_{read/written} required by the SMART log we need to shift the number of bytes value by BDRV_SECTORS_BITS and multiply by 1000. This is a prep patch that moves this adjustment to where the SMART log is calculated in order to use the stats struct for calculating OCP

[PATCH v3 0/2] Add OCP extended log to nvme QEMU

2022-11-16 Thread Joel Granados
The motivation and description are contained in the last patch in this set. Will copy paste it here for convenience: In order to evaluate write amplification factor (WAF) within the storage stack it is important to know the number of bytes written to the controller. The existing SMART

[PATCH v3 2/2] nvme: Add physical writes/reads from OCP log

2022-11-16 Thread Joel Granados
In order to evaluate write amplification factor (WAF) within the storage stack it is important to know the number of bytes written to the controller. The existing SMART log value of Data Units Written is too coarse (given in units of 500 Kb) and so we add the SMART health information extended from

Re: [PATCH v3] block/rbd: Add support for layered encryption

2022-11-16 Thread Ilya Dryomov
On Wed, Nov 16, 2022 at 12:15 PM Daniel P. Berrangé wrote: > > On Wed, Nov 16, 2022 at 10:23:52AM +, Daniel P. Berrangé wrote: > > On Wed, Nov 16, 2022 at 09:03:31AM +, Or Ozeri wrote: > > > > -Original Message- > > > > From: Daniel P. Berrangé > > > > Sent: 15 November 2022

Re: [PATCH v2 2/9] block-copy: add missing coroutine_fn annotations

2022-11-16 Thread Paolo Bonzini
On 11/15/22 16:41, Emanuele Giuseppe Esposito wrote: To sum up on what was discussed in this serie, I don't really see any strong objection against these patches, so I will soon send v3 which is pretty much the same except for patch 1, which will be removed. I think these patches are useful and

Re: [PATCH v2 3/3] nvme: Add physical writes/reads from OCP log

2022-11-16 Thread Joel Granados
On Tue, Nov 15, 2022 at 12:26:17PM +0100, Klaus Jensen wrote: > On Nov 14 14:50, Joel Granados wrote: > > In order to evaluate write amplification factor (WAF) within the storage > > stack it is important to know the number of bytes written to the > > controller. The existing SMART log value of

Re: [PATCH maybe-7.2 1/3] hw/i2c: only schedule pending master when bus is idle

2022-11-16 Thread Cédric Le Goater
On 11/16/22 09:43, Klaus Jensen wrote: From: Klaus Jensen It is not given that the current master will release the bus after a transfer ends. Only schedule a pending master if the bus is idle. Fixes: 37fa5ca42623 ("hw/i2c: support multiple masters") Signed-off-by: Klaus Jensen ---

Fwd: [FOSDEM] CfP Software Defined Storage devroom FOSDEM23

2022-11-16 Thread Niels de Vos
Hi! In a few montsh time FOSDEM will host an in-person Software Defined Storage devroom again. It would be a great oppertunity to show what storage related things the QEMU project has been doing, and what is planned for the future. Please consider proposing a talk! Thanks, Niels -

RE: [PULL 00/30] Next patches

2022-11-16 Thread Xu, Ling1
Hi, All, Very appreciated for your time on reviewing our patch. The second CI failure caused by our patch has been addressed. One simple way is moving "#endif" in qemu/tests/bench/xbzrle-bench.c from line 46 to line 450. We have submitted patch v7 to update this modification. Thanks

Re: [PATCH v2 2/3] nvme: Add ocp to the subsys

2022-11-16 Thread Joel Granados
On Tue, Nov 15, 2022 at 12:11:50PM +0100, Klaus Jensen wrote: > On Nov 14 14:50, Joel Granados wrote: > > The Open Compute Project defines a Datacenter NVMe SSD Spec that sits on > > top of the NVMe spec. Additional commands and NVMe behaviors specific for > > the Datacenter. This is a preparation

Re: [PATCH RFC 2/3] hw/i2c: add mctp core

2022-11-16 Thread Corey Minyard
On Wed, Nov 16, 2022 at 09:43:11AM +0100, Klaus Jensen wrote: > From: Klaus Jensen > > Add an abstract MCTP over I2C endpoint model. This implements MCTP > control message handling as well as handling the actual I2C transport > (packetization). > > Devices are intended to derive from this and

[PATCH 05/15] block: use bdrv_co_refresh_total_sectors when possible

2022-11-16 Thread Emanuele Giuseppe Esposito
In some places we are sure we are always running in a coroutine, therefore it's useless to call the generated_co_wrapper, instead call directly the _co_ function. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 6 +++--- block/copy-on-read.c | 2 +- block/io.c

[PATCH 03/15] block-backend: use bdrv_getlength instead of blk_getlength

2022-11-16 Thread Emanuele Giuseppe Esposito
The only difference is that blk_ checks if the block is available, but this check is already performed above in blk_check_byte_request(). This is in preparation for the graph rdlock, which will be taken by both the callers of blk_check_byte_request() and blk_getlength(). Signed-off-by: Emanuele

[PATCH 09/15] block-coroutine-wrapper: support void functions

2022-11-16 Thread Emanuele Giuseppe Esposito
Just omit the various 'return' when the return type is void. Signed-off-by: Emanuele Giuseppe Esposito --- scripts/block-coroutine-wrapper.py | 19 ++- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/scripts/block-coroutine-wrapper.py

[PATCH 08/15] block: convert bdrv_is_inserted in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver->bdrv_is_inserted is categorized as IO callback, and it currently doesn't run in a coroutine. This makes very difficult to add the graph rdlock, since the callback traverses the block nodes graph. Therefore use generated_co_wrapper_simple to automatically creates a wrapper with the

[PATCH 06/15] block: convert bdrv_get_allocated_file_size in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver->bdrv_get_allocated_file_size is categorized as IO callback, and it currently doesn't run in a coroutine. This makes very difficult to add the graph rdlock, since the callback traverses the block nodes graph. Therefore use generated_co_wrapper_simple to automatically create a wrapper

[PATCH 14/15] block: convert bdrv_io_unplug in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver->bdrv_io_unplug is categorized as IO callback, and it currently doesn't run in a coroutine. This makes very difficult to add the graph rdlock, since the callback traverses the block nodes graph. The only caller of this function is blk_unplug, therefore make blk_unplug a

[PATCH 11/15] block: convert bdrv_lock_medium in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver->bdrv_lock_medium is categorized as IO callback, and it currently doesn't run in a coroutine. This makes very difficult to add the graph rdlock, since the callback traverses the block nodes graph. The only caller of this function is blk_lock_medium, therefore make blk_lock_medium a

[PATCH 13/15] block: convert bdrv_io_plug in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver->bdrv_io_plug is categorized as IO callback, and it currently doesn't run in a coroutine. This makes very difficult to add the graph rdlock, since the callback traverses the block nodes graph. The only caller of this function is blk_plug, therefore make blk_plug a

[PATCH 07/15] block: convert bdrv_get_info in generated_co_wrapper

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver->bdrv_get_info is categorized as IO callback, and it currently doesn't run in a coroutine. This makes very difficult to add the graph rdlock, since the callback traverses the block nodes graph. Therefore use generated_co_wrapper to automatically create a wrapper with the same name.

[PATCH 04/15] block: convert bdrv_refresh_total_sectors in generated_co_wrapper

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver->bdrv_getlength is categorized as IO callb ack, and it currently doesn't run in a coroutine. This makes very difficult to add the graph rdlock, sin ce the callback traverses the block nodes graph. Therefore use generated_co_wrapper to automatically create a wrapper for

[PATCH 12/15] block: convert bdrv_debug_event in generated_co_wrapper

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver->bdrv_debug_event is categorized as IO callback, and it currently doesn't run in a coroutine. This makes very difficult to add the graph rdlock, since the callback traverses the block nodes graph. Therefore use generated_co_wrapper to automatically create a wrapper with the same name.

[PATCH 02/15] block: rename refresh_total_sectors in bdrv_refresh_total_sectors

2022-11-16 Thread Emanuele Giuseppe Esposito
Name is not right, since we are going to convert this in a generated_co_wrapper. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 8 block/io.c | 8 +--- include/block/block_int-io.h | 2 +- 3 files

[PATCH 10/15] block: convert bdrv_eject in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver->bdrv_eject is categorized as IO callback, and it currently doesn't run in a coroutine. This makes very difficult to add the graph rdlock, since the callback traverses the block nodes graph. The only caller of this function is blk_eject, therefore make blk_eject a

[PATCH 15/15] block: rename newly converted BlockDriver IO coroutine functions

2022-11-16 Thread Emanuele Giuseppe Esposito
Since these functions alwayas run in coroutine context, adjust their name to include "_co_", just like all other BlockDriver callbacks. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 32 ++--- block/blkdebug.c

[PATCH 01/15] block/qed: add missing graph rdlock in qed_need_check_timer_entry

2022-11-16 Thread Emanuele Giuseppe Esposito
This function is called in two different places: - timer callback, which does not take the graph rdlock. - bdrv_qed_drain_begin(), which is a .bdrv_drain_begin() callback that will soon take the lock. Since it calls recursive functions that traverse the graph, we need to protect them with the

[PATCH 00/15] Protect the block layer with a rwlock: part 3

2022-11-16 Thread Emanuele Giuseppe Esposito
Please read "Protect the block layer with a rwlock: part 1" and "Protect the block layer with a rwlock: part 2" for an additional introduction and aim of this series. In this serie, we cover the remaining BlockDriver IO callbacks that were not running in coroutine, therefore not using the graph

Re: [PATCH 0/3] hw/{i2c,nvme}: mctp endpoint, nvme management interface model

2022-11-16 Thread Jeremy Kerr
Hi Klaus, [+CC Matt] > This adds a generic MCTP endpoint model that other devices may derive > from. I'm not 100% happy with the design of the class methods, but > it's a start. Thanks for posting these! I'll have a more thorough look through soon, but wanted to tackle some of the larger

[PATCH 3/6] block: assert that BlockDriver->bdrv_co_copy_range_{from/to} is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only non-protected caller is convert_co_copy_range(), all other callers are BlockDriver callbacks that already take the rdlock. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c| 2 ++ block/io.c | 5 + include/block/block_int-common.h

[PATCH 4/6] block/dirty-bitmap: assert that BlockDriver->bdrv_co_*_dirty_bitmap are always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only callers are the respective bdrv_*_dirty_bitmap() functions that take care of creating a new coroutine (that already takes the graph rdlock). Signed-off-by: Emanuele Giuseppe Esposito --- block/dirty-bitmap.c | 2 ++ include/block/block_int-common.h | 2 ++ 2 files changed,

[PATCH 5/6] block/io: assert that BlockDriver->bdrv_co_*_snapshot_* are always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only callers are other callback functions that already run with the graph rdlock taken. Signed-off-by: Emanuele Giuseppe Esposito --- block/io.c | 2 ++ include/block/block_int-common.h | 3 +++ 2 files changed, 5 insertions(+) diff --git a/block/io.c b/block/io.c

[PATCH 11/20] block-gen: assert that bdrv_co_{check/invalidate_cache} are always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only callers of these functions are the respective generated_co_wrapper, and they already take the lock. Protecting bdrv_co_{check/invalidate_cache}() implies that BlockDriver->bdrv_co_{check/invalidate_cache}() is always called with graph rdlock taken. Signed-off-by: Emanuele Giuseppe

[PATCH 12/20] block-gen: assert that bdrv_co_pwrite is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
This function, in addition to be called by a generated_co_wrapper, is also called elsewhere else. The strategy is to always take the lock at the function called when the coroutine is created, to avoid recursive locking. By protecting brdv_co_pwrite, we also automatically protect the following

[PATCH 6/6] block: assert that BlockDriver->bdrv_co_delete_file is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only callers are other callback functions that already run with the graph rdlock taken. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 1 + include/block/block_int-common.h | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/block.c

[PATCH 19/20] block-gen: assert that bdrv_co_ioctl is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only caller of this function is blk_ioctl, a generated_co_wrapper functions that needs to take the graph read lock. Protecting bdrv_co_ioctl() implies that BlockDriver->bdrv_co_ioctl() is always called with graph rdlock taken, and BlockDriver->bdrv_aio_ioctl is a coroutine_fn callback (called

[PATCH 0/6] Protect the block layer with a rwlock: part 2

2022-11-16 Thread Emanuele Giuseppe Esposito
Please read "Protect the block layer with a rwlock: part 1" for an additional introduction and aim of this series. This second part aims to add the graph rdlock to the BlockDriver functions that already run in coroutine context and are classified as IO. Such functions will recursively traverse

[PATCH 04/20] block.c: wrlock in bdrv_replace_child_noperm

2022-11-16 Thread Emanuele Giuseppe Esposito
Protect the main function where graph is modified. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 6 -- include/block/block_int-common.h | 1 + 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/block.c b/block.c index d3e168408a..4ef537a9f2

[PATCH 07/20] graph-lock: implement WITH_GRAPH_RDLOCK_GUARD and GRAPH_RDLOCK_GUARD macros

2022-11-16 Thread Emanuele Giuseppe Esposito
Similar to the implementation in lockable.h, implement macros to automatically take and release the rdlock. Create the empty GraphLockable struct only to use it as a type for G_DEFINE_AUTOPTR_CLEANUP_FUNC. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/graph-lock.h | 35

[PATCH 2/6] block: assert that BlockDriver->bdrv_co_{amend/create} are called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
Both functions are only called by Job->run() callbacks, therefore they must take the lock in the *_run() implementation. Signed-off-by: Emanuele Giuseppe Esposito --- block/amend.c| 1 + block/create.c | 1 + include/block/block_int-common.h | 2 ++ 3 files

[PATCH 01/20] block: introduce a lock to protect graph operations

2022-11-16 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini block layer graph operations are always run under BQL in the main loop. This is proved by the assertion qemu_in_main_thread() and its wrapper macro GLOBAL_STATE_CODE. However, there are also concurrent coroutines running in other iothreads that always try to traverse the

[PATCH 18/20] block-gen: assert that bdrv_co_common_block_status_above is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
This function, in addition to be called by a generated_co_wrapper, is also called elsewhere else. The strategy is to always take the lock at the function called when the coroutine is created, to avoid recursive locking. Protecting bdrv_co_block_status() called by

[PATCH 10/20] block-gen: assert that {bdrv/blk}_co_truncate is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
This function, in addition to be called by a generated_co_wrapper, is also called by the blk_* API. The strategy is to always take the lock at the function called when the coroutine is created, to avoid recursive locking. Protecting bdrv_co_truncate() implies that BlockDriver->bdrv_co_truncate()

[PATCH 08/20] block-coroutine-wrapper.py: take the graph rdlock in bdrv_* functions

2022-11-16 Thread Emanuele Giuseppe Esposito
All generated_co_wrapper functions create a coroutine when called from non-coroutine context. The format can be one of the two: bdrv_something() if(qemu_in_coroutine()): bdrv_co_something(); else: // create coroutine that calls bdrv_co_something(); blk_something()

[PATCH 1/6] block: assert that bdrv_co_create is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
This function is either called by bdrv_create(), which always takes care of creating a new coroutine, or by bdrv_create_file(), which is only called by BlockDriver->bdrv_co_create_opts callbacks, invoked by bdrv_co_create(). Protecting bdrv_co_create() implies that

[PATCH 02/20] graph-lock: introduce BdrvGraphRWlock structure

2022-11-16 Thread Emanuele Giuseppe Esposito
Just a wrapper to simplify what is available to the struct AioContext. Signed-off-by: Emanuele Giuseppe Esposito --- block/graph-lock.c | 59 ++ include/block/aio.h| 12 include/block/graph-lock.h | 1 + 3 files changed, 48

[PATCH 14/20] block-gen: assert that bdrv_co_pread is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
This function, in addition to be called by a generated_co_wrapper, is also called elsewhere else. The strategy is to always take the lock at the function called when the coroutine is created, to avoid recursive locking. By protecting brdv_co_pread, we also automatically protect the following

[PATCH 06/20] block: assert that graph read and writes are performed correctly

2022-11-16 Thread Emanuele Giuseppe Esposito
Remove the old assert_bdrv_graph_writable, and replace it with the new version using graph-lock API. See the function documentation for more information. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 4 ++-- block/graph-lock.c | 11

[PATCH 05/20] block: remove unnecessary assert_bdrv_graph_writable()

2022-11-16 Thread Emanuele Giuseppe Esposito
We don't protect bdrv->aio_context with the graph rwlock, so these assertions are not needed Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block.c b/block.c index 4ef537a9f2..afab74d4da 100644 --- a/block.c +++ b/block.c @@ -7183,7

[PATCH 16/20] block-gen: assert that bdrv_co_{read/write}v_vmstate are always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only caller of these functions is bdrv_{read/write}v_vmstate, a generated_co_wrapper function that already takes the graph read lock. Protecting bdrv_co_{read/write}v_vmstate() implies that BlockDriver->bdrv_{load/save}_vmstate() is always called with graph rdlock taken. Signed-off-by:

[PATCH 09/20] block-backend: introduce new generated_co_wrapper_blk annotation

2022-11-16 Thread Emanuele Giuseppe Esposito
This annotation will be used to distinguish the blk_* API from the bdrv_* API in block-gen.c. The reason for this distinction is that blk_* API eventually result in always calling bdrv_*, which has implications when we introduce the read graph lock. Signed-off-by: Emanuele Giuseppe Esposito ---

[PATCH 03/20] async: register/unregister aiocontext in graph lock list

2022-11-16 Thread Emanuele Giuseppe Esposito
Add/remove the AioContext in aio_context_list in graph-lock.c only when it is being effectively created/destroyed. Signed-off-by: Emanuele Giuseppe Esposito --- util/async.c | 4 util/meson.build | 1 + 2 files changed, 5 insertions(+) diff --git a/util/async.c b/util/async.c index

[PATCH 17/20] block-gen: assert that bdrv_co_pdiscard is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
This function, in addition to be called by a generated_co_wrapper, is also called by the blk_* API. The strategy is to always take the lock at the function called when the coroutine is created, to avoid recursive locking. Protecting bdrv_co_pdiscard{_snapshot}() implies that the following

[PATCH 20/20] block-gen: assert that nbd_co_do_establish_connection is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only caller of this function is nbd_do_establish_connection, a generated_co_wrapper that already take the graph read lock. Signed-off-by: Emanuele Giuseppe Esposito --- block/nbd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/nbd.c b/block/nbd.c index 7d485c86d2..5cad58aaf6

[PATCH 15/20] block-gen: assert that {bdrv/blk}_co_flush is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
This function, in addition to be called by a generated_co_wrapper, is also called by the blk_* API. The strategy is to always take the lock at the function called when the coroutine is created, to avoid recursive locking. Protecting bdrv_co_flush() implies that the following BlockDriver callbacks

[PATCH 00/20] Protect the block layer with a rwlock: part 1

2022-11-16 Thread Emanuele Giuseppe Esposito
This serie is the first of four series that aim to introduce and use a new graph rwlock in the QEMU block layer. The aim is to replace the current AioContext lock with much fine-grained locks, aimed to protect only specific data. Currently the AioContext lock is used pretty much everywhere, and

[PATCH 13/20] block-gen: assert that bdrv_co_pwrite_{zeros/sync} is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
Already protected by bdrv_co_pwrite callers. Protecting bdrv_co_do_pwrite_zeroes() implies that BlockDriver->bdrv_co_pwrite_zeroes() is always called with graph rdlock taken. Signed-off-by: Emanuele Giuseppe Esposito --- block/io.c | 3 +++

Re: [PATCH maybe-7.2 1/3] hw/i2c: only schedule pending master when bus is idle

2022-11-16 Thread Corey Minyard
On Wed, Nov 16, 2022 at 09:43:10AM +0100, Klaus Jensen wrote: > From: Klaus Jensen > > It is not given that the current master will release the bus after a > transfer ends. Only schedule a pending master if the bus is idle. > Yes, I think this is correct. Acked-by: Corey Minyard Is there a

[PATCH 00/20] Protect the block layer with a rwlock: part 1

2022-11-16 Thread Emanuele Giuseppe Esposito
This serie is the first of four series that aim to introduce and use a new graph rwlock in the QEMU block layer. The aim is to replace the current AioContext lock with much fine-grained locks, aimed to protect only specific data. Currently the AioContext lock is used pretty much everywhere, and

[PATCH 00/20] Protect the block layer with a rwlock: part 1

2022-11-16 Thread Emanuele Giuseppe Esposito
This serie is the first of four series that aim to introduce and use a new graph rwlock in the QEMU block layer. The aim is to replace the current AioContext lock with much fine-grained locks, aimed to protect only specific data. Currently the AioContext lock is used pretty much everywhere, and

Re: [PATCH v3 10/17] vfio/migration: Move migration v1 logic to vfio_migration_init()

2022-11-16 Thread Avihai Horon
On 16/11/2022 1:56, Alex Williamson wrote: External email: Use caution opening links or attachments On Thu, 3 Nov 2022 18:16:13 +0200 Avihai Horon wrote: Move vfio_dev_get_region_info() logic from vfio_migration_probe() to vfio_migration_init(). This logic is specific to v1 protocol and

Re: [PATCH v3 07/17] vfio/migration: Allow migration without VFIO IOMMU dirty tracking support

2022-11-16 Thread Avihai Horon
On 16/11/2022 1:36, Alex Williamson wrote: External email: Use caution opening links or attachments On Thu, 3 Nov 2022 18:16:10 +0200 Avihai Horon wrote: Currently, if IOMMU of a VFIO container doesn't support dirty page tracking, migration is blocked. This is because a DMA-able VFIO

Re: [PATCH maybe-7.2 1/3] hw/i2c: only schedule pending master when bus is idle

2022-11-16 Thread Peter Maydell
On Wed, 16 Nov 2022 at 08:43, Klaus Jensen wrote: > > From: Klaus Jensen > > It is not given that the current master will release the bus after a > transfer ends. Only schedule a pending master if the bus is idle. > > Fixes: 37fa5ca42623 ("hw/i2c: support multiple masters") > Signed-off-by:

[PATCH v4 04/11] block-coroutine-wrapper.py: introduce generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
This new annotation creates just a function wrapper that creates a new coroutine. It assumes the caller is not a coroutine. This is much better as g_c_w, because it is clear if the caller is a coroutine or not, and provides the advantage of automating the code creation. In the future all g_c_w

[PATCH v4 05/11] block-coroutine-wrapper.py: default to main loop aiocontext if function does not have a BlockDriverState parameter

2022-11-16 Thread Emanuele Giuseppe Esposito
Basically BdrvPollCo->bs is only used by bdrv_poll_co(), and the functions that it uses are both using bdrv_get_aio_context, that defaults to qemu_get_aio_context() if bs is NULL. Therefore pass NULL to BdrvPollCo to automatically generate a function that create and runs a coroutine in the main

[PATCH v4 09/11] block: bdrv_create_file is a coroutine_fn

2022-11-16 Thread Emanuele Giuseppe Esposito
It is always called in coroutine_fn callbacks, therefore it can directly call bdrv_co_create(). Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 6 -- include/block/block-global-state.h | 3 ++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git

[PATCH v4 01/11] block-copy: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
These functions end up calling bdrv_common_block_status_above(), a generated_co_wrapper function. In addition, they also happen to be always called in coroutine context, meaning all callers are coroutine_fn. This means that the g_c_w function will enter the qemu_in_coroutine() case and eventually

[PATCH v4 07/11] block/vmdk: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
These functions end up calling bdrv_create() implemented as generated_co_wrapper functions. In addition, they also happen to be always called in coroutine context, meaning all callers are coroutine_fn. This means that the g_c_w function will enter the qemu_in_coroutine() case and eventually

[PATCH v4 02/11] nbd/server.c: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
These functions end up calling bdrv_*() implemented as generated_co_wrapper functions. In addition, they also happen to be always called in coroutine context, meaning all callers are coroutine_fn. This means that the g_c_w function will enter the qemu_in_coroutine() case and eventually suspend (or

[PATCH v4 03/11] block-backend: replace bdrv_*_above with blk_*_above

2022-11-16 Thread Emanuele Giuseppe Esposito
Avoid mixing bdrv_* functions with blk_*, so create blk_* counterparts for: - bdrv_block_status_above - bdrv_is_allocated_above Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 21 + block/commit.c| 4 ++--

[PATCH v4 11/11] block/dirty-bitmap: convert coroutine-only functions to generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
bdrv_can_store_new_dirty_bitmap and bdrv_remove_persistent_dirty_bitmap check if they are running in a coroutine, directly calling the coroutine callback if it's the case. Except that no coroutine calls such functions, therefore that check can be removed, and function creation can be offloaded to

[PATCH v4 00/11] Still more coroutine and various fixes in block layer

2022-11-16 Thread Emanuele Giuseppe Esposito
This is a dump of all minor coroutine-related fixes found while looking around and testing various things in the QEMU block layer. Patches aim to: - add missing coroutine_fn annotation to the functions - simplify to avoid the typical "if in coroutine: fn() // else create_coroutine(fn)" already

[PATCH v4 06/11] block-coroutine-wrapper.py: support also basic return types

2022-11-16 Thread Emanuele Giuseppe Esposito
Extend the regex to cover also return type, pointers included. This implies that the value returned by the function cannot be a simple "int" anymore, but the custom return type. Therefore remove poll_state->ret and instead use a per-function custom "ret" field. Signed-off-by: Emanuele Giuseppe

[PATCH v4 08/11] block: distinguish between bdrv_create running in coroutine and not

2022-11-16 Thread Emanuele Giuseppe Esposito
Call two different functions depending on whether bdrv_create is in coroutine or not, following the same pattern as generated_co_wrapper functions. This allows to also call the coroutine function directly, without using CreateCo or relying in bdrv_create(). Signed-off-by: Emanuele Giuseppe

[PATCH v4 10/11] block: convert bdrv_create to generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
This function is never called in coroutine context, therefore instead of manually creating a new coroutine, delegate it to the block-coroutine-wrapper script, defining it as g_c_w_simple. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 38

Re: [PATCH v2] m25p80: Improve error when the backend file size does not match the device

2022-11-16 Thread Francisco Iglesias
On [2022 Nov 15] Tue 16:10:00, Cédric Le Goater wrote: > Currently, when a block backend is attached to a m25p80 device and the > associated file size does not match the flash model, QEMU complains > with the error message "failed to read the initial flash content". > This is confusing for the

Re: [PATCH v3] block/rbd: Add support for layered encryption

2022-11-16 Thread Daniel P . Berrangé
On Wed, Nov 16, 2022 at 10:23:52AM +, Daniel P. Berrangé wrote: > On Wed, Nov 16, 2022 at 09:03:31AM +, Or Ozeri wrote: > > > -Original Message- > > > From: Daniel P. Berrangé > > > Sent: 15 November 2022 19:47 > > > To: Or Ozeri > > > Cc: qemu-de...@nongnu.org;

Re: [PATCH v3 0/8] Still more coroutine and various fixes in block layer

2022-11-16 Thread Emanuele Giuseppe Esposito
I apologize, as discussed also in v2 I just realized I could introduce generated_co_wrapper_simple already here and simplify patches 6 and 8. Also I think commit messages are the old ones from v1. I'll resend. Please ignore this serie. Emanuele Am 16/11/2022 um 09:50 schrieb Emanuele Giuseppe

Re: [PATCH v3] block/rbd: Add support for layered encryption

2022-11-16 Thread Daniel P . Berrangé
On Wed, Nov 16, 2022 at 09:03:31AM +, Or Ozeri wrote: > > -Original Message- > > From: Daniel P. Berrangé > > Sent: 15 November 2022 19:47 > > To: Or Ozeri > > Cc: qemu-de...@nongnu.org; qemu-block@nongnu.org; Danny Harnik > > ; idryo...@gmail.com > > Subject: [EXTERNAL] Re: [PATCH

RE: [PATCH v3] block/rbd: Add support for layered encryption

2022-11-16 Thread Or Ozeri
> -Original Message- > From: Daniel P. Berrangé > Sent: 15 November 2022 19:47 > To: Or Ozeri > Cc: qemu-de...@nongnu.org; qemu-block@nongnu.org; Danny Harnik > ; idryo...@gmail.com > Subject: [EXTERNAL] Re: [PATCH v3] block/rbd: Add support for layered > encryption > > AFAICT,

[PATCH v3 2/8] nbd/server.c: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
There are probably more missing, but right now it is necessary that we extend coroutine_fn to block{allock/status}_to_extents, because they use bdrv_* functions calling the generated_co_wrapper API, which checks for the qemu_in_coroutine() case. Signed-off-by: Emanuele Giuseppe Esposito ---

[PATCH v3 4/8] block: distinguish between bdrv_create running in coroutine and not

2022-11-16 Thread Emanuele Giuseppe Esposito
Call two different functions depending on whether bdrv_create is in coroutine or not, following the same pattern as generated_co_wrapper functions. This allows to also call the coroutine function directly, without using CreateCo or relying in bdrv_create(). It will also be useful when we add the

[PATCH v3 8/8] block/dirty-bitmap: remove unnecessary qemu_in_coroutine() case

2022-11-16 Thread Emanuele Giuseppe Esposito
Some functions check if they are running in a coroutine, calling the coroutine callback directly if it's the case. Except that no coroutine calls such functions, therefore that case can be removed. Signed-off-by: Emanuele Giuseppe Esposito --- block/dirty-bitmap.c | 66

[PATCH v3 1/8] block-copy: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
block_copy_reset_unallocated and block_copy_is_cluster_allocated are only called by backup_run, a corotuine_fn itself. Same applies to block_copy_block_status, called by block_copy_dirty_clusters. Therefore mark them as coroutine too. Signed-off-by: Emanuele Giuseppe Esposito ---

[PATCH v3 5/8] block/vmdk: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
vmdk_co_create_opts() is a coroutine_fn, and calls vmdk_co_do_create() which in turn can call two callbacks: vmdk_co_create_opts_cb and vmdk_co_create_cb. Mark all these functions as coroutine_fn, since vmdk_co_create_opts() is the only caller. Signed-off-by: Emanuele Giuseppe Esposito ---

[PATCH v3 6/8] block: bdrv_create_file is a coroutine_fn

2022-11-16 Thread Emanuele Giuseppe Esposito
It is always called in coroutine_fn callbacks, therefore it can directly call bdrv_co_create(). Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 6 -- include/block/block-global-state.h | 3 ++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git

[PATCH v3 3/8] block-backend: replace bdrv_*_above with blk_*_above

2022-11-16 Thread Emanuele Giuseppe Esposito
Avoid mixing bdrv_* functions with blk_*, so create blk_* counterparts for: - bdrv_block_status_above - bdrv_is_allocated_above Note that these functions will take the rdlock, so they must always run in a coroutine. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c

[PATCH v3 0/8] Still more coroutine and various fixes in block layer

2022-11-16 Thread Emanuele Giuseppe Esposito
This is a dump of all minor coroutine-related fixes found while looking around and testing various things in the QEMU block layer. Patches aim to: - add missing coroutine_fn annotation to the functions - simplify to avoid the typical "if in coroutine: fn() // else create_coroutine(fn)" already

[PATCH v3 7/8] block: bdrv_create is never called in coroutine context

2022-11-16 Thread Emanuele Giuseppe Esposito
Delete the if case and make sure it won't be called again in coroutines. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 37 - 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/block.c b/block.c index dcac28756c..7a4ce7948c 100644 ---

[PATCH RFC 2/3] hw/i2c: add mctp core

2022-11-16 Thread Klaus Jensen
From: Klaus Jensen Add an abstract MCTP over I2C endpoint model. This implements MCTP control message handling as well as handling the actual I2C transport (packetization). Devices are intended to derive from this and implement the class methods. Parts of this implementation is inspired by

[PATCH maybe-7.2 1/3] hw/i2c: only schedule pending master when bus is idle

2022-11-16 Thread Klaus Jensen
From: Klaus Jensen It is not given that the current master will release the bus after a transfer ends. Only schedule a pending master if the bus is idle. Fixes: 37fa5ca42623 ("hw/i2c: support multiple masters") Signed-off-by: Klaus Jensen --- hw/i2c/aspeed_i2c.c | 2 ++ hw/i2c/core.c

[PATCH RFC 3/3] hw/nvme: add nvme management interface model

2022-11-16 Thread Klaus Jensen
From: Klaus Jensen Add the 'nmi-i2c' device that emulates an NVMe Management Interface controller. Initial support is very basic (Read NMI DS, Configuration Get). This is based on previously posted code by Padmakar Kalghatgi, Arun Kumar Agasar and Saurav Kumar. Signed-off-by: Klaus Jensen

[PATCH 0/3] hw/{i2c, nvme}: mctp endpoint, nvme management interface model

2022-11-16 Thread Klaus Jensen
From: Klaus Jensen This adds a generic MCTP endpoint model that other devices may derive from. I'm not 100% happy with the design of the class methods, but it's a start. Patch 1 is a bug fix, but since there are currently no in-tree users of the API, it is not critical. I'd like to have Peter

  1   2   >