Re: query dirty areas according to bitmap via QMP or qemu-nbd

2024-07-29 Thread Fiona Ebner
Am 26.07.24 um 17:38 schrieb Eric Blake: > On Fri, Jul 26, 2024 at 04:16:41PM GMT, Fiona Ebner wrote: >> Hi, >> >> sorry if I'm missing the obvious, but is there a way to get the dirty >> areas according to a dirty bitmap via QMP? I mean as something like >>

query dirty areas according to bitmap via QMP or qemu-nbd

2024-07-26 Thread Fiona Ebner
Hi, sorry if I'm missing the obvious, but is there a way to get the dirty areas according to a dirty bitmap via QMP? I mean as something like offset + size + dirty-flag triples. In my case, the bitmap is also exported via NBD, so same question for qemu-nbd being the client. I can get the info wit

[PATCH v2] block/reqlist: allow adding overlapping requests

2024-07-12 Thread Fiona Ebner
t ../block/io.c:2474 > #12 0x6152853f2016 in bdrv_co_common_block_status_above (...) at > ../block/io.c:2652 > #13 0x6152853f22cf in bdrv_co_block_status_above (...) at > ../block/io.c:2732 > #14 0x6152853d9a86 in blk_co_block_status_above (...) at > ../block/block-backend

[PATCH] block/copy-before-write: wait for conflicts when read locking to avoid assertion failure

2024-07-11 Thread Fiona Ebner
t ../block/io.c:2474 > #12 0x6152853f2016 in bdrv_co_common_block_status_above (...) at > ../block/io.c:2652 > #13 0x6152853f22cf in bdrv_co_block_status_above (...) at > ../block/io.c:2732 > #14 0x6152853d9a86 in blk_co_block_status_above (...) at > ../block/bl

[PATCH v3 0/2] backup: allow specifying minimum cluster size

2024-07-11 Thread Fiona Ebner
um cluster size used for block-copy operations and thus in particular also the granularity for discard requests to the source. Fiona Ebner (2): copy-before-write: allow specifying minimum cluster size backup: add minimum cluster size to performance options block/backup.c | 2 +- b

[PATCH v3 1/2] copy-before-write: allow specifying minimum cluster size

2024-07-11 Thread Fiona Ebner
ulation in block_copy_calculate_cluster_size() is done in the target int64_t type. Suggested-by: Vladimir Sementsov-Ogievskiy Acked-by: Markus Armbruster (QAPI schema) Signed-off-by: Fiona Ebner --- Changes in v3: * Pass min_cluster_size option directly without checking has_min_cluster_size, because the de

[PATCH v3 2/2] backup: add minimum cluster size to performance options

2024-07-11 Thread Fiona Ebner
Armbruster (QAPI schema) Signed-off-by: Fiona Ebner --- Changes in v3: * Use PRI{i,u}64 macros * Do not leak qdict in error case block/backup.c| 2 +- block/copy-before-write.c | 9 + block/copy-before-write.h | 1 + blockdev.c| 3 +++ qapi/block-core.json

Re: [PATCH] scsi: Don't ignore most usb-storage properties

2024-07-01 Thread Fiona Ebner
Hi, we got a user report about bootindex for an 'usb-storage' device not working anymore [0] and I reproduced it and bisected it to this patch. Am 31.01.24 um 14:06 schrieb Kevin Wolf: > @@ -399,11 +397,10 @@ SCSIDevice *scsi_bus_legacy_add_drive(SCSIBus *bus, > BlockBackend *blk, > object_

Re: [PATCH v4 0/5] mirror: allow specifying working bitmap

2024-06-10 Thread Fiona Ebner
Ping Am 21.05.24 um 14:20 schrieb Fiona Ebner: > Changes from v3 (discussion here [3]): > * Improve/fix QAPI documentation. > > Changes from v2 (discussion here [2]): > * Cluster size caveats only apply to non-COW diff image, adapt the > cluster size check and documentation a

Re: [PATCH v3 2/4] block-backend: fix edge case in bdrv_next() where BDS associated to BB changes

2024-06-05 Thread Fiona Ebner
Am 04.06.24 um 17:28 schrieb Kevin Wolf: > Am 04.06.2024 um 09:58 hat Fiona Ebner geschrieben: >> Am 03.06.24 um 18:21 schrieb Kevin Wolf: >>> Am 03.06.2024 um 16:17 hat Fiona Ebner geschrieben: >>>> Am 26.03.24 um 13:44 schrieb Kevin Wolf: >>>>> >

[RFC PATCH] block-coroutine-wrapper: support generating wrappers for functions without arguments

2024-06-04 Thread Fiona Ebner
Signed-off-by: Fiona Ebner --- An alternative would be to detect whether the argument list is 'void' in FuncDecl's __init__, assign the empty list to self.args there and special case based on that in the rest of the code. Not super happy about the introduction of the 'void_v

Re: [PATCH v3 2/4] block-backend: fix edge case in bdrv_next() where BDS associated to BB changes

2024-06-04 Thread Fiona Ebner
Am 03.06.24 um 18:21 schrieb Kevin Wolf: > Am 03.06.2024 um 16:17 hat Fiona Ebner geschrieben: >> Am 26.03.24 um 13:44 schrieb Kevin Wolf: >>> >>> The fix for bdrv_flush_all() is probably to make it bdrv_co_flush_all() >>> with a coroutine wrapper so that

Re: [PATCH] block/copy-before-write: use uint64_t for timeout in nanoseconds

2024-06-03 Thread Fiona Ebner
Am 28.05.24 um 18:06 schrieb Kevin Wolf: > Am 29.04.2024 um 16:19 hat Fiona Ebner geschrieben: >> rather than the uint32_t for which the maximum is slightly more than 4 >> seconds and larger values would overflow. The QAPI interface allows >> specifying the number of seconds,

Re: [PATCH v3 2/4] block-backend: fix edge case in bdrv_next() where BDS associated to BB changes

2024-06-03 Thread Fiona Ebner
Hi Kevin, Am 26.03.24 um 13:44 schrieb Kevin Wolf: > Am 22.03.2024 um 10:50 hat Fiona Ebner geschrieben: >> The old_bs variable in bdrv_next() is currently determined by looking >> at the old block backend. However, if the block graph changes before >> the next bdrv_next() c

Re: [PATCH 1/2] Revert "monitor: use aio_co_reschedule_self()"

2024-05-29 Thread Fiona Ebner
CC-ing stable since 1f25c172f83704e350c0829438d832384084a74d is in 9.0.0 Am 06.05.24 um 21:06 schrieb Stefan Hajnoczi: > Commit 1f25c172f837 ("monitor: use aio_co_reschedule_self()") was a code > cleanup that uses aio_co_reschedule_self() instead of open coding > coroutine rescheduling. > > Bug R

Re: block snapshot issue with RBD

2024-05-29 Thread Fiona Ebner
Hi, Am 28.05.24 um 20:19 schrieb Jin Cao: > Hi Ilya > > On 5/28/24 11:13 AM, Ilya Dryomov wrote: >> On Mon, May 27, 2024 at 9:06 PM Jin Cao wrote: >>> >>> Supplementary info: VM is paused after "migrate" command. After being >>> resumed with "cont", snapshot_delete_blkdev_internal works again, w

[PATCH v2 1/2] copy-before-write: allow specifying minimum cluster size

2024-05-28 Thread Fiona Ebner
ulation in block_copy_calculate_cluster_size() is done in the target int64_t type. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona Ebner --- Changes in v2: * Use 'size' type in QAPI. * Remove option in cbw_parse_options(), i.e. before parsing generic blockdev options. block/block

[PATCH v2 2/2] backup: add minimum cluster size to performance options

2024-05-28 Thread Fiona Ebner
and thus effectively ignore then. To make @discard-source work in such a scenario, allow specifying the minimum cluster size used for block-copy operations and thus in particular also the granularity for discard requests to the source. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona

[PATCH v2 0/2] backup: allow specifying minimum cluster size

2024-05-28 Thread Fiona Ebner
cular also the granularity for discard requests to the source. Fiona Ebner (2): copy-before-write: allow specifying minimum cluster size backup: add minimum cluster size to performance options block/backup.c | 2 +- block/block-copy.c | 22 ++ block/c

[PATCH v4 0/5] mirror: allow specifying working bitmap

2024-05-21 Thread Fiona Ebner
l.org/qemu-devel/20240307134711.709816-1-f.eb...@proxmox.com/ [3]: https://lore.kernel.org/qemu-devel/20240510131647.1256467-1-f.eb...@proxmox.com/ Fabian Grünbichler (1): iotests: add test for bitmap mirror Fiona Ebner (3): qapi/block-core: avoid the re-use of MirrorSyncMode for backup bl

[PATCH v4 3/5] mirror: allow specifying working bitmap

2024-05-21 Thread Fiona Ebner
t rid of bitmap mode parameter use caller-provided bitmap as working bitmap turn bitmap parameter experimental] Signed-off-by: Fiona Ebner Acked-by: Markus Armbruster --- block/mirror.c | 80 +- blockdev.c | 44

[PATCH v4 5/5] blockdev: mirror: check for target's cluster size when using bitmap

2024-05-21 Thread Fiona Ebner
iff image use-case, simply skip the check then. Signed-off-by: Fiona Ebner --- blockdev.c | 57 ++ tests/qemu-iotests/tests/mirror-bitmap | 6 +++ tests/qemu-iotests/tests/mirror-bitmap.out | 7 +++ 3 files changed, 70 insertions(+) diff --

[PATCH v4 4/5] iotests: add test for bitmap mirror

2024-05-21 Thread Fiona Ebner
-bitmap' use backing files, copy-mode=write-blocking, larger cluster size] Signed-off-by: Fiona Ebner --- tests/qemu-iotests/tests/mirror-bitmap | 597 tests/qemu-iotests/tests/mirror-bitmap.out | 3191 2 files changed, 3788 insertions(+) create mode 1

[PATCH v4 2/5] block/mirror: replace is_none_mode with sync_mode in MirrorBlockJob struct

2024-05-21 Thread Fiona Ebner
a clean error. Signed-off-by: Fiona Ebner --- block/mirror.c | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index c0597039a5..ca23d6ef65 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -51,7 +51,7 @@ typedef struct

[PATCH v4 1/5] qapi/block-core: avoid the re-use of MirrorSyncMode for backup

2024-05-21 Thread Fiona Ebner
introspection: query-qmp-schema no longer reports drive-mirror and blockdev-mirror accepting @sync values they actually reject. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona Ebner Acked-by: Markus Armbruster Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/backup.c

Re: [PATCH v4] hw/pflash: fix block write start

2024-05-16 Thread Fiona Ebner
Am 16.05.24 um 10:46 schrieb Gerd Hoffmann: > Move the pflash_blk_write_start() call. We need the offset of the > first data write, not the offset for the setup (number-of-bytes) > write. Without this fix u-boot can do block writes to the first > flash block only. > > While being at it drop a le

Re: [PATCH 1/2] copy-before-write: allow specifying minimum cluster size

2024-05-13 Thread Fiona Ebner
Am 26.03.24 um 10:06 schrieb Markus Armbruster: >> @@ -365,7 +368,13 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, >> BdrvChild *target, >> >> GLOBAL_STATE_CODE(); >> >> -cluster_size = block_copy_calculate_cluster_size(target->bs, errp); >> +if (min_cluster_size && !

[PATCH v3 2/5] block/mirror: replace is_none_mode with sync_mode in MirrorBlockJob struct

2024-05-10 Thread Fiona Ebner
a clean error. Signed-off-by: Fiona Ebner --- New in v3. block/mirror.c | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index c0597039a5..ca23d6ef65 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -51,7 +51,7

[PATCH v3 0/5] mirror: allow specifying working bitmap

2024-05-10 Thread Fiona Ebner
TSUP. [0]: https://lore.kernel.org/qemu-devel/b91dba34-7969-4d51-ba40-96a91038c...@yandex-team.ru/T/#m4ae27dc8ca1fb053e0a32cc4ffa2cfab6646805c [1]: https://lore.kernel.org/qemu-devel/1599127031.9uxdp5h9o2.astr...@nora.none/ [2]: https://lore.kernel.org/qemu-devel/20240307134711.709816-1-f.eb...@proxmox.com/ Fabian Gr

[PATCH v3 4/5] iotests: add test for bitmap mirror

2024-05-10 Thread Fiona Ebner
-bitmap' use backing files, copy-mode=write-blocking, larger cluster size] Signed-off-by: Fiona Ebner --- Changes in v3: * avoid script potentially waiting on non-existent job when blockdev-mirror QMP command fails by asserting that there is no error when none is expected. * fix py

[PATCH v3 1/5] qapi/block-core: avoid the re-use of MirrorSyncMode for backup

2024-05-10 Thread Fiona Ebner
introspection: query-qmp-schema no longer reports drive-mirror and blockdev-mirror accepting @sync values they actually reject. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona Ebner Acked-by: Markus Armbruster Reviewed-by: Vladimir Sementsov-Ogievskiy --- Changes in v3: * add

[PATCH v3 5/5] blockdev: mirror: check for target's cluster size when using bitmap

2024-05-10 Thread Fiona Ebner
iff image use-case, simply skip the check then. Signed-off-by: Fiona Ebner --- Changes in v3: * detect when the target does COW and do not error out in that case * treat ENOTSUP differently from other failure when querying the cluster size blockdev.c | 57 +++

[PATCH v3 3/5] mirror: allow specifying working bitmap

2024-05-10 Thread Fiona Ebner
t rid of bitmap mode parameter use caller-provided bitmap as working bitmap turn bitmap parameter experimental] Signed-off-by: Fiona Ebner --- Changes in v3: * remove duplicate "use" in QAPI description * clarify that cluster size caveats only applies to non-COW diff image * split c

Re: [PATCH v2 2/4] mirror: allow specifying working bitmap

2024-05-08 Thread Fiona Ebner
Am 07.05.24 um 14:15 schrieb Fiona Ebner: > Am 02.04.24 um 22:14 schrieb Vladimir Sementsov-Ogievskiy: >> On 07.03.24 16:47, Fiona Ebner wrote: >>> +# @bitmap: The name of a bitmap to use as a working bitmap for >>> +# sync=full mode.  This argument must

Re: [PATCH v2 2/4] mirror: allow specifying working bitmap

2024-05-07 Thread Fiona Ebner
Am 02.04.24 um 22:14 schrieb Vladimir Sementsov-Ogievskiy: > On 07.03.24 16:47, Fiona Ebner wrote: >> diff --git a/block/mirror.c b/block/mirror.c >> index 1609354db3..5c9a00b574 100644 >> --- a/block/mirror.c >> +++ b/block/mirror.c >> @@ -51,7 +51,7

Re: [PATCH] block/copy-before-write: use uint64_t for timeout in nanoseconds

2024-04-29 Thread Fiona Ebner
Am 29.04.24 um 16:36 schrieb Philippe Mathieu-Daudé: > Hi Fiona, > > On 29/4/24 16:19, Fiona Ebner wrote: > > Not everybody uses an email client that shows the patch content just > after the subject (your first lines wasn't making sense at first). > > Simply dup

[PATCH] block/copy-before-write: use uint64_t for timeout in nanoseconds

2024-04-29 Thread Fiona Ebner
() call where this is used already takes a uint64_t for the timeout, so no change required there. Fixes: 6db7fd1ca9 ("block/copy-before-write: implement cbw-timeout option") Reported-by: Friedrich Weber Signed-off-by: Fiona Ebner --- block/copy-before-write.c | 2 +- 1 file changed, 1 inser

Re: [PATCH] block: Remove unnecessary NULL check in bdrv_pad_request()

2024-03-28 Thread Fiona Ebner
gt; > Resolves: Coverity CID 1542668 > Signed-off-by: Kevin Wolf Reviewed-by: Fiona Ebner Thank you for the fix, Fiona

Question about block graph lock limitation with generated co-wrappers

2024-03-26 Thread Fiona Ebner
Hi, we have a custom block driver downstream, which currently calls bdrv_get_info() (for its file child) in the bdrv_refresh_limits() callback. However, with graph locking, this doesn't work anymore. AFAICT, the reason is the following: The block driver has a backing file option. During initializa

[PATCH v3 1/4] block/io: accept NULL qiov in bdrv_pad_request

2024-03-22 Thread Fiona Ebner
ot;blockdev-add", "arguments": { "driver": "compress", "file": > "node0", "node-name": "node1" } } > {"execute": "block-stream", "arguments": { "job-id": "stream0", &

[PATCH v3 4/4] iotests: add test for stream job with an unaligned prefetch read

2024-03-22 Thread Fiona Ebner
bdrv_flush_all() could lead to unreferencing the wrong block driver state and an assertion failure later. Signed-off-by: Fiona Ebner --- No changes in v3. New in v2. .../tests/stream-unaligned-prefetch | 86 +++ .../tests/stream-unaligned-prefetch.out | 5 ++ 2

[PATCH v3 0/4] fix two edge cases related to stream block jobs

2024-03-22 Thread Fiona Ebner
wanted to have (good call :)), so include a fix for that and add the test. I didn't notice during manual testing, because I hadn't used a scripted QMP 'quit', so there was no race. Fiona Ebner (3): block-backend: fix edge case in bdrv_next() where BDS associate

[PATCH v3 2/4] block-backend: fix edge case in bdrv_next() where BDS associated to BB changes

2024-03-22 Thread Fiona Ebner
) > #8 job_do_finalize_locked (job=...) > #9 job_exit (opaque=...) > #10 aio_bh_poll (ctx=...) > #11 aio_poll (ctx=..., blocking=...) > #12 bdrv_poll_co (s=...) > #13 bdrv_flush (bs=...) > #14 bdrv_flush_all () > #15 do_vm_stop (state=..., send_stop=...) > #16 vm_

[PATCH v3 3/4] block-backend: fix edge case in bdrv_next_cleanup() where BDS associated to BB changes

2024-03-22 Thread Fiona Ebner
ly anymore. Instead, rely on bdrv_next() to set it->bs to the BDS it referenced and unreference that one in any case. Signed-off-by: Fiona Ebner --- New in v3. block/block-backend.c | 11 --- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/block/block-backend.c b/block/bl

[PATCH v2 0/3] fix two edge cases related to stream block jobs

2024-03-21 Thread Fiona Ebner
Changes in v2: * Ran into another issue while writing the IO test Stefan wanted to have (good call :)), so include a fix for that and add the test. I didn't notice during manual testing, because I hadn't used a scripted QMP 'quit', so there was no r

[PATCH v2 3/3] iotests: add test for stream job with an unaligned prefetch read

2024-03-21 Thread Fiona Ebner
bdrv_flush_all() could lead to unreferencing the wrong block driver state and an assertion failure later. Signed-off-by: Fiona Ebner --- New in v2. .../tests/stream-unaligned-prefetch | 86 +++ .../tests/stream-unaligned-prefetch.out | 5 ++ 2 files changed, 91

[PATCH v2 1/3] block/io: accept NULL qiov in bdrv_pad_request

2024-03-21 Thread Fiona Ebner
ot;blockdev-add", "arguments": { "driver": "compress", "file": > "node0", "node-name": "node1" } } > {"execute": "block-stream", "arguments": { "job-id": "stream0", &

[PATCH v2 2/3] block-backend: fix edge case in bdrv_next() where BDS associated to BB changes

2024-03-21 Thread Fiona Ebner
) > #8 job_do_finalize_locked (job=...) > #9 job_exit (opaque=...) > #10 aio_bh_poll (ctx=...) > #11 aio_poll (ctx=..., blocking=...) > #12 bdrv_poll_co (s=...) > #13 bdrv_flush (bs=...) > #14 bdrv_flush_all () > #15 do_vm_stop (state=..., send_stop=...) > #16 vm_shutdown ()

[PATCH] block/io: accept NULL qiov in bdrv_pad_request

2024-03-19 Thread Fiona Ebner
ot;blockdev-add", "arguments": { "driver": "compress", "file": > "node0", "node-name": "node1" } } > {"execute": "block-stream", "arguments": { "job-id": "stream0", &

[PATCH 1/2] copy-before-write: allow specifying minimum cluster size

2024-03-08 Thread Fiona Ebner
small, they will just be aligned down in cbw_co_pdiscard_snapshot() and thus effectively ignored. The QAPI uses uint32 so the value will be non-negative, but still fit into a uint64_t. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona Ebner --- block/block-copy.c | 17

[PATCH 0/2] backup: allow specifying minimum cluster size

2024-03-08 Thread Fiona Ebner
copy operations and in particular, discard requests to the backup source will too. If the granularity is too small, they will just be aligned down in cbw_co_pdiscard_snapshot() and thus effectively ignored. Fiona Ebner (2): copy-before-write: allow specifying minimum cluster size backup: add

[PATCH 2/2] backup: add minimum cluster size to performance options

2024-03-08 Thread Fiona Ebner
granularity is too small, they will just be aligned down in cbw_co_pdiscard_snapshot() and thus effectively ignored. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona Ebner --- block/backup.c| 2 +- block/copy-before-write.c | 2 ++ block/copy-before-write.h | 1 + blockdev.c

Re: [PATCH v3 3/5] block/copy-before-write: create block_copy bitmap in filter node

2024-03-08 Thread Fiona Ebner
eate copy_bitmap in filter node, where > anyway two other bitmaps are created. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Fiona Ebner

Re: [PATCH v3 0/5] backup: discard-source parameter

2024-03-08 Thread Fiona Ebner
qemu-iotests/tests/backup-discard-source > create mode 100644 tests/qemu-iotests/tests/backup-discard-source.out > Tested-by: Fiona Ebner

Re: [PATCH v3 5/5] iotests: add backup-discard-source

2024-03-08 Thread Fiona Ebner
# Copyright (c) 2022 Virtuozzo International GmbH. > +# Title and copyright year are wrong. Apart from that: Reviewed-by: Fiona Ebner

Re: [PATCH v3 2/5] block/copy-before-write: support unligned snapshot-discard

2024-03-08 Thread Fiona Ebner
Am 28.02.24 um 15:14 schrieb Vladimir Sementsov-Ogievskiy: > First thing that crashes on unligned access here is > bdrv_reset_dirty_bitmap(). Correct way is to align-down the > snapshot-discard request. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Fiona Ebner

Re: [PATCH v3 4/5] qapi: blockdev-backup: add discard-source parameter

2024-03-08 Thread Fiona Ebner
card to work. Still we can't take it > unconditionally, as it will break normal backup from RO source. So, we > have to add a parameter and pass it thorough bdrv_open flags. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Fiona Ebner

Re: [PATCH v2 00/10] mirror: allow switching from background to active mode

2024-03-08 Thread Fiona Ebner
Am 07.03.24 um 20:42 schrieb Vladimir Sementsov-Ogievskiy: > On 04.03.24 14:09, Peter Krempa wrote: >> On Mon, Mar 04, 2024 at 11:48:54 +0100, Kevin Wolf wrote: >>> Am 28.02.2024 um 19:07 hat Vladimir Sementsov-Ogievskiy geschrieben: On 03.11.23 18:56, Markus Armbruster wrote: > Kevin Wolf

[PATCH v2 1/4] qapi/block-core: avoid the re-use of MirrorSyncMode for backup

2024-03-07 Thread Fiona Ebner
-Ogievskiy Signed-off-by: Fiona Ebner --- I felt like keeping the "Since: X.Y" as before makes the most sense as to not lose history. Or is it necessary to change this for BackupSyncMode (and its members) since it got a new name? block/backup.c

[PATCH v2 0/4] mirror: allow specifying working bitmap

2024-03-07 Thread Fiona Ebner
team.ru/T/#m4ae27dc8ca1fb053e0a32cc4ffa2cfab6646805c [1]: https://lore.kernel.org/qemu-devel/1599127031.9uxdp5h9o2.astr...@nora.none/ Fabian Grünbichler (1): iotests: add test for bitmap mirror Fiona Ebner (2): qapi/block-core: avoid the re-use of MirrorSyncMode for backup blockdev: mirror:

[PATCH v2 3/4] iotests: add test for bitmap mirror

2024-03-07 Thread Fiona Ebner
is used as the working bitmap. Signed-off-by: Fabian Grünbichler Signed-off-by: Thomas Lamprecht [FE: rebase for 9.0 adapt to changes to mirror bitmap interface rename test from '384' to 'bitmap-sync-mirror'] Signed-off-by: Fiona Ebner --- tests/qemu-iotests/tes

[PATCH v2 2/4] mirror: allow specifying working bitmap

2024-03-07 Thread Fiona Ebner
t rid of bitmap mode parameter use caller-provided bitmap as working bitmap turn bitmap parameter experimental] Signed-off-by: Fiona Ebner --- block/mirror.c | 95 -- blockdev.c | 39 +-- include/block/block_in

[PATCH v2 4/4] blockdev: mirror: check for target's cluster size when using bitmap

2024-03-07 Thread Fiona Ebner
ly skip the check then. Signed-off-by: Fiona Ebner --- blockdev.c| 19 +++ tests/qemu-iotests/tests/bitmap-sync-mirror | 6 ++ .../qemu-iotests/tests/bitmap-sync-mirror.out | 7 +++ 3 files changed, 32 insertions(+) diff --

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-03-06 Thread Fiona Ebner
Am 29.02.24 um 13:47 schrieb Fiona Ebner: > Am 29.02.24 um 12:48 schrieb Vladimir Sementsov-Ogievskiy: >> On 29.02.24 13:11, Fiona Ebner wrote: >>> >>> The iotest creates a new target image for each incremental sync which >>> only records the diff relative t

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-03-03 Thread Fiona Ebner
Am 01.03.24 um 16:46 schrieb Vladimir Sementsov-Ogievskiy: > On 01.03.24 18:14, Fiona Ebner wrote: >> Am 01.03.24 um 16:02 schrieb Vladimir Sementsov-Ogievskiy: >>>>> About documentation: actually, I never liked that we use for backup >>>>> job >>

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-03-01 Thread Fiona Ebner
Am 01.03.24 um 16:02 schrieb Vladimir Sementsov-Ogievskiy: > On 01.03.24 17:52, Fiona Ebner wrote: >> Am 01.03.24 um 15:14 schrieb Vladimir Sementsov-Ogievskiy: >>> >>> As we already understood, (block-)job-api needs some spring-cleaning. >>> Unfortunately I don

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-03-01 Thread Fiona Ebner
Am 01.03.24 um 15:14 schrieb Vladimir Sementsov-Ogievskiy: > > As we already understood, (block-)job-api needs some spring-cleaning. > Unfortunately I don't have much time on it, but still I decided to start > from finally depreacting block-job-* API and moving to job-*.. Probably > bitmap/bitmap-

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-02-29 Thread Fiona Ebner
Am 29.02.24 um 13:00 schrieb Vladimir Sementsov-Ogievskiy: > > But anyway, this all could be simply achieved with > bitmap-copying/merging API, if we allow to pass user-given bitmap to the > mirror as working bitmap. > >> >> I see, I'll drop the 'bitmap-mode' in the next version if nobody >> comp

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-02-29 Thread Fiona Ebner
Am 29.02.24 um 12:48 schrieb Vladimir Sementsov-Ogievskiy: > On 29.02.24 13:11, Fiona Ebner wrote: >> >> The iotest creates a new target image for each incremental sync which >> only records the diff relative to the previous mirror and those diff >> images are later reb

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-02-29 Thread Fiona Ebner
Am 28.02.24 um 17:24 schrieb Vladimir Sementsov-Ogievskiy: > On 16.02.24 13:55, Fiona Ebner wrote: >> Previous discussion from when this was sent upstream [0] (it's been a >> while). I rebased the patches and re-ordered and squashed like >> suggested back then [1]. >

Re: [RFC 0/4] mirror: implement incremental and bitmap modes

2024-02-29 Thread Fiona Ebner
Am 28.02.24 um 17:06 schrieb Vladimir Sementsov-Ogievskiy: > On 28.02.24 19:00, Vladimir Sementsov-Ogievskiy wrote: >> On 16.02.24 13:55, Fiona Ebner wrote: >>> Now, the IO test added in patch 4/4 actually contains yet another use >>> case, namely doing incremental m

Re: [RFC 1/4] drive-mirror: add support for sync=bitmap mode=never

2024-02-21 Thread Fiona Ebner
Am 21.02.24 um 07:55 schrieb Markus Armbruster: >> diff --git a/qapi/block-core.json b/qapi/block-core.json >> index ab5a93a966..ac05483958 100644 >> --- a/qapi/block-core.json >> +++ b/qapi/block-core.json >> @@ -2181,6 +2181,15 @@ >> # destination (all the disk, only the sectors allocated in

[RFC 1/4] drive-mirror: add support for sync=bitmap mode=never

2024-02-16 Thread Fiona Ebner
amprecht [FE: rebase for 9.0 update version and formatting in QAPI] Signed-off-by: Fiona Ebner --- block/mirror.c | 96 -- blockdev.c | 38 +- include/block/block_int-global-state.h | 4 +- qapi/block-core.js

[RFC 3/4] mirror: move some checks to qmp

2024-02-16 Thread Fiona Ebner
] Signed-off-by: Fiona Ebner --- block/mirror.c | 28 +--- blockdev.c | 29 + 2 files changed, 34 insertions(+), 23 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 84155b1f78..15d1c060eb 100644 --- a/block/mirror.c +++ b/block

[RFC 4/4] iotests: add test for bitmap mirror

2024-02-16 Thread Fiona Ebner
names like vm.command -> vm.cmd, specifying explicit image format for rebase, adapt to new behavior of qemu_img(), dropping of 'status' field in output, etc. rename test from '384' to 'bitmap-sync-mirror'] Signed-off-by: Fiona Ebner --- tests/qem

[RFC 0/4] mirror: implement incremental and bitmap modes

2024-02-16 Thread Fiona Ebner
Previous discussion from when this was sent upstream [0] (it's been a while). I rebased the patches and re-ordered and squashed like suggested back then [1]. This implements two new mirror modes: - bitmap mirror mode with always/on-success/never bitmap sync mode - incremental mirror mode as sugar

[RFC 2/4] drive-mirror: add support for conditional and always bitmap sync modes

2024-02-16 Thread Fiona Ebner
-by: Fiona Ebner --- The original patch this was based on came from a WIP git branch and thus has no Signed-off-by trailer from John, see [0]. I added an Originally-by trailer for now. Let me know if I should drop that and wait for John's Signed-off-by instead. [0] https://lore.kernel.org

[PATCH] iotests: adapt to output change for recently introduced 'detached header' field

2024-02-16 Thread Fiona Ebner
Failure was noticed when running the tests for the qcow2 image format. Fixes: 0bd779e27e ("crypto: Introduce 'detached-header' field in QCryptoBlockInfoLUKS") Signed-off-by: Fiona Ebner --- tests/qemu-iotests/198.out | 2 ++ tests/qemu-iotests/206.out | 1 + 2 files c

Re: double free or corruption (out) in iscsi virtual machine

2024-02-15 Thread Fiona Ebner
Am 17.01.24 um 08:23 schrieb M_O_Bz: > Basic Info: > 1. Issue: I got a " double free or corruption (out)", head for > attachment debug.log for details, the debug.log print the backtrace of > one virtual machine > 2. Reproduce: currently I cann't destribe how to reproduce this bug, > because it's in

Re: [PATCH v2 3/4] qapi: blockdev-backup: add discard-source parameter

2024-01-26 Thread Fiona Ebner
Am 25.01.24 um 18:22 schrieb Vladimir Sementsov-Ogievskiy: > > Hmm. Taking maximum is not optimal for usual case without > discard-source: user may want to work in smaller granularity than > source, to save disk space. > > In case with discarding we have two possibilities: > > - either take larg

Re: [PATCH v2 3/4] qapi: blockdev-backup: add discard-source parameter

2024-01-25 Thread Fiona Ebner
Am 24.01.24 um 16:03 schrieb Fiona Ebner: > Am 17.01.24 um 17:07 schrieb Vladimir Sementsov-Ogievskiy: >> Add a parameter that enables discard-after-copy. That is mostly useful >> in "push backup with fleecing" scheme, when source is snapshot-access >> format dr

Re: [PATCH 2/2] virtio: Keep notifications disabled during drain

2024-01-25 Thread Fiona Ebner
ause we will always miss virtqueue updates in the drained section, we > also need to poll the virtqueue once after attaching the notifiers. > > Buglink: https://issues.redhat.com/browse/RHEL-3934 > Signed-off-by: Hanna Czenczek Tested-by: Fiona Ebner Reviewed-by: Fiona Ebner

Re: [PATCH 1/2] virtio-scsi: Attach event vq notifier with no_poll

2024-01-25 Thread Fiona Ebner
ix it by using virtio_queue_aio_attach_host_notifier_no_poll() for the > event virtqueue. > > Reported-by: Fiona Ebner > Fixes: 766aa2de0f29b657148e04599320d771c36fd126 > ("virtio-scsi: implement BlockDevOps->drained_begin()") > Signed-off-by: Hanna Czenczek Tested-by: Fiona Ebner Reviewed-by: Fiona Ebner

Re: [PATCH v2 3/4] qapi: blockdev-backup: add discard-source parameter

2024-01-24 Thread Fiona Ebner
Am 17.01.24 um 17:07 schrieb Vladimir Sementsov-Ogievskiy: > Add a parameter that enables discard-after-copy. That is mostly useful > in "push backup with fleecing" scheme, when source is snapshot-access > format driver node, based on copy-before-write filter snapshot-access > API: > > [guest]

[PATCH] block/io_uring: improve error message when init fails

2024-01-23 Thread Fiona Ebner
ode is returned. The caller should not > rely on errno variable. Tested using 'sysctl kernel.io_uring_disabled=2'. Output before this change: > failed to init linux io_uring ring Output after this change: > failed to init linux io_uring ring: Operation not permitted Signed-off-by

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-23 Thread Fiona Ebner
Am 22.01.24 um 18:52 schrieb Hanna Czenczek: > On 22.01.24 18:41, Hanna Czenczek wrote: >> On 05.01.24 15:30, Fiona Ebner wrote: >>> Am 05.01.24 um 14:43 schrieb Fiona Ebner: >>>> Am 03.01.24 um 14:35 schrieb Paolo Bonzini: >>>>> On 1/3/24 12:40, Fiona

Re: [PATCH v2 3/4] qapi: blockdev-backup: add discard-source parameter

2024-01-19 Thread Fiona Ebner
Am 17.01.24 um 17:07 schrieb Vladimir Sementsov-Ogievskiy: > Add a parameter that enables discard-after-copy. That is mostly useful > in "push backup with fleecing" scheme, when source is snapshot-access > format driver node, based on copy-before-write filter snapshot-access > API: > > [guest]

Re: [PATCH 3/3] monitor: only run coroutine commands in qemu_aio_context

2024-01-18 Thread Fiona Ebner
id a bit of smoke testing with some other QMP and QGA commands and didn't find any obvious breakage, so: Tested-by: Fiona Ebner P.S. Unfortunately, the patch does not solve the other issue I came across back then [0] with snapshot_save_job_bh() being executed during a vCPU thread'

[PATCH] block/io: clear BDRV_BLOCK_RECURSE flag after recursing in bdrv_co_block_status

2024-01-16 Thread Fiona Ebner
e1", "node-name": "node3" } } > {"execute": "blockdev-add", "arguments": { "driver": "snapshot-access", > "file": "node3", "node-name": "snap0" } } > {"execute": &q

Re: [PATCH 2/3] qapi: blockdev-backup: add discard-source parameter

2024-01-11 Thread Fiona Ebner
Hi Vladimir, hope I didn't miss a newer version of this series. I'm currently evaluating fleecing backup for Proxmox downstream, so I pulled in this series and wanted to let you know about two issues I encountered while testing. We are still based on 8.1, but if I'm not mistaken, they are still re

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-05 Thread Fiona Ebner
Am 05.01.24 um 14:43 schrieb Fiona Ebner: > Am 03.01.24 um 14:35 schrieb Paolo Bonzini: >> On 1/3/24 12:40, Fiona Ebner wrote: >>> I'm happy to report that I cannot reproduce the CPU-usage-spike issue >>> with the patch, but I did run into an assertion failure w

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-05 Thread Fiona Ebner
Am 03.01.24 um 14:35 schrieb Paolo Bonzini: > On 1/3/24 12:40, Fiona Ebner wrote: >> I'm happy to report that I cannot reproduce the CPU-usage-spike issue >> with the patch, but I did run into an assertion failure when trying to >> verify that it fixes my original stuck

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-03 Thread Fiona Ebner
Am 02.01.24 um 16:24 schrieb Hanna Czenczek: > > I’ve attached the preliminary patch that I didn’t get to send (or test > much) last year.  Not sure if it has the same CPU-usage-spike issue > Fiona was seeing, the only functional difference is that I notify the vq > after attaching the notifiers i

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-19 Thread Fiona Ebner
Am 18.12.23 um 15:49 schrieb Paolo Bonzini: > On Mon, Dec 18, 2023 at 1:41 PM Fiona Ebner wrote: >> I think it's because of nested drains, because when additionally >> checking that the drain count is zero and only executing the loop then, >> that issue doesn't

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-18 Thread Fiona Ebner
Am 14.12.23 um 20:53 schrieb Stefan Hajnoczi: > > I will still try the other approach that Hanna and Paolo have suggested. > It seems more palatable. I will send a v2. > FYI, what I already tried downstream (for VirtIO SCSI): > diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c > index

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-14 Thread Fiona Ebner
Am 13.12.23 um 22:15 schrieb Stefan Hajnoczi: > But there you have it. Please let me know what you think and try your > reproducers to see if this fixes the missing io_poll_end() issue. Thanks! > Thanks to you! I applied the RFC (and the series it depends on) on top of 8.2.0-rc4 and this fixes my

Re: [PULL 29/32] virtio-blk: implement BlockDevOps->drained_begin()

2023-12-11 Thread Fiona Ebner
Am 08.12.23 um 09:32 schrieb Kevin Wolf: > > I'm not involved in it myself, but the kind of theme reminds me of this > downstream bug that Hanna analysed recently: > > https://issues.redhat.com/browse/RHEL-3934 > > Does it look like the same root cause to you? > Thank you for the reference! Ye

Re: [PULL 29/32] virtio-blk: implement BlockDevOps->drained_begin()

2023-12-07 Thread Fiona Ebner
Am 03.11.23 um 14:12 schrieb Fiona Ebner: > Hi, > > I ran into a strange issue where guest IO would get completely stuck > during certain block jobs a while ago and finally managed to find a > small reproducer [0]. I'm using a VM with virtio-blk-pci (or > virtio-scsi-pc

Re: [PULL 29/32] virtio-blk: implement BlockDevOps->drained_begin()

2023-11-13 Thread Fiona Ebner
Am 03.11.23 um 14:12 schrieb Fiona Ebner: > Hi, > > Am 30.05.23 um 18:32 schrieb Kevin Wolf: >> From: Stefan Hajnoczi >> >> Detach ioeventfds during drained sections to stop I/O submission from >> the guest. virtio-blk is no longer reliant on aio_disable_external

Re: deadlock when using iothread during backup_clean()

2023-11-03 Thread Fiona Ebner
Am 20.10.23 um 15:52 schrieb Fiona Ebner: > And I found that with drive-mirror, the issue during starting seems to > manifest with the bdrv_open() call. Adding a return before it, the guest > IO didn't get stuck in my testing, but adding a return after it, it can > get stuck. I&#

  1   2   3   >