[PULL 17/31] iotests: qemu-img tests for luks key management

2020-07-06 Thread Max Reitz
and 294 from the auto group] Signed-off-by: Max Reitz Message-Id: <20200625125548.870061-16-mre...@redhat.com> --- tests/qemu-iotests/293 | 208 + tests/qemu-iotests/293.out | 99 ++ tests/qemu-iotests/294 | 90 test

[PULL 06/31] iotests.py: Add qemu_img_pipe_and_status()

2020-07-06 Thread Max Reitz
This function will be used by the next patch, which intends to check both the exit code and qemu-img's output. Signed-off-by: Max Reitz Message-Id: <20200625125548.870061-5-mre...@redhat.com> Reviewed-by: Maxim Levitsky [mreitz: Rebased on 49438972b8c2e] Signed-off-by: Max Reitz ---

[PULL 10/31] qcrypto/luks: implement encryption key management

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Next few patches will expose that functionality to the user. Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Message-Id: <20200608094030.670121-3-mlevi...@redhat.com> Signed-off-by: Max Reitz --- qapi/crypto.json| 59 ++- crypto/block-luks.c

[PULL 09/31] qcrypto/core: add generic infrastructure for crypto options amendment

2020-07-06 Thread Max Reitz
evi...@redhat.com> Signed-off-by: Max Reitz --- qapi/crypto.json | 16 block/crypto.h | 3 +++ crypto/blockpriv.h | 8 include/crypto/block.h | 22 ++ block/crypto.c | 17 + crypto/block.c

[PULL 04/31] iotests: filter few more luks specific create options

2020-07-06 Thread Max Reitz
From: Maxim Levitsky This allows more tests to be able to have same output on both qcow2 luks encrypted images and raw luks images Signed-off-by: Maxim Levitsky Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky Message-Id: <20200625125548.870061-3-mre...@redhat.com> --- test

[PULL 01/31] block/block-copy: block_copy_dirty_clusters: fix failure check

2020-07-06 Thread Max Reitz
igned-off-by: Max Reitz --- block/block-copy.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/block/block-copy.c b/block/block-copy.c index bb8d0569f2..f7428a7c08 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -622,8 +622,10 @@ out: * block_copy

[PULL 05/31] iotests/common.rc: Add _require_working_luks

2020-07-06 Thread Max Reitz
That the luks driver is present is little indication on whether it is actually working. Without the crypto libraries linked in, it does not work. So add this function, which tries to create a luks image to see whether that actually works. Signed-off-by: Max Reitz Message-Id

[PULL 02/31] qcow2: Fix preallocation on images with unaligned sizes

2020-07-06 Thread Max Reitz
be filled with data from the backing file. This patch fixes both issues. Signed-off-by: Alberto Garcia Message-Id: <20200617140036.20311-1-be...@igalia.com> Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- block/qcow2.c | 17 ++--- tests/qemu-iotests/125

[PULL 11/31] block/amend: add 'force' option

2020-07-06 Thread Max Reitz
Levitsky Reviewed-by: Daniel P. Berrangé Reviewed-by: Max Reitz Message-Id: <20200608094030.670121-4-mlevi...@redhat.com> Signed-off-by: Max Reitz --- docs/tools/qemu-img.rst | 5 - include/block/block.h | 1 + include/block/block_int.h | 1 + block.c | 4 +++-

[PULL 00/31] Block patches

2020-07-06 Thread Max Reitz
preallocation when the image size is not a multiple of the cluster size - Fix in block-copy code Alberto Garcia (1): qcow2: Fix preallocation on images with unaligned sizes Eric Blake (1): qed: Simplify backing reads Max Reitz (5

[PULL 03/31] iotests: Make _filter_img_create more active

2020-07-06 Thread Max Reitz
tps://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00339.html But that patch needs to come later so we can get away with changing as few reference outputs in this patch here as possible. Signed-off-by: Max Reitz Message-Id: <20200625125548.870061-2-mre...@redhat.com> Reviewed-b

Re: [PATCH v3 04/10] block/vpc: return ZERO block-status when appropriate

2020-07-06 Thread Max Reitz
On 28.05.20 11:43, Vladimir Sementsov-Ogievskiy wrote: > In case when get_image_offset() returns -1, we do zero out the > corresponding chunk of qiov. So, this should be reported as ZERO. > > Note that this changes visible output of "qemu-img map --output=json" > and "qemu-io -c map" commands.

Re: [PATCH v3 00/10] drop unallocated_blocks_are_zero

2020-07-03 Thread Max Reitz
On 28.05.20 11:43, Vladimir Sementsov-Ogievskiy wrote: > This is first step to block-status refactoring, and solves most simple > problem mentioned in my investigation of block-status described in > the thread "backing chain & block status & filters": >

Re: [PATCH v9 34/34] iotests: Add tests for qcow2 images with extended L2 entries

2020-07-03 Thread Max Reitz
On 03.07.20 15:06, Alberto Garcia wrote: > On Fri 03 Jul 2020 11:49:14 AM CEST, Max Reitz wrote: [...] >>> +expected_bitmap=0 >>> +for bit in $expected_alloc; do >>> +expected_bitmap=$(($expected_bitmap | (1 << $bit))) >>> +

Re: [PATCH 00/19] block: LUKS encryption slot management + iotest tweaks

2020-07-03 Thread Max Reitz
On 25.06.20 14:55, Max Reitz wrote: > Hi, > > This series collects all patches from Maxim’s series “LUKS: encryption > slot management using amend interface”, plus iotest patches that seem > useful to have before it. (And then I modified some of Maxim’s patches > to take

Re: [PULL v2 0/2] Block patches

2020-07-03 Thread Max Reitz
On 03.07.20 12:47, Peter Maydell wrote: > On Fri, 3 Jul 2020 at 10:52, Max Reitz wrote: >> >> On 24.06.20 10:13, Max Reitz wrote: >>> The following changes since commit d88d5a3806d78dcfca648c62dae9d88d3e803bd2: >>> >>> Merge remote-tracking branch

Re: [PATCH v2 01/25] iotests: Fix 051 output after qdev_init_nofail() removal

2020-07-03 Thread Max Reitz
On 03.07.20 12:02, Alex Bennée wrote: > > Max Reitz writes: > >> On 03.07.20 10:07, Kevin Wolf wrote: >>> Am 24.06.2020 um 16:04 hat Alex Bennée geschrieben: >>>> From: Philippe Mathieu-Daudé >>>> >>>> Commit 96927c744 replaced qdev_

Re: [PATCH v2 01/25] iotests: Fix 051 output after qdev_init_nofail() removal

2020-07-03 Thread Max Reitz
On 03.07.20 10:07, Kevin Wolf wrote: > Am 24.06.2020 um 16:04 hat Alex Bennée geschrieben: >> From: Philippe Mathieu-Daudé >> >> Commit 96927c744 replaced qdev_init_nofail() call by >> isa_realize_and_unref() which has a different error >> message. Update the test output accordingly. >> >> Gitlab

Re: [PULL v2 0/2] Block patches

2020-07-03 Thread Max Reitz
On 24.06.20 10:13, Max Reitz wrote: > The following changes since commit d88d5a3806d78dcfca648c62dae9d88d3e803bd2: > > Merge remote-tracking branch > 'remotes/philmd-gitlab/tags/renesas-hw-20200622' into staging (2020-06-23 > 13:55:52 +0100) > > are available

Re: [PATCH v9 34/34] iotests: Add tests for qcow2 images with extended L2 entries

2020-07-03 Thread Max Reitz
On 28.06.20 13:02, Alberto Garcia wrote: > Signed-off-by: Alberto Garcia > --- > tests/qemu-iotests/271 | 901 + > tests/qemu-iotests/271.out | 724 + > tests/qemu-iotests/group | 1 + > 3 files changed, 1626 insertions(+) >

Re: [PATCH v9 33/34] qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters

2020-07-03 Thread Max Reitz
are no plans to implement this at the > moment. > > Signed-off-by: Alberto Garcia > Reviewed-by: Eric Blake > --- > block/qcow2-cluster.c | 8 +++- > tests/qemu-iotests/061 | 6 ++ > tests/qemu-iotests/061.out | 5 + > 3 files changed, 18 insertions(+)

Re: [PATCH v9 32/34] qcow2: Allow preallocation and backing files if extended_l2 is set

2020-07-03 Thread Max Reitz
tests/206.out | 2 +- > 2 files changed, 5 insertions(+), 4 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v9 30/34] qcow2: Add prealloc field to QCowL2Meta

2020-07-03 Thread Max Reitz
On 03.07.20 01:05, Alberto Garcia wrote: > On Thu 02 Jul 2020 05:09:47 PM CEST, Max Reitz wrote: >>> Without a backing file, there is no read required - writing to an >>> unallocated subcluster within a preallocated cluster merely has to >>> provide zeros to the res

Re: [PATCH v9 28/34] qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()

2020-07-03 Thread Max Reitz
On 03.07.20 00:40, Alberto Garcia wrote: > On Thu 02 Jul 2020 04:28:57 PM CEST, Max Reitz wrote: >>> +/* For full clusters use zero_in_l2_slice() instead */ >>> +assert(nb_subclusters > 0 && nb_subclusters < >>> s->subclusters_per_clu

Re: [PATCH v9 14/34] qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()

2020-07-03 Thread Max Reitz
On 03.07.20 00:00, Alberto Garcia wrote: > On Thu 02 Jul 2020 11:57:46 AM CEST, Max Reitz wrote: >>> The reason why we would want to check it is, of course, because that >>> bit does have a meaning in regular L2 entries. >>> >>> But that bit is ignored in ima

Re: [PATCH v9 31/34] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2020-07-02 Thread Max Reitz
/common.filter | 1 + > 23 files changed, 272 insertions(+), 142 deletions(-) Looks OK. Reviewed-by: Max Reitz [...] > diff --git a/block/qcow2.c b/block/qcow2.c > index 003f166024..37bfae823c 100644 > --- a/block/qcow2.c > +++ b/bl

Re: [PATCH v9 30/34] qcow2: Add prealloc field to QCowL2Meta

2020-07-02 Thread Max Reitz
On 02.07.20 16:58, Eric Blake wrote: > On 7/2/20 9:50 AM, Max Reitz wrote: >> On 28.06.20 13:02, Alberto Garcia wrote: >>> This field allows us to indicate that the L2 metadata update does not >>> come from a write request with actual data but from

Re: [PATCH v9 30/34] qcow2: Add prealloc field to QCowL2Meta

2020-07-02 Thread Max Reitz
On 28.06.20 13:02, Alberto Garcia wrote: > This field allows us to indicate that the L2 metadata update does not > come from a write request with actual data but from a preallocation > request. > > For traditional images this does not make any difference, but for > images with extended L2 entries

Re: [PATCH v9 28/34] qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()

2020-07-02 Thread Max Reitz
On 28.06.20 13:02, Alberto Garcia wrote: > This works now at the subcluster level and pwrite_zeroes_alignment is > updated accordingly. > > qcow2_cluster_zeroize() is turned into qcow2_subcluster_zeroize() with > the following changes: > >- The request can now be subcluster-aligned. > >

Re: [PATCH v9 25/34] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()

2020-07-02 Thread Max Reitz
e > --- > block/qcow2-cluster.c | 18 ++ > 1 file changed, 18 insertions(+) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v9 24/34] qcow2: Add subcluster support to check_refcounts_l2()

2020-07-02 Thread Max Reitz
+++- > tests/qemu-iotests/060.out | 2 +- > 2 files changed, 12 insertions(+), 6 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v9 23/34] qcow2: Add subcluster support to discard_in_l2_slice()

2020-07-02 Thread Max Reitz
gt; Signed-off-by: Alberto Garcia > Reviewed-by: Eric Blake > --- > block/qcow2-cluster.c | 52 +++ > 1 file changed, 23 insertions(+), 29 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v9 22/34] qcow2: Add subcluster support to zero_in_l2_slice()

2020-07-02 Thread Max Reitz
ng the L2 entry and also > when deciding that an existing entry does not need to be updated. > > Signed-off-by: Alberto Garcia > Reviewed-by: Eric Blake > --- > block/qcow2-cluster.c | 36 +++- > 1 file changed, 19 insertions(+), 17

Re: [PATCH v9 21/34] qcow2: Add subcluster support to qcow2_get_host_offset()

2020-07-02 Thread Max Reitz
On 28.06.20 13:02, Alberto Garcia wrote: > The logic of this function remains pretty much the same, except that > it uses count_contiguous_subclusters(), which combines the logic of > count_contiguous_clusters() / count_contiguous_clusters_unallocated() > and checks individual subclusters. > >

Re: [PATCH v9 20/34] qcow2: Add subcluster support to calculate_l2_meta()

2020-07-02 Thread Max Reitz
le_dependencies() to > guarantee that no two requests try to write on the same cluster. > > Signed-off-by: Alberto Garcia > Reviewed-by: Eric Blake > --- > block/qcow2-cluster.c | 163 +++++++++- > 1 file changed, 131 insertions(+), 32 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v9 14/34] qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()

2020-07-02 Thread Max Reitz
On 01.07.20 18:26, Alberto Garcia wrote: > On Wed 01 Jul 2020 02:52:14 PM CEST, Max Reitz wrote: >>> if (l2_entry & QCOW_OFLAG_COMPRESSED) { >>> return QCOW2_CLUSTER_COMPRESSED; >>> -} else if (l2_entry & QCOW_OFLAG_ZERO) { >>&

Re: [PATCH 2/4] migration: Add block-bitmap-mapping parameter

2020-07-02 Thread Max Reitz
On 02.07.20 11:19, Vladimir Sementsov-Ogievskiy wrote: > 02.07.2020 11:09, Max Reitz wrote: >> On 01.07.20 16:34, Vladimir Sementsov-Ogievskiy wrote: >>> 30.06.2020 11:45, Max Reitz wrote: >>>> This migration parameter allows mapping block node names an

Re: [PATCH 1/4] migration: Prevent memleak by ...params_test_apply

2020-07-02 Thread Max Reitz
On 01.07.20 16:38, Eric Blake wrote: > On 6/30/20 3:45 AM, Max Reitz wrote: >> The created structure is not really a proper QAPI object, so we cannot >> and will not free its members.  Strings therein should therefore not be >> duplicated, or we will leak them. > &g

Re: [PATCH 2/4] migration: Add block-bitmap-mapping parameter

2020-07-02 Thread Max Reitz
On 01.07.20 16:34, Vladimir Sementsov-Ogievskiy wrote: > 30.06.2020 11:45, Max Reitz wrote: >> This migration parameter allows mapping block node names and bitmap >> names to aliases for the purpose of block dirty bitmap migration. >> >> This way, management tools can us

Re: [PATCH v9 16/34] qcow2: Add qcow2_cluster_is_allocated()

2020-07-01 Thread Max Reitz
insertions(+) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v9 15/34] qcow2: Add qcow2_get_subcluster_range_type()

2020-07-01 Thread Max Reitz
t; --- > block/qcow2-cluster.c | 51 +++ > 1 file changed, 51 insertions(+) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v9 14/34] qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()

2020-07-01 Thread Max Reitz
On 28.06.20 13:02, Alberto Garcia wrote: > This patch adds QCow2SubclusterType, which is the subcluster-level > version of QCow2ClusterType. All QCOW2_SUBCLUSTER_* values have the > the same meaning as their QCOW2_CLUSTER_* equivalents (when they > exist). See below for details and caveats. > >

Re: [PATCH v9 13/34] qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()

2020-07-01 Thread Max Reitz
d has no meaning. > > Signed-off-by: Alberto Garcia > Reviewed-by: Eric Blake > --- > block/qcow2.h | 21 + > 1 file changed, 21 insertions(+) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v9 11/34] qcow2: Add offset_into_subcluster() and size_to_subclusters()

2020-07-01 Thread Max Reitz
On 28.06.20 13:02, Alberto Garcia wrote: > Like offset_into_cluster() and size_to_clusters(), but for > subclusters. > > Signed-off-by: Alberto Garcia > Reviewed-by: Eric Blake > --- > block/qcow2.h | 10 ++ > 1 file changed, 10 insertions(+) Reviewed-by: Max

Re: [PATCH 2/4] migration: Add block-bitmap-mapping parameter

2020-07-01 Thread Max Reitz
On 30.06.20 12:51, Dr. David Alan Gilbert wrote: > * Max Reitz (mre...@redhat.com) wrote: >> This migration parameter allows mapping block node names and bitmap >> names to aliases for the purpose of block dirty bitmap migration. >> >> This way, management tools can us

Re: [PATCH v9 05/34] qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied()

2020-06-30 Thread Max Reitz
t; --- > block/qcow2-cluster.c | 256 +++--- > 1 file changed, 141 insertions(+), 115 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v9 02/34] qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()

2020-06-30 Thread Max Reitz
On 28.06.20 13:02, Alberto Garcia wrote: > qcow2_get_cluster_offset() takes an (unaligned) guest offset and > returns the (aligned) offset of the corresponding cluster in the qcow2 > image. > > In practice none of the callers need to know where the cluster starts > so this patch makes the

Re: [PATCH v2] qcow2: Fix preallocation on images with unaligned sizes

2020-06-30 Thread Max Reitz
On 17.06.20 16:00, Alberto Garcia wrote: > When resizing an image with qcow2_co_truncate() using the falloc or > full preallocation modes the code assumes that both the old and new > sizes are cluster-aligned. > > There are two problems with this: > > 1) The calculation of how many clusters

Re: [PATCH 15/19] iotests: qemu-img tests for luks key management

2020-06-30 Thread Max Reitz
On 29.06.20 14:05, Maxim Levitsky wrote: > On Thu, 2020-06-25 at 14:55 +0200, Max Reitz wrote: >> From: Maxim Levitsky >> >> This commit adds two tests, which test the new amend interface >> of both luks raw images and qcow2 luks encrypted images. >> >> Sig

Re: [PATCH 06/19] iotests: Check whether luks works

2020-06-30 Thread Max Reitz
On 29.06.20 14:03, Maxim Levitsky wrote: > On Thu, 2020-06-25 at 14:55 +0200, Max Reitz wrote: >> Whenever running an iotest for the luks format, we should check whether >> luks actually really works. >> >> Tests that try to create luks-encrypted qcow2 images should do

Re: [PATCH 05/19] iotests.py: Add (verify|has)_working_luks()

2020-06-30 Thread Max Reitz
On 29.06.20 12:12, Maxim Levitsky wrote: > On Thu, 2020-06-25 at 14:55 +0200, Max Reitz wrote: >> Similar to _require_working_luks for bash tests, these functions can be >> used to check whether our luks driver can actually create images. >> >> Signed-off-by: Max R

[PATCH 4/4] iotests: Test node/bitmap aliases during migration

2020-06-30 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/300 | 487 + tests/qemu-iotests/300.out | 5 + tests/qemu-iotests/group | 1 + 3 files changed, 493 insertions(+) create mode 100755 tests/qemu-iotests/300 create mode 100644 tests/qemu-iotests/300

[PATCH 0/4] migration: Add block-bitmap-mapping parameter

2020-06-30 Thread Max Reitz
command - Added patch 1 for good measure Max Reitz (4): migration: Prevent memleak by ...params_test_apply migration: Add block-bitmap-mapping parameter iotests.py: Add wait_for_runstate() iotests: Test node/bitmap aliases during migration qapi/migration.json| 83

[PATCH 1/4] migration: Prevent memleak by ...params_test_apply

2020-06-30 Thread Max Reitz
The created structure is not really a proper QAPI object, so we cannot and will not free its members. Strings therein should therefore not be duplicated, or we will leak them. Signed-off-by: Max Reitz --- migration/migration.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff

[PATCH 2/4] migration: Add block-bitmap-mapping parameter

2020-06-30 Thread Max Reitz
(on the source, the destination, or even both with arbitrary aliases in the migration stream). Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz --- qapi/migration.json| 83 +++- migration/migration.h | 3 + migration/block-dirty-bitmap.c | 372

[PATCH 3/4] iotests.py: Add wait_for_runstate()

2020-06-30 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 4 1 file changed, 4 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index ef739dd1e3..32f262db5a 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -799,6

[PATCH] iotests.py: Do not wait() before communicate()

2020-06-30 Thread Max Reitz
subprocess.Popen() in qemu_nbd_early_pipe(), because stderr is passed on to the child process, so if we do not drop this parameter, communicate() will hang (because the pipe is not closed). Signed-off-by: Max Reitz --- I hit this at some point with some test when writing my dirty bitmap migration

[PATCH v7 45/47] iotests: Add filter mirror test cases

2020-06-25 Thread Max Reitz
-by: Max Reitz --- tests/qemu-iotests/041 | 146 - tests/qemu-iotests/041.out | 4 +- 2 files changed, 147 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 index b843f88a66..588bb76626 100755 --- a/tests/qemu-iotests/041

[PATCH v7 37/47] qemu-img: Use child access functions

2020-06-25 Thread Max Reitz
This changes iotest 204's output, because blkdebug on top of a COW node used to make qemu-img map disregard the rest of the backing chain (the backing chain was broken by the filter). With this patch, the allocation in the base image is reported correctly. Signed-off-by: Max Reitz --- qemu

[PATCH v7 36/47] nbd: Use CAF when looking for dirty bitmap

2020-06-25 Thread Max Reitz
When looking for a dirty bitmap to share, we should handle filters by just including them in the search (so they do not break backing chains). Signed-off-by: Max Reitz --- nbd/server.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/nbd/server.c b/nbd/server.c index

[PATCH v7 30/47] block: Report data child for query-blockstats

2020-06-25 Thread Max Reitz
It makes no sense to report the block stats of a purely metadata-storing child in query-blockstats. So if the primary child does not have any data, try to find a unique data-storing child. Signed-off-by: Max Reitz --- block/qapi.c | 31 +-- 1 file changed, 29

[PATCH v7 28/47] block/null: Implement bdrv_get_allocated_file_size

2020-06-25 Thread Max Reitz
It is trivial, so we might as well do it. Signed-off-by: Max Reitz --- block/null.c | 7 +++ tests/qemu-iotests/153.out | 2 +- tests/qemu-iotests/184.out | 6 -- 3 files changed, 12 insertions(+), 3 deletions(-) diff --git a/block/null.c b/block/null.c index 15e1d56746

[PATCH v7 47/47] iotests: Test committing to overridden backing

2020-06-25 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/040 | 61 ++ tests/qemu-iotests/040.out | 4 +-- 2 files changed, 63 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index e7fa244738..dfd46ddcbe 100755

[PATCH v7 27/47] blkverify: Use bdrv_sum_allocated_file_size()

2020-06-25 Thread Max Reitz
blkverify is a filter, so bdrv_get_allocated_file_size()'s default implementation will return only the size of its filtered child. However, because both of its children are disk images, it makes more sense to sum both of their allocated sizes. Signed-off-by: Max Reitz --- block/blkverify.c | 1

[PATCH v7 44/47] iotests: Add filter commit test cases

2020-06-25 Thread Max Reitz
This patch adds some tests on how commit copes with filter nodes. Signed-off-by: Max Reitz --- tests/qemu-iotests/040 | 177 + tests/qemu-iotests/040.out | 4 +- 2 files changed, 179 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/040 b

[PATCH v7 46/47] iotests: Add test for commit in sub directory

2020-06-25 Thread Max Reitz
Add a test for committing an overlay in a sub directory to one of the images in its backing chain, using both relative and absolute filenames. Signed-off-by: Max Reitz --- tests/qemu-iotests/020 | 44 ++ tests/qemu-iotests/020.out | 10 + 2 files

[PATCH v7 43/47] iotests: Let complete_and_wait() work with commit

2020-06-25 Thread Max Reitz
complete_and_wait() and wait_ready() currently only work for mirror jobs. Let them work for active commit jobs, too. Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests

[PATCH v7 42/47] iotests: Test that qcow2's data-file is flushed

2020-06-25 Thread Max Reitz
Flushing a qcow2 node must lead to the data-file node being flushed as well. Signed-off-by: Max Reitz --- tests/qemu-iotests/244 | 49 ++ tests/qemu-iotests/244.out | 7 ++ 2 files changed, 56 insertions(+) diff --git a/tests/qemu-iotests/244 b

[PATCH v7 40/47] block: Inline bdrv_co_block_status_from_*()

2020-06-25 Thread Max Reitz
With bdrv_filter_bs(), we can easily handle this default filter behavior in bdrv_co_block_status(). blkdebug wants to have an additional assertion, so it keeps its own implementation, except bdrv_co_block_status_from_file() needs to be inlined there. Suggested-by: Eric Blake Signed-off-by: Max

[PATCH v7 41/47] block: Leave BDS.backing_file constant

2020-06-25 Thread Max Reitz
n reopening a node with neither a default nor a current backing file even if it used to have a backing node at some point. Signed-off-by: Max Reitz --- include/block/block_int.h | 21 - block.c| 35 +++ block/qapi.c

[PATCH v7 38/47] block: Drop backing_bs()

2020-06-25 Thread Max Reitz
We want to make it explicit where bs->backing is used, and we have done so. The old role of backing_bs() is now effectively taken by bdrv_cow_bs(). Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- include/block/block_int.h | 5 - 1 file changed, 5 deletions(-) d

[PATCH v7 39/47] blockdev: Fix active commit choice

2020-06-25 Thread Max Reitz
We have to perform an active commit whenever the top node has a parent that has taken the WRITE permission on it. Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- blockdev.c | 24 +--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git

[PATCH v7 32/47] block-copy: Use CAF to find sync=top base

2020-06-25 Thread Max Reitz
Signed-off-by: Max Reitz --- block/block-copy.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index f7428a7c08..5e80569bb8 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -437,8 +437,8 @@ static int

[PATCH v7 35/47] commit: Deal with filters

2020-06-25 Thread Max Reitz
This includes some permission limiting (for example, we only need to take the RESIZE permission if the base is smaller than the top). Signed-off-by: Max Reitz --- block/block-backend.c | 9 +++- block/commit.c | 96 +- block/monitor

[PATCH v7 33/47] mirror: Deal with filters

2020-06-25 Thread Max Reitz
y refers to. Signed-off-by: Max Reitz --- qapi/block-core.json | 6 ++- block/mirror.c | 118 +-- blockdev.c | 36 + 3 files changed, 121 insertions(+), 39 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.j

[PATCH v7 22/47] block: Use CAF in bdrv_co_rw_vmstate()

2020-06-25 Thread Max Reitz
If a node whose driver does not provide VM state functions has a metadata child, the VM state should probably go there; if it is a filter, the VM state should probably go there. It follows that we should generally go down to the primary child. Signed-off-by: Max Reitz Reviewed-by: Vladimir

[PATCH v7 31/47] block: Use child access functions for QAPI queries

2020-06-25 Thread Max Reitz
e now appears as a backing child. Signed-off-by: Max Reitz --- block/qapi.c | 33 - tests/qemu-iotests/184.out | 8 +++- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/block/qapi.c b/block/qapi.c index c57b42d86d..2628323b63 100644 --- a

[PATCH v7 21/47] block: Use CAFs in bdrv_refresh_filename()

2020-06-25 Thread Max Reitz
bdrv_refresh_filename() and the kind of related bdrv_dirname() should look to the primary child when they wish to copy the underlying file's filename. Signed-off-by: Max Reitz --- block.c | 29 + 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/block.c

[PATCH v7 29/47] blockdev: Use CAF in external_snapshot_prepare()

2020-06-25 Thread Max Reitz
This allows us to differentiate between filters and nodes with COW backing files: Filters cannot be used as overlays at all (for this function). Signed-off-by: Max Reitz --- blockdev.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index

[PATCH v7 20/47] block: Iterate over children in refresh_limits

2020-06-25 Thread Max Reitz
Instead of looking at just bs->file and bs->backing, we should look at all children that could end up receiving forwarded requests. Signed-off-by: Max Reitz --- block/io.c | 32 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/block/io.c b

[PATCH v7 16/47] block: Use bdrv_cow_child() in bdrv_co_truncate()

2020-06-25 Thread Max Reitz
The condition modified here is not about potentially filtered children, but only about COW sources (i.e. traditional backing files). Signed-off-by: Max Reitz --- block/io.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/block/io.c b/block/io.c index dc9891d6ce

[PATCH v7 26/47] block: Improve get_allocated_file_size's default

2020-06-25 Thread Max Reitz
(2) Filters need not have bs->file, so we should actually go to the filtered child instead of hard-coding bs->file. And we can make the whole default implementation more idiomatic by using the three generic functions added by the previous patch. Signed-off-by: Max Reitz --- blo

[PATCH v7 24/47] block: Use CAFs for debug breakpoints

2020-06-25 Thread Max Reitz
When looking for a blkdebug node (which implements debug breakpoints), use bdrv_primary_bs() to iterate through the graph, because that is where a blkdebug node would be. Signed-off-by: Max Reitz --- block.c | 16 +++- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git

[PATCH v7 34/47] backup: Deal with filters

2020-06-25 Thread Max Reitz
Signed-off-by: Max Reitz --- block/backup-top.c | 2 +- block/backup.c | 9 + blockdev.c | 19 +++ 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/block/backup-top.c b/block/backup-top.c index f304df8f26..89bd3937d0 100644 --- a/block/backup

[PATCH v7 25/47] block: Def. impl.s for get_allocated_file_size

2020-06-25 Thread Max Reitz
be implemented generically, so this patch adds such generic implementations for drivers to use. Signed-off-by: Max Reitz --- include/block/block_int.h | 5 block.c | 51 +++ 2 files changed, 56 insertions(+) diff --git a/include/block/block_int.h b

[PATCH v7 14/47] stream: Deal with filters

2020-06-25 Thread Max Reitz
bottom COW node). Signed-off-by: Max Reitz --- qapi/block-core.json | 4 +++ block/stream.c | 63 blockdev.c | 4 ++- 3 files changed, 53 insertions(+), 18 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index

[PATCH v7 12/47] block: Use bdrv_filter_(bs|child) where obvious

2020-06-25 Thread Max Reitz
Places that use patterns like if (bs->drv->is_filter && bs->file) { ... something about bs->file->bs ... } should be BlockDriverState *filtered = bdrv_filter_bs(bs); if (filtered) { ... something about @filtered ... } instead. S

[PATCH v7 19/47] vmdk: Drop vmdk_co_flush()

2020-06-25 Thread Max Reitz
Before HEAD^, we needed this because bdrv_co_flush() by itself would only flush bs->file. With HEAD^, bdrv_co_flush() will flush all children on which a WRITE or WRITE_UNCHANGED permission has been taken. Thus, vmdk no longer needs to do it itself. Signed-off-by: Max Reitz --- block/vmd

[PATCH v7 23/47] block/snapshot: Fix fallback

2020-06-25 Thread Max Reitz
ile and bs->backing, because bdrv_snapshot_goto() has to modify the child link (notably, set it to NULL). Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz --- block/snapshot.c | 104 +-- 1 file changed, 83 insertions(+), 21 del

[PATCH v7 09/47] copy-on-read: Support compressed writes

2020-06-25 Thread Max Reitz
Signed-off-by: Max Reitz --- block/copy-on-read.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index a6e3c74a68..a6a864f147 100644 --- a/block/copy-on-read.c +++ b/block/copy-on-read.c @@ -107,6 +107,16 @@ static int coroutine_fn

[PATCH v7 18/47] block: Flush all children in generic code

2020-06-25 Thread Max Reitz
images with an external data file, as they so far did not flush that data_file node. In any case, the BLKDBG_EVENT() should be emitted on the primary child, because that is where a blkdebug node would be if there is any. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz --- bloc

[PATCH v7 17/47] block: Re-evaluate backing file handling in reopen

2020-06-25 Thread Max Reitz
rectly use bs->backing because these functions are about the "backing" child (which may or may not be the COW backing file). Signed-off-by: Max Reitz --- block.c | 46 ++ 1 file changed, 38 insertions(+), 8 deletions(-) diff --git a/

[PATCH v7 11/47] backup-top: Support compressed writes

2020-06-25 Thread Max Reitz
Signed-off-by: Max Reitz --- block/backup-top.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/block/backup-top.c b/block/backup-top.c index af2f20f346..f304df8f26 100644 --- a/block/backup-top.c +++ b/block/backup-top.c @@ -99,6 +99,15 @@ static coroutine_fn int

[PATCH v7 15/47] block: Use CAFs when working with backing chains

2020-06-25 Thread Max Reitz
that node and @bs). Signed-off-by: Max Reitz --- block.c | 41 + 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/block.c b/block.c index a44af9c3c1..712230ef5c 100644 --- a/block.c +++ b/block.c @@ -4724,7 +4724,8 @@ int bdrv_change_backing_file

[PATCH v7 07/47] block: Add bdrv_supports_compressed_writes()

2020-06-25 Thread Max Reitz
ressed writes. This function looks down the filter chain to see whether there is a non-filter that can actually convert the compressed writes into compressed data (and thus normal writes). Signed-off-by: Max Reitz --- include/block/block.h | 1 + block.c | 23

[PATCH v7 13/47] block: Use CAFs in block status functions

2020-06-25 Thread Max Reitz
Use the child access functions in the block status inquiry functions as appropriate. Signed-off-by: Max Reitz --- block/io.c | 19 ++- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/block/io.c b/block/io.c index 385176b331..dc9891d6ce 100644 --- a/block/io.c

[PATCH v7 00/47] block: Deal with filters

2020-06-25 Thread Max Reitz
rectory' 047/47:[0008] [FC] 'iotests: Test committing to overridden backing' Max Reitz (47): block: Add child access functions block: Add chain helper functions block: bdrv_cow_child() for bdrv_has_zero_init() block: bdrv_set_backing_hd() is about bs->backing block: Include fil

[PATCH v7 05/47] block: Include filters when freezing backing chain

2020-06-25 Thread Max Reitz
do so. Signed-off-by: Max Reitz --- block.c | 60 + 1 file changed, 39 insertions(+), 21 deletions(-) diff --git a/block.c b/block.c index d139ffb57d..b59bd776cd 100644 --- a/block.c +++ b/block.c @@ -2595,12 +2595,15 @@ static void

[PATCH v7 10/47] mirror-top: Support compressed writes

2020-06-25 Thread Max Reitz
Signed-off-by: Max Reitz --- block/mirror.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/block/mirror.c b/block/mirror.c index e8e8844afc..469acf4600 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1480,6 +1480,15 @@ static int coroutine_fn bdrv_mirror_top_pdiscard

[PATCH v7 03/47] block: bdrv_cow_child() for bdrv_has_zero_init()

2020-06-25 Thread Max Reitz
bdrv_has_zero_init() and the related bdrv_unallocated_blocks_are_zero() should use bdrv_cow_child() if they want to check whether the given BDS has a COW backing file. Signed-off-by: Max Reitz --- block.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block.c b/block.c

<    9   10   11   12   13   14   15   16   17   18   >