Re: [PATCH v4 18/30] qcow2: Add subcluster support to qcow2_get_host_offset()

2020-04-08 Thread Alberto Garcia
On Wed 08 Apr 2020 02:49:14 PM CEST, Max Reitz wrote: > (Oops, totally missed the L1 entry out of bounds / L1 entry empty part > in v3.) Yeah, and you can mix values between different enum types in C quite easily without the compiler producing a warning. Berto

Re: [PATCH v4 02/30] qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()

2020-04-08 Thread Alberto Garcia
On Wed 08 Apr 2020 12:51:24 PM CEST, Max Reitz wrote: >> -if (has_data_file(bs) && *cluster_offset != offset - >> offset_in_cluster) >> +if (has_data_file(bs) && *host_offset != offset - offset_in_cluster) >> { > > (1) The { should be moved to the preceding line; > > (2) I

Re: [PATCH v2] qcow2: Check request size in qcow2_co_pwritev_compressed_part()

2020-04-06 Thread Alberto Garcia
I forgot to add the "for-5.0" tag in the subject, do I need to resend the patch? Berto

[PATCH v2] qcow2: Check request size in qcow2_co_pwritev_compressed_part()

2020-04-06 Thread Alberto Garcia
sion introduced in 0d483dce38 Signed-off-by: Alberto Garcia --- block/qcow2.c | 5 + 1 file changed, 5 insertions(+) diff --git a/block/qcow2.c b/block/qcow2.c index 2bb536b014..587cf51948 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4349,6 +4349,11 @@ qcow2_co_pwritev_compressed_p

[PATCH for-5.0] qcow2: Check request size in qcow2_co_pwritev_compressed_part()

2020-04-03 Thread Alberto Garcia
ow2.c:4257: qcow2_co_pwritev_compressed_task: Assertion `bytes == s->cluster_size || (bytes < s->cluster_size && (offset + bytes == bs->total_sectors << BDRV_SECTOR_BITS))' failed. Aborted This patch fixes a regression introduced in 0d483dce38 Signed-o

[PATCH v5] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-31 Thread Alberto Garcia
used by qcow2_snapshot_create() to discard the clusters used by the VM state. In this case there's no risk of exposing stale data to the guest and we really want that the clusters are always discarded. Signed-off-by: Alberto Garcia --- v5: - Fix iotests 046 and 177 with compat=0.10 [Max] v4: - S

Re: [PATCH v4] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-31 Thread Alberto Garcia
On Tue 31 Mar 2020 10:57:18 AM CEST, Max Reitz wrote: > I’ll have to dequeue it again, because it breaks iotests 046 and 177 > (both of which already have special handling for v2-specific discard; > but it needs to be adjusted now that the discard operation no longer > reveals the backing file cont

Re: [PATCH v3] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-27 Thread Alberto Garcia
On Fri 27 Mar 2020 07:57:40 PM CET, Eric Blake wrote: >> +/* If the image does not support QCOW_OFLAG_ZERO then discarding >> + * clusters could expose stale data from the backing file. */ >> +if (s->qcow_version < 3 && bs->backing) { >> +return -ENOTSUP; >> +} > > Hmm. Shou

[PATCH v4] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-27 Thread Alberto Garcia
used by qcow2_snapshot_create() to discard the clusters used by the VM state. In this case there's no risk of exposing stale data to the guest and we really want that the clusters are always discarded. Signed-off-by: Alberto Garcia --- v4: - Show output of qemu-img map when there's no backi

Re: [PATCH v3] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-27 Thread Alberto Garcia
On Fri 27 Mar 2020 07:13:04 PM CET, Eric Blake wrote: >> +for qcow2_compat in 0.10 1.1; do >> +echo "# Create an image with compat=$qcow2_compat without a backing >> file" >> +_make_test_img -o "compat=$qcow2_compat" 128k >> + >> +echo "# Fill all clusters with data and then discard th

[PATCH v3] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-27 Thread Alberto Garcia
used by qcow2_snapshot_create() to discard the clusters used by the VM state. In this case there's no risk of exposing stale data to the guest and we really want that the clusters are always discarded. Signed-off-by: Alberto Garcia --- v3: - Rebase and change iotest number - Show output of qemu-img

Re: [PATCH v2] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-27 Thread Alberto Garcia
On Fri 27 Mar 2020 03:14:42 PM CET, Kevin Wolf wrote: >> +echo "# Create a backing image and fill it with data" >> +BACKING_IMG="$TEST_IMG.base" >> +TEST_IMG="$BACKING_IMG" _make_test_img 128k >> +$QEMU_IO -c 'write -P 0xff 0 128k' "$BACKING_IMG" | _filter_qemu_io >> + >> +for qcow2_compat in 0.10

Re: [PATCH] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-24 Thread Alberto Garcia
On Tue 24 Mar 2020 03:46:07 PM CET, Eric Blake wrote: >> -$QEMU_IO -c 'discard 0k 64k' "$TEST_IMG" | _filter_qemu_io >> +poke_file "$TEST_IMG" '262144' "\x00\x00\x00\x00\x00\x00\x00\x00" # 0x4 >> - L2 entry >> +poke_file "$TEST_IMG" '131082' "\x00\x00" # 0x2000a - Refcount entry > > Instead of

[PATCH v2] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-24 Thread Alberto Garcia
used by qcow2_snapshot_create() to discard the clusters used by the VM state. In this case there's no risk of exposing stale data to the guest and we really want that the clusters are always discarded. Signed-off-by: Alberto Garcia --- v2: - Don't create the image with compat=0.10 in iote

[PATCH] qcow2: Forbid discard in qcow2 v2 images with backing files

2020-03-23 Thread Alberto Garcia
used by qcow2_snapshot_create() to discard the clusters used by the VM state. In this case there's no risk of exposing stale data to the guest and we really want that the clusters are always discarded. Signed-off-by: Alberto Garcia --- block/qcow2.c | 6 +++ tests/qemu-iotests

Re: [PATCH v8 3/4] qcow2: add zstd cluster compression

2020-03-23 Thread Alberto Garcia
On Mon 23 Mar 2020 11:20:42 AM CET, Denis Plotnikov wrote: >> But consider corrupted image: it may contain any data. And we should >> not crash because of it. So, we should return error here. > If the image is corrupted we can't continue anyway. If we return -EIO > on this condition, we need to do

Re: discard and v2 qcow2 images

2020-03-20 Thread Alberto Garcia
On Fri 20 Mar 2020 08:35:44 PM CET, Eric Blake wrote: >> This flag is however only supported when qcow_version >= 3. In older >> images the cluster is simply deallocated, exposing any possible >> previous data from the backing file. > > Discard is advisory, and has no requirements that discarded d

discard and v2 qcow2 images

2020-03-20 Thread Alberto Garcia
Hi, when full_discard is false in discard_in_l2_slice() then the selected cluster should be deallocated and it should read back as zeroes. This is done by clearing the cluster offset field and setting OFLAG_ZERO in the L2 entry. This flag is however only supported when qcow_version >= 3. In older

Re: [PATCH-for-5.0] block: Assert BlockDriver::format_name is not NULL

2020-03-19 Thread Alberto Garcia
Mansour Ahmadi > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Alberto Garcia Berto

Re: [PATCH 0/2] Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()

2020-03-17 Thread Alberto Garcia
On Thu 27 Feb 2020 07:34:24 PM CET, Alberto Garcia wrote: > Hi, > > this is something I did while working on the subcluster series but > it's independent from it so I thought to send it already. I included this patches in v4 of the main series so you can ignore this thread. Berto

[PATCH v4 30/30] iotests: Add tests for qcow2 images with extended L2 entries

2020-03-17 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- tests/qemu-iotests/271 | 359 + tests/qemu-iotests/271.out | 244 + tests/qemu-iotests/group | 1 + 3 files changed, 604 insertions(+) create mode 100755 tests/qemu-iotests/271 create mode

[PATCH v4 17/30] qcow2: Add subcluster support to calculate_l2_meta()

2020-03-17 Thread Alberto Garcia
e all-zeroes bit set then we need copy-on-write on subcluster #3. - If we are overwriting an old cluster and subcluster #3 was allocated then there is no need to copy-on-write. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-cluster.c

[PATCH v4 00/30] Add subcluster allocation to qcow2

2020-03-17 Thread Alberto Garcia
qcow2: Restrict qcow2_co_pwrite_zeroes() to full clusters only' 027/30:[down] 'qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters' 028/30:[0019] [FC] 'qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit' 029/30:[] [--

[PATCH v4 03/30] qcow2: Add calculate_l2_meta()

2020-03-17 Thread Alberto Garcia
handle_alloc() creates a QCowL2Meta structure in order to update the image metadata and perform the necessary copy-on-write operations. This patch moves that code to a separate function so it can be used from other places. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2

[PATCH v4 24/30] qcow2: Clear the L2 bitmap when allocating a compressed cluster

2020-03-17 Thread Alberto Garcia
Compressed clusters always have the bitmap part of the extended L2 entry set to 0. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index dfd8b66958..1f471db98c

[PATCH v4 27/30] qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters

2020-03-17 Thread Alberto Garcia
This function is only used by qcow2_expand_zero_clusters() to downgrade a qcow2 image to a previous version. It is however not possible to downgrade an image with extended L2 entries because older versions of qcow2 do not have this feature. Signed-off-by: Alberto Garcia --- block/qcow2

[PATCH v4 19/30] qcow2: Add subcluster support to zero_in_l2_slice()

2020-03-17 Thread Alberto Garcia
Setting the QCOW_OFLAG_ZERO bit of the L2 entry is forbidden if an image has subclusters. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 14 ++ 1 file changed, 10 insertions(+), 4

[PATCH v4 14/30] qcow2: Add cluster type parameter to qcow2_get_host_offset()

2020-03-17 Thread Alberto Garcia
nd in order to detect errors more easily. This patch makes qcow2_get_host_offset() return 0 on success and puts the returned cluster type in a separate parameter. There are no semantic changes. Signed-off-by: Alberto Garcia --- block/qcow2.h | 3 ++- block/qcow2-cluster.c

[PATCH v4 09/30] qcow2: Add subcluster-related fields to BDRVQcow2State

2020-03-17 Thread Alberto Garcia
as if they had exactly one, with subcluster_size = cluster_size. Signed-off-by: Alberto Garcia --- block/qcow2.h | 5 + block/qcow2.c | 5 + 2 files changed, 10 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index 55298750bd..3052b14dc0 100644 --- a/block/qcow2.h +++ b/block/q

[PATCH v4 25/30] qcow2: Add subcluster support to handle_alloc_space()

2020-03-17 Thread Alberto Garcia
zeroes the other subclusters if we can guarantee that we are not overwriting existing data. However this would waste more disk space, so we should first evaluate if it's really worth doing. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.c | 9 + 1 file chang

[PATCH v4 28/30] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2020-03-17 Thread Alberto Garcia
Now that the implementation of subclusters is complete we can finally add the necessary options to create and read images with this feature, which we call "extended L2 entries". Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- qapi/block-core.json | 7 +++ blo

[PATCH v4 23/30] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()

2020-03-17 Thread Alberto Garcia
The L2 bitmap needs to be updated after each write to indicate what new subclusters are now allocated. This needs to happen even if the cluster was already allocated and the L2 entry was otherwise valid. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 17

[PATCH v4 21/30] qcow2: Add subcluster support to check_refcounts_l2()

2020-03-17 Thread Alberto Garcia
Setting the QCOW_OFLAG_ZERO bit of the L2 entry is forbidden if an image has subclusters. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-refcount.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletion

[PATCH v4 13/30] qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()

2020-03-17 Thread Alberto Garcia
patch. Signed-off-by: Alberto Garcia --- block/qcow2.h | 120 ++ 1 file changed, 120 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index 9611efbc52..52865787ee 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -80,6 +80,15 @@ #define

[PATCH v4 04/30] qcow2: Split cluster_needs_cow() out of count_cow_clusters()

2020-03-17 Thread Alberto Garcia
We are going to need it in other places. Signed-off-by: Alberto Garcia Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 34 +++--- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/block/qcow2-cluster.c b

[PATCH v4 01/30] qcow2: Make Qcow2AioTask store the full host offset

2020-03-17 Thread Alberto Garcia
this patch but it is documented now. Signed-off-by: Alberto Garcia --- block/qcow2.c | 68 +-- 1 file changed, 33 insertions(+), 35 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index d44b45633d..a00b0c8e45 100644 --- a/block/qcow2.c

[PATCH v4 26/30] qcow2: Restrict qcow2_co_pwrite_zeroes() to full clusters only

2020-03-17 Thread Alberto Garcia
Ideally it should be possible to zero individual subclusters using this function, but this is currently not implemented. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c

[PATCH v4 06/30] qcow2: Add get_l2_entry() and set_l2_entry()

2020-03-17 Thread Alberto Garcia
get_l2_entry() and set_l2_entry(). Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2.h | 12 block/qcow2-cluster.c | 63 ++ block/qcow2-refcount.c | 17 ++-- 3 files changed, 54 insertions(+), 38

[PATCH v4 07/30] qcow2: Document the Extended L2 Entries feature

2020-03-17 Thread Alberto Garcia
-off-by: Alberto Garcia --- docs/interop/qcow2.txt | 68 -- docs/qcow2-cache.txt | 19 +++- 2 files changed, 83 insertions(+), 4 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 5597e24474..2e8cad38c4 100644 --- a/docs

[PATCH v4 11/30] qcow2: Add l2_entry_size()

2020-03-17 Thread Alberto Garcia
function returns the proper value for the image. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 9 + block/qcow2-cluster.c | 12 ++-- block/qcow2-refcount.c | 14 -- block/qcow2.c | 8 4 files changed, 27 insertions

[PATCH v4 18/30] qcow2: Add subcluster support to qcow2_get_host_offset()

2020-03-17 Thread Alberto Garcia
The logic of this function remains pretty much the same, except that it uses count_contiguous_subclusters(), which combines the logic of count_contiguous_clusters() / count_contiguous_clusters_unallocated() and checks individual subclusters. Signed-off-by: Alberto Garcia --- block/qcow2.h

[PATCH v4 20/30] qcow2: Add subcluster support to discard_in_l2_slice()

2020-03-17 Thread Alberto Garcia
OFLAG_ZERO bit of the L2 entry is forbidden if an image has extended L2 entries. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 18 +++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/blo

[PATCH v4 08/30] qcow2: Add dummy has_subclusters() function

2020-03-17 Thread Alberto Garcia
value. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index 7754d9bd02..55298750bd 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -495,6 +495,12 @@ typedef

[PATCH v4 22/30] qcow2: Fix offset calculation in handle_dependencies()

2020-03-17 Thread Alberto Garcia
l2meta_cow_start() and l2meta_cow_end() are not necessarily cluster-aligned if the image has subclusters, so update the calculation of old_start and old_end to guarantee that no two requests try to write on the same cluster. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2

[PATCH v4 12/30] qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()

2020-03-17 Thread Alberto Garcia
the get/set_l2_bitmap() functions that are used to access the bitmaps. For convenience we allow calling get_l2_bitmap() on images without subclusters, although the caller does not need and should ignore the returned value. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h

[PATCH v4 05/30] qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied()

2020-03-17 Thread Alberto Garcia
t for subclusters. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 230 -- 1 file changed, 130 insertions(+), 100 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index e251d

[PATCH v4 16/30] qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC

2020-03-17 Thread Alberto Garcia
When dealing with subcluster types there is a new value called QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC that has no equivalent in QCow2ClusterType. This patch handles that value in all places where subcluster types are processed. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2

[PATCH v4 15/30] qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*

2020-03-17 Thread Alberto Garcia
QCow2ClusterType with their QCow2SubclusterType equivalents. This patch only changes the data types, there are no semantic changes. Signed-off-by: Alberto Garcia --- block/qcow2.h | 2 +- block/qcow2-cluster.c | 10 +++ block/qcow2.c | 70 ++- 3

[PATCH v4 02/30] qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()

2020-03-17 Thread Alberto Garcia
-off-by: Alberto Garcia --- block/qcow2.h | 4 ++-- block/qcow2-cluster.c | 38 ++ block/qcow2.c | 24 +++- 3 files changed, 31 insertions(+), 35 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 0942126232

[PATCH v4 29/30] qcow2: Add subcluster support to qcow2_measure()

2020-03-17 Thread Alberto Garcia
Extended L2 entries are bigger than normal L2 entries so this has an impact on the amount of metadata needed for a qcow2 file. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.c | 19 --- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/block

[PATCH v4 10/30] qcow2: Add offset_to_sc_index()

2020-03-17 Thread Alberto Garcia
For a given offset, return the subcluster number within its cluster (i.e. with 32 subclusters per cluster it returns a number between 0 and 31). Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 5 + 1 file changed, 5 insertions(+) diff --git a/block/qcow2.h b/block

Re: [PATCH] qcow2: remove QCowL2Meta parameter from handle_copied

2020-03-10 Thread Alberto Garcia
On Tue 10 Mar 2020 03:04:47 AM CET, John Snow wrote: >> static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, >> -uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m) >> +uint64_t *host_offset, uint64_t *bytes) >> { >> BDRVQcow2State *s = bs->opaque; >> int l2

Re: [PATCH v2 2/2] block: bdrv_reopen() with backing file in different AioContext

2020-03-06 Thread Alberto Garcia
e AioContext of the other node. > > Signed-off-by: Kevin Wolf > Tested-by: Peter Krempa Reviewed-by: Alberto Garcia Berto

Re: [PATCH v8 01/10] error: auto propagated local_err

2020-03-06 Thread Alberto Garcia
On Fri 06 Mar 2020 06:15:27 AM CET, Vladimir Sementsov-Ogievskiy wrote: Sorry I just gave a quick look at these patches and noticed this: > + * Function may use error system to return errors. In this case function > + * defines Error **errp parameter, which should be the last one (except for > +

Re: [PATCH v5 4/5] qcow2: add zstd cluster compression

2020-03-06 Thread Alberto Garcia
n -ENOMEM if the destination buffer is not big enough. But not of my comments is so important, so whether you decide to make those changes or not, Reviewed-by: Alberto Garcia Berto

Re: [PATCH v5 3/5] qcow2: rework the cluster compression routine

2020-03-06 Thread Alberto Garcia
ementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [PATCH v5 2/5] qcow2: introduce compression type feature

2020-03-05 Thread Alberto Garcia
On Wed 04 Mar 2020 02:35:35 PM CET, Denis Plotnikov wrote: > +## > +# @Qcow2CompressionType: I realized that we have a bit of a mix in the way we write this type of identifiers between QCow2FooBar (capital C) and Qcow2FooBar ... what's the recommended one? > @@ -146,6 +146,12 @@ typedef struct QC

Re: [PATCH 2/2] block: bdrv_reopen() with backing file in different AioContext

2020-03-05 Thread Alberto Garcia
On Thu 27 Feb 2020 07:18:04 PM CET, Kevin Wolf wrote: > /* > - * TODO: before removing the x- prefix from x-blockdev-reopen we > - * should move the new backing file into the right AioContext > - * instead of returning an error. > + * Check AioContext compatibility so that the

Re: [PATCH] block/qcow2-threads: fix qcow2_decompress

2020-03-02 Thread Alberto Garcia
rno return code, to be closer to > qcow2_compress API (and usual expectations). > > Revert condition in if to be more positive. Drop dead initialization of > ret. > > Cc: qemu-sta...@nongnu.org # v4.0 > Fixes: 341926ab83e2b > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

[PATCH 1/2] qcow2: Make Qcow2AioTask store the full host offset

2020-02-27 Thread Alberto Garcia
this patch but it is documented now. Signed-off-by: Alberto Garcia --- block/qcow2.c | 68 +-- 1 file changed, 33 insertions(+), 35 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 3c754f616b..b2c7c8255e 100644 --- a/block/qcow2.c

[PATCH 2/2] qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()

2020-02-27 Thread Alberto Garcia
-off-by: Alberto Garcia --- block/qcow2.h | 4 ++-- block/qcow2-cluster.c | 38 ++ block/qcow2.c | 24 +++- 3 files changed, 31 insertions(+), 35 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 0942126232

[PATCH 0/2] Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()

2020-02-27 Thread Alberto Garcia
thing like that but I don't think it's worth the effort here, so I just documented it. Berto Alberto Garcia (2): qcow2: Make Qcow2AioTask store the full host offset qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset() block/qcow2.h | 4 +-- block/

Re: [RFC PATCH v3 19/27] qcow2: Add subcluster support to expand_zero_clusters_in_l1()

2020-02-26 Thread Alberto Garcia
On Fri 21 Feb 2020 03:57:27 PM CET, Max Reitz wrote: > As noted in v2, this function is only called when downgrading qcow2 > images to v2. It kind of made sense to just call set_l2_bitmap() in > v2, but now with the if () conditional... I suppose it may make more > sense to assert that the image

Re: [RFC PATCH v3 05/27] qcow2: Document the Extended L2 Entries feature

2020-02-26 Thread Alberto Garcia
On Thu 20 Feb 2020 04:16:12 PM CET, Eric Blake wrote: +In these images standard data clusters are divided into 32 subclusters of the +same size. They are contiguous and start from the beginning of the cluster. +Subclusters can be allocated independently and the L2 entry

Re: [RFC PATCH v3 00/27] Add subcluster allocation to qcow2

2020-02-22 Thread Alberto Garcia
On Fri 21 Feb 2020 06:10:52 PM CET, Max Reitz wrote: > So now I wonder on what your plans are after this series. Apart from some fixes here and there, there are some things that I would live to solve: - I'm not 100% happy with the separation between QCow2ClusterType and QCow2SubclusterType. Th

Re: [RFC PATCH v3 12/27] qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*

2020-02-21 Thread Alberto Garcia
On Fri 21 Feb 2020 12:35:55 PM CET, Max Reitz wrote: >> @@ -2223,22 +2227,23 @@ static coroutine_fn int >> qcow2_co_preadv_part(BlockDriverState *bs, >> } >> >> qemu_co_mutex_lock(&s->lock); >> -ret = qcow2_get_cluster_offset(bs, offset, &cur_bytes, >> &cluster_offset)

Re: [RFC PATCH v3 10/27] qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()

2020-02-21 Thread Alberto Garcia
On Thu 20 Feb 2020 05:27:28 PM CET, Max Reitz wrote: >> +static inline uint64_t get_l2_bitmap(BDRVQcow2State *s, uint64_t *l2_slice, >> + int idx) >> +{ >> +if (has_subclusters(s)) { >> +idx *= l2_entry_size(s) / sizeof(uint64_t); >> +return b

Re: [RFC PATCH v3 07/27] qcow2: Add subcluster-related fields to BDRVQcow2State

2020-02-21 Thread Alberto Garcia
On Thu 20 Feb 2020 05:48:25 PM CET, Eric Blake wrote: >>> The qcow2 spec changes earlier in the series made it sound like your >>> choices are exactly 1 or 32, >> +#define QCOW_MAX_SUBCLUSTERS_PER_CLUSTER 32 + >>> >>> ...but this name sounds like other values (2, 4, 8, 16) might be >>> p

Re: [RFC PATCH v3 07/27] qcow2: Add subcluster-related fields to BDRVQcow2State

2020-02-20 Thread Alberto Garcia
On Thu 20 Feb 2020 04:28:07 PM CET, Eric Blake wrote: >> Images without subclusters are treated as if they had exactly one, >> with subcluster_size = cluster_size. > > The qcow2 spec changes earlier in the series made it sound like your > choices are exactly 1 or 32, >> +#define QCOW_MAX_SUBCLUSTE

Re: [RFC PATCH v3 05/27] qcow2: Document the Extended L2 Entries feature

2020-02-20 Thread Alberto Garcia
On Thu 20 Feb 2020 03:33:57 PM CET, Eric Blake wrote: >> Given an offset into the virtual disk, the offset into the image file can >> be >> obtained as follows: >> >> -l2_entries = (cluster_size / sizeof(uint64_t)) >> +l2_entries = (cluster_size / sizeof(uint64_t))[*] >>

Re: [RFC PATCH v3 04/27] qcow2: Add get_l2_entry() and set_l2_entry()

2020-02-20 Thread Alberto Garcia
On Thu 20 Feb 2020 04:22:24 PM CET, Eric Blake wrote: >> +++ b/block/qcow2.h > > scripts/git.orderfile can be used to hoist this part of the patch to > the front of the message (as it is more valuable to review first). I didn't know that git had this feature, thanks for the tip! Berto

Re: [RFC PATCH v3 05/27] qcow2: Document the Extended L2 Entries feature

2020-02-20 Thread Alberto Garcia
On Thu 20 Feb 2020 05:02:22 PM CET, Eric Blake wrote: >>> +Bits are assigned starting from the most significant >>> one. >>> +(i.e. bit x is used for subcluster 31 - x) >> >> I still prefer it the other way round, both personally (e.g. it’s the >> C orderin

Re: [RFC PATCH v3 05/27] qcow2: Document the Extended L2 Entries feature

2020-02-20 Thread Alberto Garcia
On Thu 20 Feb 2020 03:28:17 PM CET, Eric Blake wrote: >> +An image uses Extended L2 Entries if bit 3 is set on the >> incompatible_features >> +field of the header. >> + >> +In these images standard data clusters are divided into 32 subclusters of >> the >> +same size. They are contiguous and sta

Re: [RFC PATCH v3 25/27] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2020-02-20 Thread Alberto Garcia
On Thu 20 Feb 2020 03:12:19 PM CET, Eric Blake wrote: >> +{ >> +.type = QCOW2_FEAT_TYPE_INCOMPATIBLE, >> +.bit = QCOW2_INCOMPAT_EXTL2_BITNR, >> +.name = "extended L2 entries", >> +}, > > I'd sort this to be grouped with the ot

[PATCH] qcow2: Fix alignment checks in encrypted images

2020-02-13 Thread Alberto Garcia
ks in qcow2_co_preadv_encrypted() are also unnecessary because they are repeated immediately afterwards in qcow2_co_encdec(). Signed-off-by: Alberto Garcia --- block/qcow2-threads.c | 12 block/qcow2.c | 2 -- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/

Re: [PATCH v2 03/33] block: Add BdrvChildRole

2020-02-11 Thread Alberto Garcia
On Tue 04 Feb 2020 06:08:18 PM CET, Max Reitz wrote: > +/* Child to COW from (backing child) */ > +BDRV_CHILD_COW = (1 << 3), Without the comment in brackets I'm not sure that I would have understood that this is meant for backing files. This is the "child that contains the data

Re: [PATCH v2 02/33] block: Rename BdrvChildRole to BdrvChildClass

2020-02-11 Thread Alberto Garcia
parents can have custom callbacks even when the child fulfills a > standard role. > > Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia Berto

Re: [PATCH v2 01/33] block: Add BlockDriver.is_format

2020-02-11 Thread Alberto Garcia
That means we need something on which to distinguish > format drivers from others, and hence this flag. > > Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia Berto

Re: [PATCH 07/17] gluster: Drop useless has_zero_init callback

2020-02-10 Thread Alberto Garcia
On Fri 31 Jan 2020 06:44:26 PM CET, Eric Blake wrote: > block.c already defaults to 0 if we don't provide a callback; there's > no need to write a callback that always fails. > > Signed-off-by: Eric Blake Reviewed-by: Alberto Garcia Berto

Re: [PATCH 06/17] block: Improve bdrv_has_zero_init_truncate with backing file

2020-02-10 Thread Alberto Garcia
acking layer into the resized region. But as this > setup is rare, it penalizes the more common case of a backing layer > smaller than the current layer. > > Signed-off-by: Eric Blake Reviewed-by: Alberto Garcia Berto

Re: [PATCH 05/17] block: Don't advertise zero_init_truncate with encryption

2020-02-10 Thread Alberto Garcia
optimize zero handling of encrypted images thus > avoids the possibility of that being a security concern. > > Signed-off-by: Eric Blake Reviewed-by: Alberto Garcia Berto

Re: [PATCH 01/17] qcow2: Comment typo fixes

2020-02-09 Thread Alberto Garcia
On Fri 31 Jan 2020 06:44:20 PM CET, Eric Blake wrote: > Various trivial typos noticed while working on this file. > > Signed-off-by: Eric Blake Reviewed-by: Alberto Garcia Berto

Re: [PATCH 03/17] qcow2: Avoid feature name extension on small cluster size

2020-02-09 Thread Alberto Garcia
her than auditing for other tests that are not > robust to alternative cluster sizes). > > Signed-off-by: Eric Blake Reviewed-by: Alberto Garcia Berto

Re: [PATCH v11 2/2] docs: qcow2: introduce compression type feature

2020-01-31 Thread Alberto Garcia
On Fri 31 Jan 2020 07:15:33 PM CET, Vladimir Sementsov-Ogievskiy wrote: > I'm OK with it too, as well as I'm OK with the stricter variant, when > we don't allow incompatible images with zlib set. I don't see any > serious difference. I also think both options are fine. Berto

Re: [PATCH v11 2/2] docs: qcow2: introduce compression type feature

2020-01-31 Thread Alberto Garcia
On Fri 31 Jan 2020 03:46:12 PM CET, Eric Blake wrote: >> +If the incompatible bit "Compression type" is set: the >> field >> +must be present and non-zero (which means non-zlib >> +compression type). Otherwise, this field must not be >>

Re: [PATCH v11 1/2] docs: improve qcow2 spec about extending image header

2020-01-31 Thread Alberto Garcia
ro or not. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia Berto

Re: [PATCH] iotests: Remove the superfluous 2nd check for the availability of quorum

2020-01-29 Thread Alberto Garcia
-by: Thomas Huth Reviewed-by: Alberto Garcia Berto

[PATCH v3 5/5] qcow2: Use BDRV_SECTOR_SIZE instead of the hardcoded value

2020-01-18 Thread Alberto Garcia
This replaces all remaining instances in the qcow2 code. Signed-off-by: Alberto Garcia --- block/qcow2.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index a6b0d4ee1d..6cc13e388c 100644 --- a/block/qcow2.c +++ b/block/qcow2.c

[PATCH v3 1/5] qcow2: Don't round the L1 table allocation up to the sector size

2020-01-18 Thread Alberto Garcia
The L1 table is read from disk using the byte-based bdrv_pread() and is never accessed beyond its last element, so there's no need to allocate more memory than that. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 5 ++--- block/qcow2-refcount.c | 2 +-

[PATCH v3 3/5] qcow2: Use bs->bl.request_alignment when updating an L1 entry

2020-01-18 Thread Alberto Garcia
because then we could be overwriting data after the L1 table) Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 25 +++-- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 932fc48919..f1b5535b04 100644

[PATCH v3 2/5] qcow2: Tighten cluster_offset alignment assertions

2020-01-18 Thread Alberto Garcia
qcow2_alloc_cluster_offset() and qcow2_get_cluster_offset() always return offsets that are cluster-aligned so don't just check that they are sector-aligned. The check in qcow2_co_preadv_task() is also replaced by an assertion for the same reason. Signed-off-by: Alberto Garcia Reviewed-by

[PATCH v3 0/5] Misc BDRV_SECTOR_SIZE updates

2020-01-18 Thread Alberto Garcia
QEMU_IS_ALIGNED() macro instead of the modulus operator [Nir] - Tighten some assertions [Kevin] v1: https://lists.gnu.org/archive/html/qemu-block/2020-01/msg00139.html Alberto Garcia (5): qcow2: Don't round the L1 table allocation up to the sector size qcow2: Tighten cluster_offset alig

[PATCH v3 4/5] qcow2: Don't require aligned offsets in qcow2_co_copy_range_from()

2020-01-18 Thread Alberto Garcia
qemu-img's convert_co_copy_range() operates at the sector level and block_copy() operates at the cluster level so this condition is always true, but it is not necessary to restrict this here, so let's leave it to the driver implementation return an error if there is any. Signed-off-b

Re: [PATCH v2 4/4] qcow2: Use BDRV_SECTOR_SIZE instead of the hardcoded value

2020-01-18 Thread Alberto Garcia
On Tue 14 Jan 2020 03:15:48 PM CET, Max Reitz wrote: >> @@ -3836,7 +3837,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, >> case QCOW2_CLUSTER_NORMAL: >> child = s->data_file; >> copy_offset += offset_into_cluster(s, src_offset); >> -if ((copy_off

Re: [PATCH v2 4/4] qcow2: Use BDRV_SECTOR_SIZE instead of the hardcoded value

2020-01-17 Thread Alberto Garcia
On Fri 17 Jan 2020 12:51:04 PM CET, Kevin Wolf wrote: >> > Now, what's wrong about the logic to avoid the RMW is that it assumes >> > a fixed required alignment of 512. What it should do is looking at >> > bs->file->bl.request_alignment and rounding accordingly. >> >> Yes. > > Who'll write the pat

Re: [PATCH v2 4/4] qcow2: Use BDRV_SECTOR_SIZE instead of the hardcoded value

2020-01-16 Thread Alberto Garcia
On Tue 14 Jan 2020 03:15:48 PM CET, Max Reitz wrote: >> @@ -219,7 +219,7 @@ static int l2_load(BlockDriverState *bs, uint64_t offset, >> * Writes one sector of the L1 table to the disk (can't update single >> entries >> * and we really don't want bdrv_pread to perform a read-modify-write) >>

Re: [PATCH v2 1/4] qcow2: Require that the virtual size is a multiple of the sector size

2020-01-16 Thread Alberto Garcia
On Tue 14 Jan 2020 03:03:31 PM CET, Max Reitz wrote: >> 3. QEMU can open them but it gets the size wrong. > > Yes, but in a safe way. The user simply doesn’t get access to those > 13 bytes. Ok, I think I'll withdraw the patch then :-) Berto

Re: [PATCH v2] qcow2: Use a GString in report_unsupported_feature()

2020-01-16 Thread Alberto Garcia
On Thu 16 Jan 2020 01:19:34 PM CET, Max Reitz wrote: > On 15.01.20 14:56, Alberto Garcia wrote: >> This is a bit more efficient than having to allocate and free memory >> for each item. >> >> The default size (60) is enough for all the existing incompatible

[PATCH v2] qcow2: Use a GString in report_unsupported_feature()

2020-01-15 Thread Alberto Garcia
This is a bit more efficient than having to allocate and free memory for each item. The default size (60) is enough for all the existing incompatible features or the "Unknown incompatible feature" message. Suggested-by: Philippe Mathieu-Daudé Signed-off-by: Alberto Garcia Reviewe

Re: [PATCH] qcow2: Use a GString in report_unsupported_feature()

2020-01-15 Thread Alberto Garcia
On Tue 14 Jan 2020 07:08:04 PM CET, Alex Bennée wrote: >g_autoptr(GString) features = g_string_sized_new(60); > > will save you the clean-up later. Ok, will send v2 now. >> +if (features->len > 0) { >> +g_string_append(features, ", "); >> +

<    2   3   4   5   6   7   8   9   10   11   >