[PATCH] iotests: fix usage -machine accel= together with -accel option

Commit 6f6e1698a68ceb made these options incompatible, but it breaks iotests: -accel qtest comes from QEMUQtestMachine and -machine accel=qtest comes from QEMU_OPTIONS After this patch, -accel may still be duplicated, but this is less invasive. Also, fix extra comma in comment, added by the

[RFC PATCH v3 12/27] qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*

In order to support extended L2 entries some functions of the qcow2 driver need to start dealing with subclusters instead of clusters. qcow2_get_cluster_offset() is modified to return the subcluster type instead of the cluster type, and all callers are updated to replace all values of QCow2Cluster

[RFC PATCH v3 26/27] qcow2: Add subcluster support to qcow2_measure()

Extended L2 entries are bigger than normal L2 entries so this has an impact on the amount of metadata needed for a qcow2 file. Signed-off-by: Alberto Garcia --- block/qcow2.c | 19 --- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c in

[RFC PATCH v3 25/27] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

Now that the implementation of subclusters is complete we can finally add the necessary options to create and read images with this feature, which we call "extended L2 entries". Signed-off-by: Alberto Garcia --- block/qcow2.c| 65 ++-- block/qcow2.h

[RFC PATCH v3 23/27] qcow2: Add subcluster support to handle_alloc_space()

The bdrv_co_pwrite_zeroes() call here fills complete clusters with zeroes, but it can happen that some subclusters are not part of the write request or the copy-on-write. This patch makes sure that only the affected subclusters are overwritten. A potential improvement would be to also fill with ze

[RFC PATCH v3 21/27] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()

The L2 bitmap needs to be updated after each write to indicate what new subclusters are now allocated. This needs to happen even if the cluster was already allocated and the L2 entry was otherwise valid. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 17 + 1 file chan

[RFC PATCH v3 06/27] qcow2: Add dummy has_subclusters() function

This function will be used by the qcow2 code to check if an image has subclusters or not. At the moment this simply returns false. Once all patches needed for subcluster support are ready then QEMU will be able to create and read images with subclusters and this function will return the actual val

[RFC PATCH v3 16/27] qcow2: Add subcluster support to zero_in_l2_slice()

Setting the QCOW_OFLAG_ZERO bit of the L2 entry is forbidden if an image has subclusters. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/block/qcow

[RFC PATCH v3 10/27] qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()

Extended L2 entries are 128-bit wide: 64 bits for the entry itself and 64 bits for the subcluster allocation bitmap. In order to support them correctly get/set_l2_entry() need to be updated so they take the entry width into account in order to calculate the correct offset. This patch also adds th

[RFC PATCH v3 18/27] qcow2: Add subcluster support to check_refcounts_l2()

Setting the QCOW_OFLAG_ZERO bit of the L2 entry is forbidden if an image has subclusters. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia --- block/qcow2-refcount.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/block/qcow2-refc

[RFC PATCH v3 09/27] qcow2: Add l2_entry_size()

qcow2 images with subclusters have 128-bit L2 entries. The first 64 bits contain the same information as traditional images and the last 64 bits form a bitmap with the status of each individual subcluster. Because of that we cannot assume that L2 entries are sizeof(uint64_t) anymore. This function

[RFC PATCH v3 14/27] qcow2: Add subcluster support to calculate_l2_meta()

If an image has subclusters then there are more copy-on-write scenarios that we need to consider. Let's say we have a write request from the middle of subcluster #3 until the end of the cluster: - If the cluster is new, then subclusters #0 to #3 from the old cluster must be copied into the

[RFC PATCH v3 05/27] qcow2: Document the Extended L2 Entries feature

Subcluster allocation in qcow2 is implemented by extending the existing L2 table entries and adding additional information to indicate the allocation status of each subcluster. This patch documents the changes to the qcow2 format and how they affect the calculation of the L2 cache size. Signed-of

[RFC PATCH v3 20/27] qcow2: Fix offset calculation in handle_dependencies()

l2meta_cow_start() and l2meta_cow_end() are not necessarily cluster-aligned if the image has subclusters, so update the calculation of old_start and old_end to guarantee that no two requests try to write on the same cluster. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 4 ++-- 1 fil

[RFC PATCH v3 15/27] qcow2: Add subcluster support to qcow2_get_cluster_offset()

The logic of this function remains pretty much the same, except that it uses count_contiguous_subclusters(), which combines the logic of count_contiguous_clusters() / count_contiguous_clusters_unallocated() and checks individual subclusters. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.

[RFC PATCH v3 03/27] qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied()

When writing to a qcow2 file there are two functions that take a virtual offset and return a host offset, possibly allocating new clusters if necessary: - handle_copied() looks for normal data clusters that are already allocated and have a reference count of 1. In those clusters we ca

[RFC PATCH v3 27/27] iotests: Add tests for qcow2 images with extended L2 entries

Signed-off-by: Alberto Garcia --- tests/qemu-iotests/271 | 256 + tests/qemu-iotests/271.out | 208 ++ tests/qemu-iotests/group | 1 + 3 files changed, 465 insertions(+) create mode 100755 tests/qemu-iotests/271 create mode

[RFC PATCH v3 04/27] qcow2: Add get_l2_entry() and set_l2_entry()

The size of an L2 entry is 64 bits, but if we want to have subclusters we need extended L2 entries. This means that we have to access L2 tables and slices differently depending on whether an image has extended L2 entries or not. This patch replaces all l2_slice[] accesses with calls to get_l2_entr

[RFC PATCH v3 00/27] Add subcluster allocation to qcow2

Hi, here's the new version of the patches to add subcluster allocation support to qcow2. Please refer to the cover letter of the first version for a full description of the patches: https://lists.gnu.org/archive/html/qemu-block/2019-10/msg00983.html This version fixes many of the problems hi

[RFC PATCH v3 19/27] qcow2: Add subcluster support to expand_zero_clusters_in_l1()

Two changes are needed in order to add subcluster support to this function: deallocated clusters must have their bitmaps cleared, and expanded clusters must have all the "subcluster allocated" bits set. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 6 ++ 1 file changed, 6 inserti

[RFC PATCH v3 02/27] qcow2: Split cluster_needs_cow() out of count_cow_clusters()

We are going to need it in other places. Signed-off-by: Alberto Garcia Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-cluster.c | 34 +++--- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index

[RFC PATCH v3 08/27] qcow2: Add offset_to_sc_index()

For a given offset, return the subcluster number within its cluster (i.e. with 32 subclusters per cluster it returns a number between 0 and 31). Signed-off-by: Alberto Garcia --- block/qcow2.h | 5 + 1 file changed, 5 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index 941330cfc9

[RFC PATCH v3 22/27] qcow2: Clear the L2 bitmap when allocating a compressed cluster

Compressed clusters always have the bitmap part of the extended L2 entry set to 0. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index ed291a4042..af0f01621c 100644 --- a/block/qcow2-cl

[RFC PATCH v3 13/27] qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC

When dealing with subcluster types there is a new value called QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC that has no equivalent in QCow2ClusterType. This patch handles that value in all places where subcluster types are processed. Signed-off-by: Alberto Garcia --- block/qcow2.c | 12 +--- 1 fi

[RFC PATCH v3 11/27] qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()

This patch adds QCow2SubclusterType, which is the subcluster-level version of QCow2ClusterType. All QCOW2_SUBCLUSTER_* values have the the same meaning as their QCOW2_CLUSTER_* equivalents (when they exist). See below for details and caveats. In images without extended L2 entries clusters are trea

[RFC PATCH v3 07/27] qcow2: Add subcluster-related fields to BDRVQcow2State

This patch adds the following new fields to BDRVQcow2State: - subclusters_per_cluster: Number of subclusters in a cluster - subcluster_size: The size of each subcluster, in bytes - subcluster_bits: No. of bits so 1 << subcluster_bits = subcluster_size Images without subclusters are treated as if

[RFC PATCH v3 01/27] qcow2: Add calculate_l2_meta()

handle_alloc() creates a QCowL2Meta structure in order to update the image metadata and perform the necessary copy-on-write operations. This patch moves that code to a separate function so it can be used from other places. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 77 +++

[RFC PATCH v3 24/27] qcow2: Restrict qcow2_co_pwrite_zeroes() to full clusters only

Ideally it should be possible to zero individual subclusters using this function, but this is currently not implemented. Signed-off-by: Alberto Garcia --- block/qcow2.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/block/qcow2.c b/block/qcow2.c index 242001afa2..0267722065 100644 ---

[RFC PATCH v3 17/27] qcow2: Add subcluster support to discard_in_l2_slice()

Setting the QCOW_OFLAG_ZERO bit of the L2 entry is forbidden if an image has subclusters. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/block/qcow2-cluster.c