Re: [PATCH] qcow2: Fix removal of list members from BDRVQcow2State.cluster_allocs

2020-09-03 Thread Max Reitz
wouldn’t allow -o refcount_bits=64. > + > +refcount_table_offset=$((0x400)) I would like to suggest $(peek_file_be "$TEST_IMG" 48 8), to set an example for future generations; but not strictly necessary, of course. O:) Anyway, at least with the _unsupported_imgopts line completed: Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [RFC qemu 0/6] mirror: implement incremental and bitmap modes

2020-09-03 Thread Max Reitz
On 03.09.20 14:38, Kevin Wolf wrote: > Am 03.09.2020 um 13:04 hat Max Reitz geschrieben: >> On 03.09.20 12:13, Fabian Grünbichler wrote: >>> On August 21, 2020 3:03 pm, Max Reitz wrote: >>>> On 18.02.20 11:07, Fabian Grünbichler wrote: >>> I am not sure ho

Re: [PATCH] iotests: Allow running from different directory

2020-09-03 Thread Max Reitz
On 02.09.20 13:03, Kevin Wolf wrote: > It is convenient to be able to edit the tests and run them without > changing the current working directory back and forth. Instead of > assuming that $PWD is the qemu-iotests build directory, derive the build > directory from the executed script. > > This al

Re: [RFC qemu 0/6] mirror: implement incremental and bitmap modes

2020-09-03 Thread Max Reitz
On 03.09.20 12:13, Fabian Grünbichler wrote: > On August 21, 2020 3:03 pm, Max Reitz wrote: >> On 18.02.20 11:07, Fabian Grünbichler wrote: >> >> [Sorry :/] > > same, I've been meaning to ping/pick this back up but other stuff got in > the way. so thanks fo

Re: [PATCH] ide:do nothing for identify cmd if no any device attached

2020-09-03 Thread Max Reitz
On 02.09.20 20:02, John Snow wrote: > (CC Max for block backend model confusion, see below) > > On 8/16/20 11:38 PM, zhaoxin\RockCuioc wrote: >> This patch is for avoiding win7 IDE driver polling 0x1f7 when >> no any device attached. During Win7 VM boot procedure, if use virtio for >> disk and the

Re: [PATCH 2/3] mirror: Freeze backing chain for sync=top

2020-09-02 Thread Max Reitz
On 02.09.20 18:48, Max Reitz wrote: > Reported-by: Kevin Wolf > Signed-off-by: Max Reitz > --- > block/mirror.c | 20 +--- > 1 file changed, 17 insertions(+), 3 deletions(-) > > diff --git a/block/mirror.c b/block/mirror.c > index 11ebffdf99..27422ab

[PATCH 3/3] iotests/041: x-blockdev-reopen during mirror

2020-09-02 Thread Max Reitz
Test what happens when you remove the backing file during a mirror with sync=top. Signed-off-by: Max Reitz --- tests/qemu-iotests/041 | 92 ++ tests/qemu-iotests/041.out | 4 +- 2 files changed, 94 insertions(+), 2 deletions(-) diff --git a/tests/qemu

[PATCH 0/3] mirror: Freeze backing chain for sync=top

2020-09-02 Thread Max Reitz
h I implemented here is fine until someone complains. Max Reitz (3): mirror: Set s->base_overlay only if s->base is set mirror: Freeze backing chain for sync=top iotests/041: x-blockdev-reopen during mirror block/mirror.c | 28 +--- tests/qemu-iotests/041

[PATCH 2/3] mirror: Freeze backing chain for sync=top

2020-09-02 Thread Max Reitz
Reported-by: Kevin Wolf Signed-off-by: Max Reitz --- block/mirror.c | 20 +--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 11ebffdf99..27422ab7a5 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -649,8 +649,8 @@ static

[PATCH 1/3] mirror: Set s->base_overlay only if s->base is set

2020-09-02 Thread Max Reitz
This way, sync=full will not need a reference to any node other than the source and the target. Signed-off-by: Max Reitz --- block/mirror.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 26acf4af6f..11ebffdf99 100644 --- a/block

Re: [PATCH v8 00/43] block: Deal with filters

2020-09-02 Thread Max Reitz
On 02.09.20 12:23, Kevin Wolf wrote: > Am 01.09.2020 um 16:33 hat Max Reitz geschrieben: >> v6: https://lists.nongnu.org/archive/html/qemu-devel/2019-08/msg01715.html >> v7: https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg01357.html >> >> Branch: https://

Re: [PATCH v3 1/7] block/quorum.c: stable children names

2020-09-02 Thread Max Reitz
On 04.08.20 12:46, Lukas Straub wrote: > If we remove the child with the highest index from the quorum, > decrement s->next_child_index. This way we get stable children > names as long as we only remove the last child. > > Signed-off-by: Lukas Straub > Fixes: https://bugs.launchpad.net/bugs/18812

Re: [PATCH v3 0/5] iotest.FilePath fixes and cleanups

2020-09-02 Thread Max Reitz
On 29.08.20 01:21, Nir Soffer wrote: > Fix some issues introduced when iotests.FilePaths was added and merge it back > into FilePath keeping the option to create multiple file names. Thanks, applied to my block branch: https://git.xanclic.moe/XanClic/qemu/commits/branch/block signature.asc Des

Re: [PATCH v8 29/43] mirror: Deal with filters

2020-09-02 Thread Max Reitz
On 02.09.20 10:53, Kevin Wolf wrote: > Am 01.09.2020 um 16:34 hat Max Reitz geschrieben: >> This includes some permission limiting (for example, we only need to >> take the RESIZE permission for active commits where the base is smaller >> than the top). >> >> bas

Re: [PATCH v5 00/10] preallocate filter

2020-09-01 Thread Max Reitz
On 27.08.20 23:08, Vladimir Sementsov-Ogievskiy wrote: > 21.08.2020 17:11, Vladimir Sementsov-Ogievskiy wrote: >> Hi all! >> >> Here is a filter, which does preallocation on write. >> >> In Virtuozzo we have to deal with some custom distributed storage >> solution, where allocation is relatively ex

[PATCH v8 36/43] block: Inline bdrv_co_block_status_from_*()

2020-09-01 Thread Max Reitz
With bdrv_filter_bs(), we can easily handle this default filter behavior in bdrv_co_block_status(). blkdebug wants to have an additional assertion, so it keeps its own implementation, except bdrv_co_block_status_from_file() needs to be inlined there. Suggested-by: Eric Blake Signed-off-by: Max

[PATCH v8 34/43] block: Drop backing_bs()

2020-09-01 Thread Max Reitz
We want to make it explicit where bs->backing is used, and we have done so. The old role of backing_bs() is now effectively taken by bdrv_cow_bs(). Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- include/bl

[PATCH v8 43/43] iotests: Test committing to overridden backing

2020-09-01 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- tests/qemu-iotests/040 | 61 ++ tests/qemu-iotests/040.out | 4 +-- 2 files changed, 63 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu

[PATCH v8 28/43] block-copy: Use CAF to find sync=top base

2020-09-01 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block/block-copy.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index a30b9097ef..cd9bc47c8f 100644 --- a/block/block-copy.c +++ b/block/block

[PATCH v8 38/43] iotests: Test that qcow2's data-file is flushed

2020-09-01 Thread Max Reitz
Flushing a qcow2 node must lead to the data-file node being flushed as well. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- tests/qemu-iotests/244 | 49 ++ tests/qemu-iotests/244.out | 7 ++ 2 files changed, 56

[PATCH v8 27/43] block: Use child access functions for QAPI queries

2020-09-01 Thread Max Reitz
appears as a backing child. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block/qapi.c | 33 - tests/qemu-iotests/184.out | 8 +++- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/block/q

[PATCH v8 37/43] block: Leave BDS.backing_{file,format} constant

2020-09-01 Thread Max Reitz
without a format layer, so ImageInfo.backing-filename-format used to report "file" for the base image's overlay after blockdev-snapshot. However, the image header never says "file" anywhere, so it now reports $IMGFMT. Signed-off-by: Max Reitz --- include/block/block_in

[PATCH v8 25/43] blockdev: Use CAF in external_snapshot_prepare()

2020-09-01 Thread Max Reitz
This allows us to differentiate between filters and nodes with COW backing files: Filters cannot be used as overlays at all (for this function). Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- blockdev.c | 7 ++- 1 file changed, 6 insertions(+), 1

[PATCH v8 33/43] qemu-img: Use child access functions

2020-09-01 Thread Max Reitz
This changes iotest 204's output, because blkdebug on top of a COW node used to make qemu-img map disregard the rest of the backing chain (the backing chain was broken by the filter). With this patch, the allocation in the base image is reported correctly. Signed-off-by: Max Reitz ---

[PATCH v8 22/43] block: Use CAFs for debug breakpoints

2020-09-01 Thread Max Reitz
When looking for a blkdebug node (which implements debug breakpoints), use bdrv_primary_bs() to iterate through the graph, because that is where a blkdebug node would be. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block.c | 16 +++- 1 file

[PATCH v8 30/43] backup: Deal with filters

2020-09-01 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf --- block/backup-top.c | 2 +- block/backup.c | 9 + blockdev.c | 19 +++ 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/block/backup-top.c b/block/backup-top.c index af2f20f346..430d1be068

[PATCH v8 21/43] block/snapshot: Fix fallback

2020-09-01 Thread Max Reitz
o bs->file and bs->backing, because bdrv_snapshot_goto() has to modify the child link (notably, set it to NULL). Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block/

[PATCH v8 42/43] iotests: Add test for commit in sub directory

2020-09-01 Thread Max Reitz
Add a test for committing an overlay in a sub directory to one of the images in its backing chain, using both relative and absolute filenames. Signed-off-by: Max Reitz --- tests/qemu-iotests/020 | 44 ++ tests/qemu-iotests/020.out | 10 + 2 files

[PATCH v8 26/43] block: Report data child for query-blockstats

2020-09-01 Thread Max Reitz
It makes no sense to report the block stats of a purely metadata-storing child in query-blockstats. So if the primary child does not have any data, try to find a unique data-storing child. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block/qapi.c | 31

[PATCH v8 40/43] iotests: Add filter commit test cases

2020-09-01 Thread Max Reitz
This patch adds some tests on how commit copes with filter nodes. Signed-off-by: Max Reitz --- tests/qemu-iotests/040 | 177 + tests/qemu-iotests/040.out | 4 +- 2 files changed, 179 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/040 b

[PATCH v8 41/43] iotests: Add filter mirror test cases

2020-09-01 Thread Max Reitz
: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- tests/qemu-iotests/041 | 146 - tests/qemu-iotests/041.out | 4 +- 2 files changed, 147 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 index

[PATCH v8 20/43] block: Use CAF in bdrv_co_rw_vmstate()

2020-09-01 Thread Max Reitz
If a node whose driver does not provide VM state functions has a metadata child, the VM state should probably go there; if it is a filter, the VM state should probably go there. It follows that we should generally go down to the primary child. Signed-off-by: Max Reitz Reviewed-by: Vladimir

[PATCH v8 39/43] iotests: Let complete_and_wait() work with commit

2020-09-01 Thread Max Reitz
complete_and_wait() and wait_ready() currently only work for mirror jobs. Let them work for active commit jobs, too. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- tests/qemu-iotests/iotests.py | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions

[PATCH v8 19/43] block: Use CAFs in bdrv_refresh_filename()

2020-09-01 Thread Max Reitz
bdrv_refresh_filename() and the kind of related bdrv_dirname() should look to the primary child when they wish to copy the underlying file's filename. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf --- block.c | 29 + 1 file changed, 21 insertions(+), 8 dele

[PATCH v8 24/43] block/null: Implement bdrv_get_allocated_file_size

2020-09-01 Thread Max Reitz
It is trivial, so we might as well do it. Remove _filter_actual_image_size from iotest 184, so we get to see the result in its reference output. Signed-off-by: Max Reitz --- block/null.c | 7 +++ tests/qemu-iotests/153.out | 2 +- tests/qemu-iotests/184 | 3 +-- tests

[PATCH v8 35/43] blockdev: Fix active commit choice

2020-09-01 Thread Max Reitz
oblem in practice, because this case was basically just broken until now. (Since this commit already touches block-commit's documentation, it also moves up the chunk explaining general block-commit behavior that for some reason was situated under @backing-file.) Signed-off-by: Max Reitz

[PATCH v8 32/43] nbd: Use CAF when looking for dirty bitmap

2020-09-01 Thread Max Reitz
When looking for a dirty bitmap to share, we should handle filters by just including them in the search (so they do not break backing chains). Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- nbd/server.c | 6 +++--- 1 file changed, 3 insertions(+), 3

[PATCH v8 31/43] commit: Deal with filters

2020-09-01 Thread Max Reitz
This includes some permission limiting (for example, we only need to take the RESIZE permission if the base is smaller than the top). Signed-off-by: Max Reitz --- block/block-backend.c | 7 ++- block/commit.c | 94 +- block/monitor/block

[PATCH v8 23/43] block: Improve get_allocated_file_size's default

2020-09-01 Thread Max Reitz
eturn -ENOTSUP, because the default implementation cannot make a guess (3) For other drivers: Sum all data-bearing children's sizes Signed-off-by: Max Reitz --- block.c | 42 +++--- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/block.c b/block

[PATCH v8 14/43] block: Use bdrv_cow_child() in bdrv_co_truncate()

2020-09-01 Thread Max Reitz
The condition modified here is not about potentially filtered children, but only about COW sources (i.e. traditional backing files). Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block/io.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions

[PATCH v8 16/43] block: Flush all children in generic code

2020-09-01 Thread Max Reitz
images with an external data file, as they so far did not flush that data_file node. In any case, the BLKDBG_EVENT() should be emitted on the primary child, because that is where a blkdebug node would be if there is any. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz Revie

[PATCH v8 29/43] mirror: Deal with filters

2020-09-01 Thread Max Reitz
x27;s block_status is the same as its child node, so if there are filters on base, bdrv_is_allocated_above() on base would return information including base. Use this opportunity to rename qmp_drive_mirror()'s "source" BDS to "target_backing_bs", because that is what it really refers

[PATCH v8 12/43] stream: Deal with filters

2020-09-01 Thread Max Reitz
bottom COW node). Signed-off-by: Max Reitz --- qapi/block-core.json | 13 ++--- block/stream.c | 63 blockdev.c | 4 ++- 3 files changed, 59 insertions(+), 21 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.

[PATCH v8 17/43] vmdk: Drop vmdk_co_flush()

2020-09-01 Thread Max Reitz
Before HEAD^, we needed this because bdrv_co_flush() by itself would only flush bs->file. With HEAD^, bdrv_co_flush() will flush all children on which a WRITE or WRITE_UNCHANGED permission has been taken. Thus, vmdk no longer needs to do it itself. Signed-off-by: Max Reitz Reviewed-by: Ke

[PATCH v8 18/43] block: Iterate over children in refresh_limits

2020-09-01 Thread Max Reitz
Instead of looking at just bs->file and bs->backing, we should look at all children that could end up receiving forwarded requests. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf --- block/io.c | 32 1 file changed, 16 insertions(+), 16 deletions(-)

[PATCH v8 15/43] block: Re-evaluate backing file handling in reopen

2020-09-01 Thread Max Reitz
to directly use bs->backing because these functions are about the "backing" child (which may or may not be the COW backing file). Signed-off-by: Max Reitz --- block.c | 49 - 1 file changed, 40 insertions(+), 9 deletions(-) diff --g

[PATCH v8 09/43] copy-on-read: Support compressed writes

2020-09-01 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block/copy-on-read.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index a6e3c74a68..a6a864f147 100644 --- a/block/copy-on-read.c +++ b/block/copy

[PATCH v8 07/43] block: Add bdrv_supports_compressed_writes()

2020-09-01 Thread Max Reitz
ressed writes. This function looks down the filter chain to see whether there is a non-filter that can actually convert the compressed writes into compressed data (and thus normal writes). Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- include/block/block.

[PATCH v8 13/43] block: Use CAFs when working with backing chains

2020-09-01 Thread Max Reitz
node and @bs). Signed-off-by: Max Reitz --- block.c | 44 ++-- 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/block.c b/block.c index 887c125400..96bf8672f1 100644 --- a/block.c +++ b/block.c @@ -4745,9 +4745,9 @@ int

[PATCH v8 05/43] block: Include filters when freezing backing chain

2020-09-01 Thread Max Reitz
do so. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block.c | 60 + 1 file changed, 39 insertions(+), 21 deletions(-) diff --git a/block.c b/block.c index e8d09d46ec..edef6273b8 100644 --- a/block.c

[PATCH v8 10/43] block: Use bdrv_filter_(bs|child) where obvious

2020-09-01 Thread Max Reitz
Places that use patterns like if (bs->drv->is_filter && bs->file) { ... something about bs->file->bs ... } should be BlockDriverState *filtered = bdrv_filter_bs(bs); if (filtered) { ... something about @filtered ... } instead. S

[PATCH v8 06/43] block: Drop bdrv_is_encrypted()

2020-09-01 Thread Max Reitz
Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- include/block/block.h | 1 - block.c | 8 block/qapi.c | 2 +- 3 files changed, 1 insertion(+), 10 deletions(-) diff --git a/include/bl

[PATCH v8 08/43] throttle: Support compressed writes

2020-09-01 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block/throttle.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/block/throttle.c b/block/throttle.c index 1c1ac57bee..b21ee42d98 100644 --- a/block

[PATCH v8 02/43] block: Add chain helper functions

2020-09-01 Thread Max Reitz
Add some helper functions for skipping filters in a chain of block nodes. Signed-off-by: Max Reitz --- include/block/block_int.h | 3 ++ block.c | 62 +++ 2 files changed, 65 insertions(+) diff --git a/include/block/block_int.h b/include

[PATCH v8 04/43] block: bdrv_set_backing_hd() is about bs->backing

2020-09-01 Thread Max Reitz
bdrv_set_backing_hd() is a function that explicitly cares about the bs->backing child. Highlight that in its description and use child_bs(bs->backing) instead of backing_bs(bs) to make it more obvious. Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by:

[PATCH v8 03/43] block: bdrv_cow_child() for bdrv_has_zero_init()

2020-09-01 Thread Max Reitz
bdrv_has_zero_init() should use bdrv_cow_child() if it wants to check whether the given BDS has a COW backing file. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block.c b/block.c

[PATCH v8 11/43] block: Use CAFs in block status functions

2020-09-01 Thread Max Reitz
Use the child access functions in the block status inquiry functions as appropriate. Signed-off-by: Max Reitz --- block/io.c | 19 ++- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/block/io.c b/block/io.c index 01e3477a77..4ee8fe5465 100644 --- a/block/io.c

[PATCH v8 01/43] block: Add child access functions

2020-09-01 Thread Max Reitz
r nodes in a meaningful way. Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- include/block/block_int.h | 44 +-- block.c | 91 +++ 2 files changed, 132 insertions(+), 3 deletions(-) diff --git a/i

[PATCH v8 00/43] block: Deal with filters

2020-09-01 Thread Max Reitz
[FC] 'block: Leave BDS.backing_file constant' 038/43:[----] [--] 'iotests: Test that qcow2's data-file is flushed' 039/43:[] [--] 'iotests: Let complete_and_wait() work with commit' 040/43:[0004] [FC] 'iotests: Add filter commit test cases' 041/43:[] [-

Re: [PATCH v5 07/10] block: introduce preallocate filter

2020-08-26 Thread Max Reitz
On 26.08.20 11:15, Vladimir Sementsov-Ogievskiy wrote: > 26.08.2020 11:52, Max Reitz wrote: >> On 26.08.20 08:49, Vladimir Sementsov-Ogievskiy wrote: >>> 25.08.2020 18:11, Max Reitz wrote: >>>> On 21.08.20 16:11, Vladimir Sementsov-Ogievskiy wrote: >>>>>

Re: [PATCH v5 07/10] block: introduce preallocate filter

2020-08-26 Thread Max Reitz
On 26.08.20 08:49, Vladimir Sementsov-Ogievskiy wrote: > 25.08.2020 18:11, Max Reitz wrote: >> On 21.08.20 16:11, Vladimir Sementsov-Ogievskiy wrote: >>> It's intended to be inserted between format and protocol nodes to >>> preallocate additional space (ex

Re: [PATCH v5 06/10] block: introduce BDRV_REQ_NO_WAIT flag

2020-08-26 Thread Max Reitz
On 26.08.20 08:26, Vladimir Sementsov-Ogievskiy wrote: > 25.08.2020 16:10, Max Reitz wrote: >> On 21.08.20 16:11, Vladimir Sementsov-Ogievskiy wrote: >>> Add flag to make serialising request no wait: if there are conflicting >>> requests, just return error immedi

[PULL v2 00/34] Block patches

2020-08-26 Thread Max Reitz
The following changes since commit 30aa19446d82358a30eac3b556b4d6641e00b7c1: Merge remote-tracking branch 'remotes/cschoenebeck/tags/pull-9p-20200812' into staging (2020-08-24 16:39:53 +0100) are available in the Git repository at: https://github.com/XanClic/qemu.git tags/pull-block-2020-08

Re: [PULL 00/34] Block patches

2020-08-25 Thread Max Reitz
On 25.08.20 16:22, Alberto Garcia wrote: > On Tue 25 Aug 2020 04:01:14 PM CEST, Peter Maydell > wrote: >> On Tue, 25 Aug 2020 at 09:33, Max Reitz wrote: >>> >>> The following changes since commit 30aa19446d82358a30eac3b556b4d6641e00b7c1: >>> >

Re: [PATCH v5 07/10] block: introduce preallocate filter

2020-08-25 Thread Max Reitz
On 21.08.20 16:11, Vladimir Sementsov-Ogievskiy wrote: > It's intended to be inserted between format and protocol nodes to > preallocate additional space (expanding protocol file) on writes > crossing EOF. It improves performance for file-systems with slow > allocation. > > Signed-off-by: Vladimir

Re: [PATCH v5 06/10] block: introduce BDRV_REQ_NO_WAIT flag

2020-08-25 Thread Max Reitz
On 21.08.20 16:11, Vladimir Sementsov-Ogievskiy wrote: > Add flag to make serialising request no wait: if there are conflicting > requests, just return error immediately. It's will be used in upcoming > preallocate filter. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > include/block/blo

Re: [PATCH v5 05/10] block: bdrv_mark_request_serialising: split non-waiting function

2020-08-25 Thread Max Reitz
called tracked_request_set_serialising(). > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > include/block/block_int.h | 3 ++- > block/file-posix.c| 2 +- > block/io.c| 35 +++-------- > 3 files changed, 26 insertions(+

Re: [PATCH v2 5/5] qemu-iotests: Simplify FilePath __init__

2020-08-25 Thread Max Reitz
On 21.08.20 01:54, Nir Soffer wrote: > Use list comprehension instead of append loop. > > Signed-off-by: Nir Soffer > --- > tests/qemu-iotests/iotests.py | 5 ++--- > 1 file changed, 2 insertions(+), 3 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v2 4/5] qemu-iotests: Merge FilePaths and FilePath

2020-08-25 Thread Max Reitz
| 2 +- > tests/qemu-iotests/257| 4 ++-- > tests/qemu-iotests/iotests.py | 23 +++ > 5 files changed, 16 insertions(+), 17 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v2 3/5] qemu-iotests: Support varargs syntax in FilePaths

2020-08-25 Thread Max Reitz
-- > tests/qemu-iotests/iotests.py | 6 +++--- > 3 files changed, 9 insertions(+), 11 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v2 2/5] qemu-iotests: Fix FilePaths docstring

2020-08-25 Thread Max Reitz
On 21.08.20 01:54, Nir Soffer wrote: > When this class was extracted from FilePath, the docstring was not > updated for generating multiple files, and the example usage was > referencing unrelated file. > > Fixes: de263986b5dc > Signed-off-by: Nir Soffer > --- > tests/qemu-iotests/iotests.py | 1

Re: [PATCH v2 2/5] qemu-iotests: Fix FilePaths docstring

2020-08-25 Thread Max Reitz
/qemu-iotests/iotests.py | 14 -- > 1 file changed, 8 insertions(+), 6 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v2 1/5] qemu-iotests: Fix FilePaths cleanup

2020-08-25 Thread Max Reitz
gt; --- > tests/qemu-iotests/iotests.py | 8 > 1 file changed, 4 insertions(+), 4 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [RFC PATCH 0/9] block/curl: Add caching of data downloaded from the remote server

2020-08-25 Thread Max Reitz
On 19.08.20 16:19, David Edmondson wrote: > On Wednesday, 2020-08-19 at 15:11:37 +01, Stefan Hajnoczi wrote: > >> On Tue, Aug 18, 2020 at 12:08:36PM +0100, David Edmondson wrote: >>> When using qemu-img to convert an image that is hosted on an HTTP >>> server to some faster local (or pseudo-local)

[PULL 26/34] qcow2: Clear the L2 bitmap when allocating a compressed cluster

2020-08-25 Thread Max Reitz
From: Alberto Garcia Compressed clusters always have the bitmap part of the extended L2 entry set to 0. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-Id: <04455b3de5dfeb9d1cfe1fc7b02d7060a6e09710.1594396418.git.be...@igalia.com> Signed-off-by: Max Reitz --- block

[PULL 32/34] qcow2: Allow preallocation and backing files if extended_l2 is set

2020-08-25 Thread Max Reitz
, and therefore it is perfectly possible to have an allocated cluster with all its subclusters unallocated. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Message-Id: <6d5b0f38e7dc5f2f31d8cab1cb92044e9909aece.1594396418.git.be...@igalia.com> Signed-off-by

[PULL 33/34] qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters

2020-08-25 Thread Max Reitz
. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Message-Id: <15e65112b4144381b4d8c0bdf8fb76b0d813e3d1.1594396418.git.be...@igalia.com> [mreitz: Fixed comment style] Signed-off-by: Max Reitz --- block/qcow2-cluster.c | 14 -- tests/qemu-iotes

[PULL 34/34] iotests: Add tests for qcow2 images with extended L2 entries

2020-08-25 Thread Max Reitz
From: Alberto Garcia Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-Id: Signed-off-by: Max Reitz --- tests/qemu-iotests/271 | 901 + tests/qemu-iotests/271.out | 726 ++ tests/qemu-iotests/group | 1 + 3

[PULL 31/34] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2020-08-25 Thread Max Reitz
From: Alberto Garcia Now that the implementation of subclusters is complete we can finally add the necessary options to create and read images with this feature, which we call "extended L2 entries". Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz

[PULL 28/34] qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()

2020-08-25 Thread Max Reitz
and possibly the tail. 2) if the tail area was compressed we are writing zeroes to the head and the body areas, which are already zeroized. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-Id: <17e05e2ee7e12f10dcf012da81e83ebe27eb3bef.1594396418.git.be...@igalia.

[PULL 23/34] qcow2: Add subcluster support to discard_in_l2_slice()

2020-08-25 Thread Max Reitz
cking file, discard cannot guarantee that the image will read back as zeroes. If this is important for the caller it should forbid it as qcow2_co_pdiscard() does (see 80f5c01183 for more details). Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz

[PULL 16/34] qcow2: Add qcow2_cluster_is_allocated()

2020-08-25 Thread Max Reitz
From: Alberto Garcia This helper function tells us if a cluster is allocated (that is, there is an associated host offset for it). Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Message-Id: <6d8771c5c79cbdc6c519875a5078e1cc85856d63.1594396418.git

[PULL 25/34] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()

2020-08-25 Thread Max Reitz
he L2 bitmap (because all affected subclusters were already allocated). This is detected in calculate_l2_meta(), and qcow2_alloc_cluster_link_l2() is never called in those cases. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Messa

[PULL 21/34] qcow2: Add subcluster support to qcow2_get_host_offset()

2020-08-25 Thread Max Reitz
() is not necessary as a separate function anymore so it's inlined into its caller. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-Id: [mreitz: Initialize expected_type to anything] Signed-off-by: Max Reitz --- block/qcow2.h | 38 --- block/qcow2-cluster.c

[PULL 12/34] qcow2: Add l2_entry_size()

2020-08-25 Thread Max Reitz
) anymore. This function returns the proper value for the image. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-Id: Signed-off-by: Max Reitz --- block/qcow2.h | 9 + block/qcow2-cluster.c | 12 ++-- block/qcow2-refcount.c | 14 -- block

[PULL 27/34] qcow2: Add subcluster support to handle_alloc_space()

2020-08-25 Thread Max Reitz
be to also fill with zeroes the other subclusters if we can guarantee that we are not overwriting existing data. However this would waste more disk space, so we should first evaluate if it's really worth doing. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Seme

[PULL 29/34] qcow2: Add subcluster support to qcow2_measure()

2020-08-25 Thread Max Reitz
From: Alberto Garcia Extended L2 entries are bigger than normal L2 entries so this has an impact on the amount of metadata needed for a qcow2 file. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-Id: <7efae2efd5e36b42d2570743a12576d68ce53685.1594396418.git.be...@igalia.

[PULL 22/34] qcow2: Add subcluster support to zero_in_l2_slice()

2020-08-25 Thread Max Reitz
ng that an existing entry does not need to be updated. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Message-Id: Signed-off-by: Max Reitz --- block/qcow2-cluster.c | 38 -- 1 file changed, 20 insertions(+), 18 deletions(-) di

[PULL 11/34] qcow2: Add offset_into_subcluster() and size_to_subclusters()

2020-08-25 Thread Max Reitz
From: Alberto Garcia Like offset_into_cluster() and size_to_clusters(), but for subclusters. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Message-Id: <3cc2390dcdef3d234d47c741b708bd8734490862.1594396418.git.be...@igalia.com> Signed-off-by: Max

[PULL 24/34] qcow2: Add subcluster support to check_refcounts_l2()

2020-08-25 Thread Max Reitz
AG_ZERO but the idea is the same: if none of the subclusters are allocated then we can clear the offset field and leave the bitmap untouched. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-Id: <9f4ed1d0a34b0a545b032c31ecd8c14734065342.1594396418.git.be...@igalia.com> Sig

[PULL 09/34] qcow2: Add subcluster-related fields to BDRVQcow2State

2020-08-25 Thread Max Reitz
ithout subclusters are treated as if they had exactly one subcluster per cluster (i.e. subcluster_size = cluster_size). Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <55bfeac86b092fa2c9d182a95cbeb479ff7eca4f.1594396418.git.be...@igalia.co

[PULL 17/34] qcow2: Add cluster type parameter to qcow2_get_host_offset()

2020-08-25 Thread Max Reitz
ad of integers for clarity and in order to detect errors more easily. This patch makes qcow2_get_host_offset() return 0 on success and puts the returned cluster type in a separate parameter. There are no semantic changes. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Seme

[PULL 30/34] qcow2: Add prealloc field to QCowL2Meta

2020-08-25 Thread Max Reitz
: Max Reitz Message-Id: <960d4c444a4f5a870e2b47e5da322a73cd9a2f5a.1594396418.git.be...@igalia.com> Signed-off-by: Max Reitz --- block/qcow2.h | 8 block/qcow2-cluster.c | 2 +- block/qcow2.c | 6 ++ 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a

[PULL 18/34] qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*

2020-08-25 Thread Max Reitz
values of QCow2ClusterType with their QCow2SubclusterType equivalents. This patch only changes the data types, there are no semantic changes. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: Signed-off-by: Max Reitz --- block/qcow2.h

[PULL 20/34] qcow2: Add subcluster support to calculate_l2_meta()

2020-08-25 Thread Max Reitz
iewed-by: Max Reitz Message-Id: <4292dd56e4446d386a2fe307311737a711c00708.1594396418.git.be...@igalia.com> Signed-off-by: Max Reitz --- block/qcow2-cluster.c | 167 +- 1 file changed, 133 insertions(+), 34 deletions(-) diff --git a/block/qcow2-cluste

[PULL 06/34] qcow2: Add get_l2_entry() and set_l2_entry()

2020-08-25 Thread Max Reitz
with calls to get_l2_entry() and set_l2_entry(). Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <9586363531fec125ba1386e561762d3e4224e9fc.1594396418.git.be...@igalia.com> Signed-off-by: Max Reitz ---

[PULL 10/34] qcow2: Add offset_to_sc_index()

2020-08-25 Thread Max Reitz
From: Alberto Garcia For a given offset, return the subcluster number within its cluster (i.e. with 32 subclusters per cluster it returns a number between 0 and 31). Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id

[PULL 15/34] qcow2: Add qcow2_get_subcluster_range_type()

2020-08-25 Thread Max Reitz
determined the type of a subcluster we can check the rest efficiently by counting the number of adjacent ones (or zeroes) in the bitmap. This is what this function does. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Message-Id: Signed-off-by: Max Reitz --- block

[PULL 19/34] qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC

2020-08-25 Thread Max Reitz
Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: Signed-off-by: Max Reitz --- block/qcow2.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 477d60dd71..60192f1be3 100644 --- a/block/qcow2.c +++ b/block/qcow2.c

[PULL 02/34] qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()

2020-08-25 Thread Max Reitz
now documented. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-Id: Signed-off-by: Max Reitz --- block/qcow2.h | 4 ++-- block/qcow2-cluster.c | 41 +++-- block/qcow2.c | 24 +++- 3 files changed, 32

<    7   8   9   10   11   12   13   14   15   16   >