Backup and commit re-query the source block status for every task and
size a task by the copy buffer, so a long run of zeroes turns into a
crowd of small write-zeroes requests. This series reuses what the
up-front scan already learned and lets one write-zeroes task cover a
whole run.
Backup of a 16G qcow2 image holding 1G of data, the rest never
allocated, sync=full to a raw target on ext4:
tasks time
before 16384 1.6s
after 1088 1.0s
Such an image is the normal case rather than a corner one: a guest with
discard enabled on a disk which is mostly free leaves exactly this
shape behind.
Eric, this is the v4 you asked about on 10-28; please take it instead
of v3.
v4, all from Vladimir's review of v3:
- v3 5/9 "don't reserve memory for zero tasks" is dropped. On its own
it let a write-zeroes task bounce through a buffer that s->mem never
charged. Its accounting change moves into 12/12 and applies only to a
task asking for BDRV_REQ_NO_FALLBACK, the one sure to need no buffer.
Its Reviewed-by tags go with it.
- 7/12: each prescan step moves past the dirty area it searched, so a
full backup of a fragmented source no longer searches the same range
again and again.
- 8/12, 9/12 and 10/12 are new, the follow-ups asked for on v3 7/9:
block_copy_is_cluster_allocated() becomes block_copy_scan_status(),
the two prescan helpers merge into block_copy_prescan(), and
zero_bitmap_valid becomes scan_done, published together with clearing
skip_unallocated.
- 11/12 is new: co_get_from_shres() and co_put_to_shres() skip the
mutex for zero bytes, which 12/12 makes the common case.
- 12/12: the serial fallback loop is gone. A refused task shrinks to one
cluster and the rest of its range is dirty again, so it comes back as
buffer-sized tasks running in parallel instead of one request holding
off guest writes to the whole range. A task asking for NO_FALLBACK is
left out of s->mem and of speed=. Widening uses
bdrv_dirty_bitmap_next_zero(), drops the MAX() with cluster_size, and
is no longer bounded by x-perf max-chunk, whose documentation now says
so. Andrey's Reviewed-by on v3 is dropped since the patch changed.
- rebased on master
v3:
https://lore.kernel.org/qemu-devel/[email protected]/
Signed-off-by: Denis V. Lunev <[email protected]>
CC: Vladimir Sementsov-Ogievskiy <[email protected]>
CC: John Snow <[email protected]>
CC: Andrey Drobyshev <[email protected]>
CC: Eric Blake <[email protected]>
Denis V. Lunev (12):
block/commit: pass BDRV_WANT_PRECISE to block-status
iotests/040: cover large and fragmented commit runs
block/commit: batch block-status queries
iotests/124: cover backup of zero clusters and holes
block/block-copy: extract block_copy_set_task_method()
block/block-copy: track known-zero source clusters
block/backup: pre-fill zero_bitmap for full/bitmap
block/block-copy: rename block_copy_is_cluster_allocated()
block/block-copy: merge the two backup scan steps into
block_copy_prescan()
block/block-copy: replace zero_bitmap_valid with scan_done
util/co-shared-resource: skip the mutex for an empty request
block/block-copy: coalesce write-zeroes tasks
block/backup.c | 86 ++++----
block/block-copy.c | 326 +++++++++++++++++++++++++------
block/commit.c | 58 ++++--
include/block/block-copy.h | 6 +-
qapi/block-core.json | 4 +-
tests/qemu-iotests/040 | 102 +++++++++-
tests/qemu-iotests/040.out | 4 +-
tests/qemu-iotests/124 | 347 ++++++++++++++++++++++++++++++++-
tests/qemu-iotests/124.out | 4 +-
util/qemu-co-shared-resource.c | 8 +
10 files changed, 829 insertions(+), 116 deletions(-)
base-commit: 73294d096cc591e122fde9e63db4739de6d84f2b
--
2.53.0
