Hi all, this is the resubmission of my block job patches, originally
meant for 1.2. This still does not include a persistent dirty bitmap,
which I hope to post in October.
The patches are organized as follows:
01-13 preparatory work for block job errors, including support for
pausing and resuming jobs
14-18 introduce block job errors, and add support in block-stream
19-25 preparatory work for block mirroring: new commands/concepts
and creating new functions out of existing code.
26-33 introduce a simple version of mirroring. The initial patch
add the mirroring logic, followed by the ability to switch to
the destination of migration and to handle errors during the job.
All these changes come with testcases. Removing the ability to
query the target file is the main change from v1.
34-41 These patches introduce the first optimizations, namely supporting
an arbitrary granularity for the dirty bitmap. The current default,
1M, is too coarse to let the job converge quickly and in almost
real-time. These patches reimplement the block device dirty bitmap
to allow efficient iteration, and add cluster copy-on-write logic.
Cluster copy-on-write is needed because management will want to
start the copy before the backing file is in place in the destination;
if mirroring takes care of copy-on-write, BDRV_O_NO_BACKING can be
used even if the granularity is smaller than the cluster size.
42-45 A second round optimizations, replacing serialized read-write
operations with multiple asynchronous I/O operations. The various
in-flight operations can be of arbitrary size. The initial copy
will end up reading large chunks sequentially (10M by default),
while subsequent passes can mimic more closely the guest's I/O
patterns.
All comments from Kevin's partial review are addressed, so I believe
the first 29 patches should be ready to go. Laszlo already reviewed some
of the subsequent parts.
Please review!
Jeff Cody (1):
blockdev: rename block_stream_cb to a generic block_job_cb
Paolo Bonzini (44):
qerror/block: introduce QERR_BLOCK_JOB_NOT_ACTIVE
block: fix documentation of block_job_cancel_sync
block: move job APIs to separate files
block: add block_job_query
block: add support for job pause/resume
qmp: add block-job-pause and block-job-resume
qemu-iotests: add test for pausing a streaming operation
block: rename block_job_complete to block_job_completed
iostatus: rename BlockErrorAction, BlockQMPEventAction
iostatus: move BlockdevOnError declaration to QAPI
iostatus: change is_read to a bool
iostatus: reorganize io error code
block: introduce block job error
stream: add on-error argument
blkdebug: process all set_state rules in the old state
qemu-iotests: map underscore to dash in QMP argument names
qemu-iotests: add tests for streaming error handling
block: add bdrv_query_info
block: add bdrv_query_stats
block: add bdrv_open_backing_file
block: introduce new dirty bitmap functionality
block: export dirty bitmap information in query-block
block: add block-job-complete
block: introduce BLOCK_JOB_READY event
mirror: introduce mirror job
qmp: add drive-mirror command
mirror: implement completion
qemu-iotests: add mirroring test case
iostatus: forward bdrv_iostatus_reset to block job
mirror: add support for on-source-error/on-target-error
qmp: add pull_event function
qemu-iotests: add testcases for mirroring
on-source-error/on-target-error
host-utils: add ffsl
add hierarchical bitmap data type and test cases
block: implement dirty bitmap using HBitmap
block: make round_to_clusters public
mirror: perform COW if the cluster size is bigger than the
granularity
block: return count of dirty sectors, not chunks
block: allow customizing the granularity of the dirty bitmap
mirror: allow customizing the granularity
mirror: switch mirror_iteration to AIO
mirror: add buf-size argument to drive-mirror
mirror: support more than one in-flight AIO operation
mirror: support arbitrarily-sized iterations
Makefile.objs | 5 +-
QMP/qmp-events.txt | 42 +++
QMP/qmp.py | 20 ++
block-migration.c | 7 +-
block.c | 480 +++++++++++++-----------------
block.h | 37 ++-
block/Makefile.objs | 3 +-
block/blkdebug.c | 12 +-
block/mirror.c | 576 ++++++++++++++++++++++++++++++++++++
block/stream.c | 33 ++-
block_int.h | 192 +++---------
blockdev.c | 259 ++++++++++++++---
blockjob.c | 282 ++++++++++++++++++
blockjob.h | 280 ++++++++++++++++++
hbitmap.c | 400 +++++++++++++++++++++++++
hbitmap.h | 207 +++++++++++++
hmp-commands.hx | 73 ++++-
hmp.c | 65 ++++-
hmp.h | 4 +
host-utils.h | 26 ++
hw/fdc.c | 4 +-
hw/ide/core.c | 22 +-
hw/ide/pci.c | 4 +-
hw/scsi-disk.c | 25 +-
hw/scsi-generic.c | 4 +-
hw/virtio-blk.c | 23 +-
monitor.c | 2 +
monitor.h | 2 +
qapi-schema.json | 206 ++++++++++++-
qemu-tool.c | 6 +
qerror.h | 9 +
qmp-commands.hx | 75 ++++-
tests/Makefile | 2 +
tests/qemu-iotests/030 | 260 ++++++++++++++++-
tests/qemu-iotests/030.out | 4 +-
tests/qemu-iotests/040 | 658 ++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/040.out | 5 +
tests/qemu-iotests/group | 5 +-
tests/qemu-iotests/iotests.py | 19 +-
tests/test-hbitmap.c | 408 ++++++++++++++++++++++++++
trace-events | 24 +-
41 file modificati, 4176 inserzioni(+), 594 rimozioni(-)
create mode 100644 block/mirror.c
create mode 100644 blockjob.c
create mode 100644 blockjob.h
create mode 100644 hbitmap.c
create mode 100644 hbitmap.h
create mode 100755 tests/qemu-iotests/040
create mode 100644 tests/qemu-iotests/040.out
create mode 100644 tests/test-hbitmap.c
--
1.7.12
