Re: [PATCH 2/2] qemu-img: Add --skip-broken for 'convert --bitmaps'
08.07.2021 04:30, Eric Blake wrote: The point of 'qemu-img convert --bitmaps' is to be a convenience for actions that are already possible through a string of smaller 'qemu-img bitmap' sub-commands. One situation not accounted for already is that if a source image contains an inconsistent bitmap (for example, because a qemu process died abruptly before flushing bitmap state), the user MUST delete those inconsistent bitmaps before anything else useful can be done with the image. We don't want to delete inconsistent bitmaps by default: although a corrupt bitmap is only a loss of optimization rather than a corruption of user-visible data, it is still nice to require the user to opt in to the fact that they are aware of the loss of the bitmap. Still, requiring the user to check 'qemu-img info' to see whether bitmaps are consistent, then use 'qemu-img bitmap --remove' to remove offenders, all before using 'qemu-img convert', is a lot more work than just adding a knob 'qemu-img convert --bitmaps --skip-broken' which opts in to skipping the broken bitmaps. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1946084 Signed-off-by: Eric Blake --- docs/tools/qemu-img.rst | 8 +++- qemu-img.c| 20 +-- tests/qemu-iotests/tests/qemu-img-bitmaps | 4 tests/qemu-iotests/tests/qemu-img-bitmaps.out | 14 + 4 files changed, 43 insertions(+), 3 deletions(-) diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst index 1d8470eada0e..5cf1c764597b 100644 --- a/docs/tools/qemu-img.rst +++ b/docs/tools/qemu-img.rst @@ -414,7 +414,7 @@ Command description: 4 Error on reading data -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps [--skip-broken]] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME Of course, [--bitmaps [--skip-broken]] looks like --skip-broken is a suboption.. But actually it's not so. So, shouldn't it be named more explicit, like --skip-broken-bitmaps ? To be sure that we will not interfere in future with some other broken things we want to skip? And to avoid strange but correct command lines like qemu-img convert --skip-broken --bitmaps src dst Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM* to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can @@ -456,6 +456,12 @@ Command description: *NUM_COROUTINES* specifies how many coroutines work in parallel during the convert process (defaults to 8). + Use of ``--bitmaps`` requests that any persistent bitmaps present in + the original are also copied to the destination. If any bitmap is + inconsistent in the source, the conversion will fail unless + ``--skip-broken`` is also specified to copy only the consistent + bitmaps. + .. option:: create [--object OBJECTDEF] [-q] [-f FMT] [-b BACKING_FILE] [-F BACKING_FMT] [-u] [-o OPTIONS] FILENAME [SIZE] Create the new disk image *FILENAME* of size *SIZE* and format diff --git a/qemu-img.c b/qemu-img.c index 68a4d298098f..e8b012f39c0c 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -82,6 +82,7 @@ enum { OPTION_MERGE = 274, OPTION_BITMAPS = 275, OPTION_FORCE = 276, +OPTION_SKIP_BROKEN = 277, }; typedef enum OutputFormat { @@ -2101,7 +2102,8 @@ static int convert_do_copy(ImgConvertState *s) return s->ret; } -static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst) +static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst, +bool skip_broken) { BdrvDirtyBitmap *bm; Error *err = NULL; @@ -2113,6 +2115,10 @@ static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst) continue; } name = bdrv_dirty_bitmap_name(bm); +if (skip_broken && bdrv_dirty_bitmap_inconsistent(bm)) { +warn_report("Skipping inconsistent bitmap %s", name); +continue; +} qmp_block_dirty_bitmap_add(dst->node_name, name, true, bdrv_dirty_bitmap_granularity(bm), true, true, @@ -2167,6 +2173,7 @@ static int img_convert(int argc, char **argv) bool force_share = false; bool explict_min_sparse = false; bool bitmaps = false; +bool skip_broken = false; int64_t rate_limit = 0;
[PATCH 2/2] qemu-img: Add --skip-broken for 'convert --bitmaps'
The point of 'qemu-img convert --bitmaps' is to be a convenience for actions that are already possible through a string of smaller 'qemu-img bitmap' sub-commands. One situation not accounted for already is that if a source image contains an inconsistent bitmap (for example, because a qemu process died abruptly before flushing bitmap state), the user MUST delete those inconsistent bitmaps before anything else useful can be done with the image. We don't want to delete inconsistent bitmaps by default: although a corrupt bitmap is only a loss of optimization rather than a corruption of user-visible data, it is still nice to require the user to opt in to the fact that they are aware of the loss of the bitmap. Still, requiring the user to check 'qemu-img info' to see whether bitmaps are consistent, then use 'qemu-img bitmap --remove' to remove offenders, all before using 'qemu-img convert', is a lot more work than just adding a knob 'qemu-img convert --bitmaps --skip-broken' which opts in to skipping the broken bitmaps. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1946084 Signed-off-by: Eric Blake --- docs/tools/qemu-img.rst | 8 +++- qemu-img.c| 20 +-- tests/qemu-iotests/tests/qemu-img-bitmaps | 4 tests/qemu-iotests/tests/qemu-img-bitmaps.out | 14 + 4 files changed, 43 insertions(+), 3 deletions(-) diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst index 1d8470eada0e..5cf1c764597b 100644 --- a/docs/tools/qemu-img.rst +++ b/docs/tools/qemu-img.rst @@ -414,7 +414,7 @@ Command description: 4 Error on reading data -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps [--skip-broken]] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM* to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can @@ -456,6 +456,12 @@ Command description: *NUM_COROUTINES* specifies how many coroutines work in parallel during the convert process (defaults to 8). + Use of ``--bitmaps`` requests that any persistent bitmaps present in + the original are also copied to the destination. If any bitmap is + inconsistent in the source, the conversion will fail unless + ``--skip-broken`` is also specified to copy only the consistent + bitmaps. + .. option:: create [--object OBJECTDEF] [-q] [-f FMT] [-b BACKING_FILE] [-F BACKING_FMT] [-u] [-o OPTIONS] FILENAME [SIZE] Create the new disk image *FILENAME* of size *SIZE* and format diff --git a/qemu-img.c b/qemu-img.c index 68a4d298098f..e8b012f39c0c 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -82,6 +82,7 @@ enum { OPTION_MERGE = 274, OPTION_BITMAPS = 275, OPTION_FORCE = 276, +OPTION_SKIP_BROKEN = 277, }; typedef enum OutputFormat { @@ -2101,7 +2102,8 @@ static int convert_do_copy(ImgConvertState *s) return s->ret; } -static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst) +static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst, +bool skip_broken) { BdrvDirtyBitmap *bm; Error *err = NULL; @@ -2113,6 +2115,10 @@ static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst) continue; } name = bdrv_dirty_bitmap_name(bm); +if (skip_broken && bdrv_dirty_bitmap_inconsistent(bm)) { +warn_report("Skipping inconsistent bitmap %s", name); +continue; +} qmp_block_dirty_bitmap_add(dst->node_name, name, true, bdrv_dirty_bitmap_granularity(bm), true, true, @@ -2167,6 +2173,7 @@ static int img_convert(int argc, char **argv) bool force_share = false; bool explict_min_sparse = false; bool bitmaps = false; +bool skip_broken = false; int64_t rate_limit = 0; ImgConvertState s = (ImgConvertState) { @@ -2188,6 +2195,7 @@ static int img_convert(int argc, char **argv) {"salvage", no_argument, 0, OPTION_SALVAGE}, {"target-is-zero", no_argument, 0, OPTION_TARGET_IS_ZERO}, {"bitmaps", no_argument, 0, OPTION_BITMAPS}, +{"skip-broken", no_argument, 0, OPTION_SKIP_BROKEN}, {0, 0, 0, 0} }; c = getopt_long(argc, argv,