On Sun, Oct 25, 2020 at 5:55 PM Nir Soffer <nsof...@redhat.com> wrote:
>
> We have tests for vdsm qemuimg wrapper module converting
> images and verifying the operation using "qemu-img compare".
>
> I found that tests using "qemu-img compare -s" started to fail
> recently on RHEL 8.3.0 nightly
> (qemu-img-5.1.0-10.module+el8.3.0+8254+568ca30d.x86_6)
> and Fedora 32 using virt-preview (qemu-img-5.1.0-5.fc32.x86_64)
>
> Here is an example reproducing this issue on ext4 filesystem
> on Fedora 32. I did check this case on RHEL 8.3.0, but I think the
> same test fails there.
>
> Create source chain:
>
> $ qemu-img create -f qcow2 src_base.qcow2 10m
> Formatting 'src_base.qcow2', fmt=qcow2 cluster_size=65536
> compression_type=zlib size=10485760 lazy_refcounts=off
> refcount_bits=16
>
> $ qemu-img create -f qcow2 src_top.qcow2 -b src_base.qcow2 -F qcow2
> Formatting 'src_top.qcow2', fmt=qcow2 cluster_size=65536
> compression_type=zlib size=10485760 backing_file=src_base.qcow2
> backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
>
> Create destination chain:
>
> $ qemu-img create -f qcow2 dst_base.qcow2 10m
> Formatting 'dst_base.qcow2', fmt=qcow2 cluster_size=65536
> compression_type=zlib size=10485760 lazy_refcounts=off
> refcount_bits=16
>
> $ qemu-img create -f qcow2 dst_top.qcow2 -b dst_base.qcow2 -F qcow2
> Formatting 'dst_top.qcow2', fmt=qcow2 cluster_size=65536
> compression_type=zlib size=10485760 backing_file=dst_base.qcow2
> backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
>
> Write data to source chain:
>
> $ qemu-io -f qcow2 -c 'write -P 240 0 65536' src_base.qcow2
> wrote 65536/65536 bytes at offset 0
> 64 KiB, 1 ops; 00.01 sec (7.355 MiB/sec and 117.6809 ops/sec)
>
> $ qemu-io -f qcow2 -c 'write -P 241 1048576 65536' src_top.qcow2
> wrote 65536/65536 bytes at offset 1048576
> 64 KiB, 1 ops; 00.01 sec (7.509 MiB/sec and 120.1387 ops/sec)
>
> Compare images:
>
> $ qemu-img compare src_base.qcow2 dst_base.qcow2
> Images are identical.
>
> $ qemu-img compare src_base.qcow2 dst_base.qcow2 -s
> Strict mode: Offset 65536 block status mismatch!
>
> This check succeeds with qemu-img 4.2.0-19 in our Travis CI.
>
> $ qemu-img map --output json src_base.qcow2
> [{ "start": 0, "length": 65536, "depth": 0, "zero": false, "data":
> true, "offset": 327680},
> { "start": 65536, "length": 10420224, "depth": 0, "zero": true, "data":
> false}]
>
> $ qemu-img map --output json dst_base.qcow2
> [{ "start": 0, "length": 65536, "depth": 0, "zero": false, "data":
> true, "offset": 327680},
> { "start": 65536, "length": 10420224, "depth": 0, "zero": true, "data":
> false}]
>
> Allocation looks identical - the first extent is data, and the second is a
> hole.
>
> But if we check using qemu-nbd[1]:
>
> $ python
> >>> from ovirt_imageio import client
> >>> from pprint import pprint
> >>> pprint(list(client.extents("src_base.qcow2")))
> [ZeroExtent(start=0, length=65536, zero=False, hole=False),
> ZeroExtent(start=65536, length=10420224, zero=True, hole=True)]
> >>> pprint(list(client.extents("dst_base.qcow2")))
> [ZeroExtent(start=0, length=65536, zero=False, hole=False),
> ZeroExtent(start=65536, length=10420224, zero=True, hole=False)]
>
> This shows an issue - the second extent is reported as a hole in the
> source image, and as an allocated area in the destination image.
>
> $ ls -lhs *_base.qcow2
> 324K -rw-r--r--. 1 nsoffer nsoffer 384K Oct 25 17:12 dst_base.qcow2
> 324K -rw-r--r--. 1 nsoffer nsoffer 384K Oct 25 17:10 src_base.qcow2
>
> Maybe the clusters in the source image are zeroed? But if they are
> zeroed, why qemu-img map reports them as a hole? Seems that we have
> a behavior change in qemu-img convert, and a bug in qemu-img map.
>
> I compared also the top images, after removing the backing file:
>
> $ qemu-img rebase -u -b "" src_top.qcow2
> $ qemu-img rebase -u -b "" dst_top.qcow2
> $ qemu-img compare src_top.qcow2 dst_top.qcow2
> Images are identical.
> $ qemu-img compare src_top.qcow2 dst_top.qcow2 -s
> Images are identical.
>
> So the issue is only when converting single image to single image
> without a backing file.
>
> [1] export the image with qemu-nbd, and list the extents using
> NBD_CMD_BLOCK_STATUS. Using ovirt-imageio-client 2.1.1.
Kevin, we discussed a related issue, result of bug fix to zero images
with unknown content:
commit 4d7c487eac1652dfe4498fe84f32900ad461d61b
Author: Max Reitz <mre...@redhat.com>
Date: Wed Jul 24 19:12:29 2019 +0200
qemu-img: Fix bdrv_has_zero_init() use in convert
Could it be the same issue?
"qemu-img convert -n" must zero the target if there is no backing file,
so the dst image is zeroed so the second extent is not a hole now.
