On Fri, Oct 02, 2020 at 01:57:04AM +0300, Nir Soffer wrote: > On Wed, Sep 30, 2020 at 1:49 PM Tomáš Golembiovský <tgole...@redhat.com> > wrote: > > Hi, > > > > currently, when we run virt-sparsify on VM or user runs VM with discard > > enabled and when the disk is on block storage in qcow, the results are > > not reflected in oVirt. The blocks get discarded, storage can reuse them > > and reports correct allocation statistics, but oVirt does not. In oVirt > > one can still see the original allocation for disk and storage domain as > > it was before blocks were discarded. This is super-confusing to the > > users because when they check after running virt-sparsify and see the > > same values they think sparsification is not working. Which is not true. > > This may be documentation issue. This is a known limitation of oVirt thin > provisioned storage. We allocate space as needed, but we release the > space only when a volume is deleted. > > > It all seems to be because of our LVM layout that we have on storage > > domain. The feature page for discard [1] suggests it could be solved by > > running lvreduce. But this does not seem to be true. When blocks are > > discarded the QCOW does not necessarily change its apparent size, the > > blocks don't have to be removed from the end of the disk. So running > > lvreduce is likely to remove valuable data. > > We have an API to (safely) reduce a volume to optimal size: > http://ovirt.github.io/ovirt-engine-api-model/master/#services/disk/methods/reduce > > Reducing images depends on qcow2 image-end-offset. We can tell which > is the highest offset used by inactive disk: > https://github.com/oVirt/vdsm/blob/24f646383acb615b090078fc7aeddaf7097afe57/lib/vdsm/storage/blockVolume.py#L403 > > and reduce the logical volume to this size. > > But this will not works since qcow2 image-end-offset is not decreased by > > virt-sparsify --in-place
Right - this doesn't "defragment" the qcow2 file, ie. moving clusters to the beginning - so (except by accident) it won't make the qcow2 file smaller. Virt-sparsify in copying mode will actually do what you want, but obviously is much more heavyweight and complex to use. > So it is true that sparsify releases unused space on storage level, but it > does > not decrease the qcow2 image allocation, so we cannot reduce the logical > volumes. > > > At the moment I don't see how we could achieve the correct values. If > > anyone has any idea feel free to entertain me. The only option seems to > > be to switch to LVM thin pools. Do we have any plans on doing that? > > No, thin pools do not support clustering, this can be used only on a single > host. oVirt lvm based volumes are accessed on multiple hosts at the same > time. > > Here is an example sparisfy test showing the issue: > > Before writing data to new disk > > guest: > > # df -h /data > Filesystem Size Used Avail Use% Mounted on > /dev/sda1 10G 104M 9.9G 2% /data > > storage: > > $ ls -lhs /home/target/2/00 > 2.1G -rw-r--r--. 1 root root 100G Oct 2 00:57 /home/target/2/00 > > host: > > # qemu-img info > /dev/27f2b637-ffb1-48f9-8f68-63ed227392b9/42cf66df-43ad-4cfa-ab57-a943516155d1 > image: > /dev/27f2b637-ffb1-48f9-8f68-63ed227392b9/42cf66df-43ad-4cfa-ab57-a943516155d1 > file format: qcow2 > virtual size: 10 GiB (10737418240 bytes) > disk size: 0 B > cluster_size: 65536 > Format specific information: > compat: 1.1 > compression type: zlib > lazy refcounts: false > refcount bits: 16 > corrupt: false > > # qemu-img check > /dev/27f2b637-ffb1-48f9-8f68-63ed227392b9/42cf66df-43ad-4cfa-ab57-a943516155d1 > No errors were found on the image. > 168/163840 = 0.10% allocated, 0.60% fragmented, 0.00% compressed clusters > Image end offset: 12582912 > > > After writing 5g file to file system on this disk in the guest: > > guest: > > $ dd if=/dev/zero bs=8M count=640 of=/data/test oflag=direct > conv=fsync status=progress > > # df -h /data > Filesystem Size Used Avail Use% Mounted on > /dev/sda1 10G 5.2G 4.9G 52% /data > > storage: > > $ ls -lhs /home/target/2/00 > 7.1G -rw-r--r--. 1 root root 100G Oct 2 01:06 /home/target/2/00 > > host: > > # qemu-img check > /dev/27f2b637-ffb1-48f9-8f68-63ed227392b9/42cf66df-43ad-4cfa-ab57-a943516155d1 > No errors were found on the image. > 82088/163840 = 50.10% allocated, 5.77% fragmented, 0.00% compressed clusters > Image end offset: 5381423104 > > > After deleting the 5g file: > > guest: > > # df -h /data > Filesystem Size Used Avail Use% Mounted on > /dev/sda1 10G 104M 9.9G 2% /data > > storage: > > $ ls -lhs /home/target/2/00 > 7.1G -rw-r--r--. 1 root root 100G Oct 2 01:12 /home/target/2/00 > > host: > > # qemu-img check > /dev/27f2b637-ffb1-48f9-8f68-63ed227392b9/42cf66df-43ad-4cfa-ab57-a943516155d1 > No errors were found on the image. > 82088/163840 = 50.10% allocated, 5.77% fragmented, 0.00% compressed clusters > Image end offset: 5381423104 > > > After sparsifying disk: > > storage: > $ qemu-img check /var/tmp/download.qcow2 > No errors were found on the image. > 170/163840 = 0.10% allocated, 0.59% fragmented, 0.00% compressed clusters > Image end offset: 11927552 > > $ ls -lhs /home/target/2/00 > 2.1G -rw-r--r--. 1 root root 100G Oct 2 01:14 /home/target/2/00 > > host: > > # qemu-img check > /dev/27f2b637-ffb1-48f9-8f68-63ed227392b9/42cf66df-43ad-4cfa-ab57-a943516155d1 > No errors were found on the image. > 170/163840 = 0.10% allocated, 0.59% fragmented, 0.00% compressed clusters > Image end offset: 4822138880 > > Allocation decreased from 50% to 0.1%, but image end offset > decreased only from 5381423104 to 4822138880 (-10.5%). > > I don't know if this is a behavior change in virt-sparsify or qemu > or it was always like that. AFAIK nothing in virt-sparsify --in-place or qemu has changed here. > We had an old and unused sparsifyVolume API in vdsm before 4.4. This did not > use > --in-place and was very complicated because of this. But I think it > would work in this > case, since qemu-img convert will drop the unallocated areas. > > For example after downloading the sparsified disk, we get: > > $ qemu-img check download.qcow2 > No errors were found on the image. > 170/163840 = 0.10% allocated, 0.59% fragmented, 0.00% compressed clusters > Image end offset: 11927552 > > > Kevin, is this the expected behavior or a bug in qemu? > > The disk I tested is a single qcow2 image without the backing file, so > theoretically > qemu can deallocate all the discarded clusters. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/6BVZV63MESG7ZXAZPSVTM6VB66UVALEP/