On Fri, Jan 17, 2020 at 6:41 AM Strahil Nikolov <hunter86...@yahoo.com>
wrote:
> On January 17, 2020 12:10:56 AM GMT+02:00, Chris Adams <c...@cmadams.net>
> wrote:
> >Once upon a time, Nir Soffer <nsof...@redhat.com> said:
> >> On Tue, Jan 7, 2020 at 4:02 PM Chris Adams <c...@cmadams.net> wrote:
> >> > Once upon a time, m.skrzetu...@gmail.com <m.skrzetu...@gmail.com>
> >said:
> >> > > I'd give up on the ISO domain. I started like you and then read
> >the docs
> >> > which said that ISO domain is deprecated.
> >> > > I'd upload all files to a data domain.
> >> >
> >> > Note that that only works if your data domain is NFS... iSCSI data
> >> > domains will let you upload ISOs, but connecting them to a VM
> >fails.
> >>
> >> ISO on iSCSI/FC domains works fine for starting a VM from ISO, which
> >is the
> >> main use case.
> >
> >Okay - it didn't the last time I tried it (I just got errors). Thanks.
>
> I have opened and RFE for ISO checksumming, as currently the uploader can
> silently corrupt your DVD.
>
Can you share the bug number?
> With gluster, I have an option to check the ISO checksum and
> verify/replace the file, but with Block-based storage that will be quite
> difficult.
>
Checksumming is a general feature not related to ISO uploads. But
checksumming tools do not understand
sparseness so you should really use a tool designed for compare disk
images, like "qemu-img compare".
Here is an example:
1. Create fedora 30 image for testing:
$ virt-builder fedora-30 -o fedora-30.raw
...
$ qemu-img info fedora-30.raw
image: fedora-30.raw
file format: raw
virtual size: 6 GiB (6442450944 bytes)
disk size: 1.15 GiB
2. Create a checksum of the image
$ time shasum fedora-30.raw
991c2efee723e04b7d41d75f70d19bade02b400d fedora-30.raw
real 0m14.641s
user 0m12.653s
sys 0m1.749s
3. Create compressed qcow2 image with same content
$ qemu-img convert -f raw -O qcow2 -c fedora-30.raw fedora-30.qcow2
...
$ qemu-img info fedora-30.qcow2
image: fedora-30.qcow2
file format: qcow2
virtual size: 6 GiB (6442450944 bytes)
disk size: 490 MiB
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
This is typical file format used for publishing disk images. The contents
of this
image are the same as the raw version from the guest point of view.
3. Compare image content
$ time qemu-img compare fedora-30.raw fedora-30.qcow2
Images are identical.
real 0m4.680s
user 0m4.273s
sys 0m0.553s
3 times faster to compare 2 images with different format compared with
creating
a checksum of single image.
Now lets see how we can use this to verify uploads.
4. Upload the qcow2 compressed image to a new raw disk (requires ovirt 4.4
alpah3):
$ python3 upload_disk.py --engine-url https://engine/ --username
admin@internal --password-file password \
--cafile ca.pem --sd-name nfs1-export2 --disk-format raw --disk-sparse
fedora-30.qcow2
5. Download image to raw format:
$ python3 download_disk.py --engine-url https://engine/ --username
admin@internal --password-file password \
--cafile ca.pem --format raw f40023a5-ddc4-4fcf-b8e2-af742f372104
fedora-30.download.raw
6. Comparing original and downloaded images
$ qemu-img compare fedora-30.qcow2 fedora-30.download.raw
Images are identical.
Back to the topic of ISO uploads to block storage. Block volumes in oVirt
are always aligned to
128 MiB, so when you upload an image which is not aligned to 128 MiB, oVirt
creates a bigger
block device. The contents of the device after the image content are not
defined, unless you
zero this area during upload. The current upload_disk.py example does not
zero the end of
the device since the guest do not care about it, but this makes verifying
uploads harder.
The best way to handle this issue is to truncate the ISO image up to the
next multiple of 128 MiB
before uploading it:
$ ls -l Fedora-Server-dvd-x86_64-30-1.2.iso
-rw-rw-r--. 1 nsoffer nsoffer 3177185280 Nov 8 23:09
Fedora-Server-dvd-x86_64-30-1.2.iso
$ python3 -c 'n = 3177185280 + 128 * 1024**2 - 1; print(n - (n % (128 *
1024**2)))'
3221225472
$ truncate -s 3221225472 Fedora-Server-dvd-x86_64-30-1.2.iso
The contents of the iso image is the same as it will be on the block device
after the upload, and
uploading this image will zero the end of the device.
If we upload this image, we can check the upload using qemu img compare.
$ python3 upload_disk.py --engine-url https://engine/ --username
admin@internal --password-file password \
--cafile ca.pem --sd-name iscsi-1 --disk-format raw
Fedora-Server-dvd-x86_64-30-1.2.iso
$ python3 download_disk.py --engine-url https://engine/ --username
admin@internal --password-file password \
--cafile ca.pem --format raw 5f0b5347-bbbc-4521-9ca0-8fc17670bab0
iso.raw
$ qemu-img compare iso.raw Fedora-Server-dvd-x86_64-30-1.2.iso
Images are identical.
This is not easy to use and requires knowledge about oVirt internals (128
MiB alignment), so I guess we
need to make this simpler.
Nir
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/FLOI6T6MHGS4QJPKP7NPNPYKHWBSKTDI/
