'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1], DEFAULT_HASH_ALGORITHM is set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.
Back in QEMU, the LinuxDistro class defines the images it is going to use in the KNOWN_DISTROS attribute. All images have the 'checksum' attribute, defined using sha256, that will make avocado check if it has the right asset in the cache. So this means that we're downloading a sha1 image during check-avocado, then trying to fetch it using a sha256 hash. This doesn't work. Avocado will match the provided checksum, a sha256 hash, with the sha1 hash from the existing CHECKSUM file for the asset, and will re-download the image again. Avocado doesn't take into account that we're using a different hash algorithm than the one used to fetch the image via get-vm-image-fedora31 (in avocado: utils/asset.py, self.find_asset_file() and fetch(self, timeout=None). The new download will take into account the chosen hash algorithm, and subsequent avocado runs will not re-download the image again because now it's comparing sha256 with sha256. All of this, of course, assuming that the system was able to re-download the image before the avocado test timeouts. A more "fun" scenario happens if the network is not fast enough to download the image during the avocado timeout, the test will fail and the sha1 image will remain in the cache. And the test will keep trying to re-fetch it, and will keep failing with a timeout. There's something to be said about avocado matching different sha algorithms instead of re-calculating the hash with the chosen algorithm during fetch(), especially considering that the CHECKSUM file contains the name of the algorithm used. An avocado feature request [2] was opened to see if this is a feasible change to do. For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs. [1] https://github.com/avocado-framework/avocado.git @ 942a5d6972906 [2] https://github.com/avocado-framework/avocado/issues/5496 Cc: Alex Bennée <alex.ben...@linaro.org> Cc: Thomas Huth <th...@redhat.com> Cc: John Snow <js...@redhat.com> Cc: Cleber Rosa <cr...@redhat.com> Cc: Wainer dos Santos Moschetta <waine...@redhat.com> Signed-off-by: Daniel Henrique Barboza <danielhb...@gmail.com> --- tests/avocado/avocado_qemu/__init__.py | 30 ++++++++++++++++++-------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py index 910f3ba1ea..68c42948a3 100644 --- a/tests/avocado/avocado_qemu/__init__.py +++ b/tests/avocado/avocado_qemu/__init__.py @@ -423,12 +423,24 @@ class LinuxDistro: Holds information of known distros. """ #: A collection of known distros and their respective image checksum + # + # 'get-vm-image-fedora-31' target from Makefile.include will + # download the images using avocado default 'sha1' algorithm. + # This happens because the command line 'avocado vmimage get' + # from Makefile.include is not able to handle an extra --algorithm + # argument. + # + # To avoid a re-download of a recent image download due to a sha + # mismatch in the first test run, all Fedora 31 images are using + # 'sha1' instead of 'sha256'. + # + # FIXME: revisit this change if/when avocado learns to do CHECKSUM + # matches using different algorithms. KNOWN_DISTROS = { 'fedora': { '31': { 'x86_64': - {'checksum': ('e3c1b309d9203604922d6e255c2c5d09' - '8a309c2d46215d8fc026954f3c5c27a0'), + {'checksum': ('f62f8eabbf3687ea610c495bd59551a0025f99b7'), 'pxeboot_url': ('https://archives.fedoraproject.org/' 'pub/archive/fedora/linux/releases/31/' 'Everything/x86_64/os/images/pxeboot/'), @@ -438,8 +450,7 @@ class LinuxDistro: 'console=ttyS0,115200n8'), }, 'aarch64': - {'checksum': ('1e18d9c0cf734940c4b5d5ec592facae' - 'd2af0ad0329383d5639c997fdf16fe49'), + {'checksum': ('39d1fe099cdecacae894480d421be57d18b4e854'), 'pxeboot_url': 'https://archives.fedoraproject.org/' 'pub/archive/fedora/linux/releases/31/' 'Everything/aarch64/os/images/pxeboot/', @@ -450,11 +461,9 @@ class LinuxDistro: ' console=ttyAMA0'), }, 'ppc64': - {'checksum': ('7c3528b85a3df4b2306e892199a9e1e4' - '3f991c506f2cc390dc4efa2026ad2f58')}, + {'checksum': ('9993dc28e7c49ceb52125f9513130dfe2ace026c')}, 's390x': - {'checksum': ('4caaab5a434fd4d1079149a072fdc789' - '1e354f834d355069ca982fdcaf5a122d')}, + {'checksum': ('455f017b82decf32f366e06e7a7d0f6da86f96a7')}, }, '32': { 'aarch64': @@ -595,15 +604,18 @@ def download_boot(self): self.log.info('Downloading/preparing boot image') # Fedora 31 only provides ppc64le images image_arch = self.arch + hash_algorithm = 'sha256' if self.distro.name == 'fedora': if image_arch == 'ppc64': image_arch = 'ppc64le' + if self.distro.version == '31': + hash_algorithm = 'sha1' try: boot = vmimage.get( self.distro.name, arch=image_arch, version=self.distro.version, checksum=self.distro.checksum, - algorithm='sha256', + algorithm=hash_algorithm, cache_dir=self.cache_dirs[0], snapshot_dir=self.workdir) except: -- 2.37.3