On Tue, 26 Nov 2024 at 17:31, Daniel P. Berrangé <[email protected]> wrote: > > On Tue, Nov 26, 2024 at 05:44:29PM +0100, Philippe Mathieu-Daudé wrote: > > Hi, > > > > On 4/9/24 12:38, Thomas Huth wrote: > > fetch() can fail [*] (see previous patch, various Exceptions returned). > > > > What should we do in this case? If we ignore a missing artifact, > > the tests will eventually fail. Better bail out early and save > > credit minutes? > > We already do what you describe - 'fetch' will raise an exception > which causes the precache task to fail, and the CI job gets marked > as failed. We don't attempt to run tests if assets are missing. > > > > > @@ -58,6 +59,12 @@ def tearDown(self): > > > def main(): > > > path = os.path.basename(sys.argv[0])[:-3] > > > + > > > + cache = os.environ.get("QEMU_TEST_PRECACHE", None) > > > + if cache is not None: > > > + Asset.precache_suites(path, cache) > > > + return > > > + > > > tr = pycotap.TAPTestRunner(message_log = > > > pycotap.LogMode.LogToError, > > > test_output_log = > > > pycotap.LogMode.LogToError) > > > unittest.main(module = None, testRunner = tr, > > > argv=["__dummy__", path]) > > > > [*] Peter reported the following CI failure: > > > > https://gitlab.com/qemu-project/qemu/-/jobs/8474928266 > > > > 2024-11-26 14:58:53,170 - qemu-test - ERROR - Unable to download > > https://apt.armbian.com/pool/main/l/linux-6.6.16/linux-image-current-sunxi_24.2.1_armhf__6.6.16-Seb3e-D6b4a-P2359-Ce96bHfe66-HK01ba-V014b-B067e-R448a.deb: > > This looks to be working as intended. We failed to cache > the asset, and so we stopped the job, without trying to > run the tests.
The job ended up in state "failed", with a red X mark in the gitlab UI. If we intend that not being able to fetch the assets doesn't count as a test failure, that didn't work here. If we do intend that fetch failures should be CI failures, we need to make our process of fetching and caching the images more robust, because otherwise the result is flaky CI jobs. thanks -- PMM
