Control: tags -1 wontfix Hi Paul,
2018-05-08 21:48 GMT+02:00 Paul Gevers <elb...@debian.org>: > Source: unattended-upgrades > Version: 1.0 > Severity: important > User: debian...@lists.debian.org > Usertags: flaky > > While inspecting regressions in autopkgtest results¹, I noticed that > your package unattended-upgrades fails regularly, without obvious > changes. There are multiple failure modes (some copied below). Thank you for working on autopkgtest gating migration! > Could you please investigate and make your autopkgtest more robust? > Please contact me if you need help and you think I can provide that (I > am not subscribed to this bug). Checking the logs the failures seem to be infrastructure issues, most often temporary problems with downloading files from snapshot.debian.org (via local proxy when it is present). IMO those should be fixed by monitoring the logs and retrying the test automatically rather than working around the issues in u-u/test itself. An even better fix would be fixing the proxy/snapshot.d.o, but this may be out of scope for the autopkgtest infrastructure. Please note that a major reliability bug got fixed in 1.1 which caused a lot of flakiness and crashes in the field, too. > Recent discussion of gating migration by autopkgtests on debian-devel² > noted that if this is going to work, and in particular if we are going > to *block* migration when it causes autopkgtest regressions rather than > merely delaying it, intermittent autopkgtest failures are likely to have > to be considered RC due to their impact on the tested package's > dependencies; for now I've filed it as important. I support making the delay very long or even blocking migration, but based on my experience flaky tests can easily be caused by infrastructure issues, and in those cases important would still be a more reasonable severity as infrastructure issues tend to persist for long time (unfortunately). > > Simon McVittie has previously proposed³ a way to mark autopkgtests as > unsuitable for gating CI, while still having them available for > convenient manual testing (and optionally run, but with their results > disregarded, on the real CI infrastructure so that their reliability can > be assessed), but this isn't currently possible. If the autopkgtest > can't be made reliable then it would make sense to disable it for the > moment. I believe the current reliability level would not warrant disabling this test since u-u does not have a lot of dependencies and re-triggering the test is easy. I also find the Britney overrides a good way of tackling with flaky/failing tests following Martin's recommendation in ³. Cheers, Balint > > Paul > > ¹ https://ci.debian.net/packages/u/unattended-upgrades > ² https://lists.debian.org/debian-devel/2018/05/msg00061.html > ³ https://bugs.debian.org/851558 > > https://ci.debian.net/data/autopkgtest/unstable/amd64/u/unattended-upgrades/250960/log.gz > > I: Retrieving InRelease > I: Retrieving Release > I: Retrieving Release.gpg > E: Failed getting release signature file > http://snapshot.debian.org/archive/debian/20171001T000000Z//dists/sid/Release.gpg > autopkgtest [05:43:53]: test upgrade-between-snapshots: > -----------------------] > > > https://ci.debian.net/data/autopkgtest/unstable/amd64/u/unattended-upgrades/220435/log.gz > > Get:391 http://snapshot.debian.org/archive/debian/20171001T000000Z > sid/main amd64 xserver-xorg-legacy amd64 2:1.19.3-2 [2069 kB] > E: Failed to fetch > http://snapshot.debian.org/archive/debian/20171001T000000Z/pool/main/x/xorg-server/xserver-xorg-core_1.19.3-2_amd64.deb > 503 Backend fetch failed [IP: 193.62.202.30 80] > E: Unable to fetch some archives, maybe run apt-get update or try with > --fix-missing? > Fetched 189 MB in 16min 5s (195 kB/s) > autopkgtest [06:15:55]: test upgrade-between-snapshots: > -----------------------] > > > https://ci.debian.net/data/autopkgtest/testing/amd64/u/unattended-upgrades/226860/log.gz > > ls: cannot access > '/tmp/autopkgtest-lxc.eglx_gdx/downtmp/autopkgtest_tmp/chroot': No such > file or directory > I: Retrieving InRelease > I: Checking Release signature > I: Valid Release signature (key id 126C0D24BD8A2942CC7DF8AC7638D0442B90D010) > I: Retrieving Packages > I: Retrieving Packages > I: Retrieving Packages > I: Retrieving Packages > E: unknown location unstable/dists/sid/main/binary-amd64/Packages.xz > autopkgtest [08:24:49]: test upgrade-between-snapshots: > -----------------------] > > > https://ci.debian.net/data/autopkgtest/testing/amd64/u/unattended-upgrades/225434/log.gz > > autopkgtest [02:09:26]: test upgrade-between-snapshots: - - - - - - - - > - - results - - - - - - - - - - > upgrade-between-snapshots FAIL stderr: ls: cannot access > '/tmp/autopkgtest-lxc.4kr__7dt/downtmp/autopkgtest_tmp/chroot': No such > file or directory > autopkgtest [02:09:26]: test upgrade-between-snapshots: - - - - - - - - > - - stderr - - - - - - - - - - > ls: cannot access > '/tmp/autopkgtest-lxc.4kr__7dt/downtmp/autopkgtest_tmp/chroot': No such > file or directory >
