superficial tests that pass are marked as neutral. In Debian's implementation, this means that a package that only has superficial autopkgtests will NOT get the autopkgtest "bounty" which reduces the number of days a package must wait in Unstable before automatically migrating to Testing. (The bounty is set to 3 so a typical upload will take 2 days instead of 5 days).
https://salsa.debian.org/ci- team/autopkgtest/-/blob/master/doc/README.package-tests.rst https://salsa.debian.org/release- team/britney2/-/blob/master/etc/britney.conf#L81 However, if a superficial autopkgtest fails, as happens with libwacom 2.9.0-1 , it must be treated as a failure. A migration-reference/0 of neutral should not allow a failing autopkgtest to migrate. As of today, libwacom is still not fixed and is only held in proposed by my block-proposed bug so I think this is a clear test case right now. -- You received this bug notification because you are a member of Canonical's Ubuntu QA, which is subscribed to Auto Package Testing. https://bugs.launchpad.net/bugs/2035101 Title: autopkgtest failures are ignored, apparently for packages with only superficial tests Status in Auto Package Testing: New Status in britney: New Bug description: Example 1 ========= Yesterday, I noticed that gnome-metronome 1.3.0-0ubuntu2 was allowed to migrate from mantic-proposed to mantic despite its own autopkgtest failing on every architecture. This migration happened after an automatic migration-reference run happened which returned the result "neutral". "neutral" is expected for superficial autopkgtests like the one in gnome-metronome. Before that migration-reference run, the failures were correctly shown on the excuses page. Screenshots attached. https://autopkgtest.ubuntu.com/packages/gnome-metronome/mantic/amd64 (I then uploaded 1.3.0-0ubuntu3 which fixed the autopkgtest regression) Here's the gnome-metronome autopkgtest code: https://salsa.debian.org/a-wai/gnome-metronome/-/blob/debian/master/debian/tests/control Example 2 ========= https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_excuses.html#webkit2gtk does not show that the 2.41 series of webkit2gtk triggered an autopkgtest failure for devhelp https://autopkgtest.ubuntu.com/packages/devhelp/mantic/amd64 However, Debian's britney is showing the failure: https://release.debian.org/britney/pseudo-excuses-experimental.html#webkit2gtk Here's the devhelp autopkgetst code: https://salsa.debian.org/gnome-team/devhelp/-/blob/debian/latest/debian/tests/control Theory ====== My theory is that failures are being ignored for packages which only have superficial autopkgtests, like for devhelp and gnome-metronome. Possibly, this issue has been fixed in Debian's copy of britney already To manage notifications about this bug go to: https://bugs.launchpad.net/auto-package-testing/+bug/2035101/+subscriptions -- Mailing list: https://launchpad.net/~canonical-ubuntu-qa Post to : [email protected] Unsubscribe : https://launchpad.net/~canonical-ubuntu-qa More help : https://help.launchpad.net/ListHelp
