Hi all, I've just pushed a bunch of commits to our deployed branch of autopkgtestĀ¹ that bring us up to date with upstream, where the developers have done a lot of work. Many thanks to them. The most interesting thing to subscribers to this list will be the changes - mainly additions - to the list of known test restrictions. I've pasted the diff of README.package-tests.rst below.
I don't currently expect any breakage to result from this, but if you notice anything weird please reply here or come to #ubuntu-release and we'll take a look. Cheers, -- Iain Lane [ i...@orangesquash.org.uk ] Debian Developer [ la...@debian.org ] Ubuntu Developer [ la...@ubuntu.com ] [1] https://code.launchpad.net/~ubuntu-release/autopkgtest/+git/development/ @@ -237,14 +245,84 @@ needs-reboot The test wants to reboot the machine using ``/tmp/autopkgtest-reboot`` (see below). -needs-recommends +needs-recommends (deprecated) Enable installation of recommended packages in apt for the test dependencies. This does not affect build dependencies. + Don't use this for new tests, we want to remove it altogether. + See https://lists.debian.org/debian-ci/2018/06/msg00016.html + and bug 896698 for the rationale. + +flaky + The test is expected to fail intermittently, and is not suitable for + gating continuous integration. This indicates a bug in either the + package under test, a dependency or the test itself, but such bugs + can be difficult to fix, and it is often difficult to know when the + bug has been fixed without running the test for a while. If a + ``flaky`` test succeeds, it will be treated like any other + successful test, but if it fails it will be treated as though it + had been skipped. + +skippable + The test might need to be skipped for reasons that cannot be + described by an existing restriction such as isolation-machine or + breaks-testbed, but must instead be detected at runtime. If the + test exits with status 77 (a convention borrowed from Automake), it + will be treated as though it had been skipped. If it exits with any + other status, its success or failure will be derived from the exit + status and stderr as usual. Test authors must be careful to ensure + that ``skippable`` tests never exit with status 77 for reasons that + should be treated as a failure. + +skip-not-installable + This test might have test dependencies that can't be fulfilled on all + architectures. Therefore, when apt-get installs the dependencies, it will + fail. Don't treat this as a test failure, but instead treat it as if the + test was skipped. + +hint-testsuite-triggers + This test exists purely as a hint to suggest when rerunning the + tests is likely to be useful. Specifically, it exists to + influence the way dpkg-source generates the Testsuite-Triggers + .dsc header from test metadata: the Depends for this test are + to be added to Testsuite-Triggers. (Just as they are for any other + test.) + + The test with the hint-testsuite-triggers restriction should not + actually be run. + + The packages listed as Depends for this test are usually indirect + dependencies, updates to which are considered to pose a risk of + regressions in other tests defined in this package. + + There is currently no way to specify this hint on a per-test + basis; but in any case the debian.org machinery is not able to + think about triggering individual tests. + +superficial + The test does not provide significant test coverage, so if it + passes, that does not necessarily mean that the package under test + is actually functional. If a ``superficial`` test fails, it will be + treated like any other failing test, but if it succeeds, this is + only a weak indication of success. Continuous integration systems + should treat a package where all non-superficial tests are skipped as + equivalent to a package where all tests are skipped. + + For example, a C library might have a superficial test that simply + compiles, links and executes a "hello world" program against the + library under test but does not attempt to make use of the library's + functionality, while a Python or Perl library might have a + superficial test that runs ``import foo`` or ``require Foo;`` but + does not attempt to use the library beyond that. + Defined features ---------------- -There are no currently defined Features. +test-name + Set an explicit test name for the log heading and the ``summary`` file + for a ``Test-Command:`` inline test. When not given, these are + enumerated like ``command1``.
signature.asc
Description: PGP signature
-- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel