Hi Paul, On Thu, Apr 16, 2020 at 10:30:17AM +0200, Paul Gevers wrote: > > Hmmmm, what exactly means "superficial". > > Please read the documentation: > https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst
Thanks for this pointer. > > Are all those > > Testsuite: autopkgtest-pkg-* > > > > superficial? > > No. Only those that only have superficial tests. E.g. ruby runs the full > upstream test-suite automatically. Ahhh! > > Do they qualify for early testing migration or not? > > Superficial tests *are* neutral, so no. > > > Wouldn't it be more informative to have a fourth category > > > > pass > > superficial > > neutral > > fail > > No, because there are only three states. I.e. superficial and flaky and > skipped tests all end up meaning the same. > > > My intention was to have a list with packages of our team where either a > > test is missing or failing. My idea was that some autopkgtest-pkg-* is > > "test is not missing". What is your opinion as debian-ci team about my > > idea? > > It seems you want to be processing the message field then, but honestly > if there is an entry, "test is not missing". If there is no entry, "test > is missing". Failing is "fail". flaky tests are also "test is not > missing", skipped tests are also "test is not missing". Why would you > need all those states? You have the "message" field in your UDD schema > to check why you get the neutral state. I definitely want to expose the full message (that's why I insisted to put it into the table). My intention is to expose only those packages where some work needs to be done. Otherwise the list will be to long. I will think about the plan after receiving your information. Kind regards Andreas. -- http://fam-tille.de