On Sun, 2020-04-12 at 10:43 +0200, Agostino Sarubbo wrote: > Hello all, > > If you work on the stabilization workflow you may have noticed that: > > - There are people that rant if you don't run src_test against their packages; > - There are people that rant if you open a test failure bug against their > packages and you block the stabilization. > > So, unless there will be a clear policy about that, I'd suggest to introduce > a > way to establish if a package is expected to pass src_test or not. > > A simple way to achieve this goal would be: > introduce a new bugzilla custom flag (like "Runtime testing required" we > already have) maybe called "src_test pass" or similar, that, by default is > yes > and the maintainer should set it to no when needed. > > In case of 'yes', the arch team member must compile with FEATURE="test" and > he > is allowed to block the stabilization in case of test-failure. > > In case there will be a test-failure there are two choices: > 1) The maintainer fixes the test failure; > 2) The maintainer does not want to take care, so he can simply remove the > blocker and set "src_test pass" to no. > > Both of the above are fine for me. > > Obviously, if there are already test-failure bugs open, the flag "src_test > pass" should be set to 'no' when the stabilization bugs is filled. >
This is not a problem that can be solved by a binary flag. If package's test suite is entirely broken and unmaintained, the package should use RESTRICT=test and not silently ask arch teams to ignore it. If package's test suite is only slightly broken, then I'd prefer saying 'please report but ignore *these* test failures' because I can't fix them right now but they don't seem major. Skipping the test suite entirely is not a solution because it doesn't disambiguate between 'few tests fail' and 'every single test explodes'. -- Best regards, Michał Górny
signature.asc
Description: This is a digitally signed message part
