On Tue, Jul 11, 2017 at 10:35 AM, Michael Palimaka <kensing...@gentoo.org> wrote: > On 07/12/2017 12:25 AM, James Le Cuirot wrote: >> On Tue, 11 Jul 2017 16:15:51 +0200 >> Kristian Fiskerstrand <k...@gentoo.org> wrote: >> >>> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote: >>>> On 07/11/2017 03:47 PM, Michael Palimaka wrote: >>>>> The main risk of breakage of a package moving from testing to >>>>> stable is always at build time anyway. >>>> >>>> citation needed >>>> >>> >>> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will >>> happily sign a third party public keyblock's UID using signature >>> subkey on smartcard, which results in useless signature that doesn't >>> have any effect, but the application builds fine. >>> >>> This means gnupg 2.1.21 is not a candidate for stabilization, but it >>> certainly builds fine. >> >> This is a good opportunity to remind ourselves what stable means. Are >> we referring exclusively to our packaging or are upstream issues taken >> into account too? 30 days seems like a reasonable time for any upstream >> issues to be reported. Unfortunately security issues mean that new >> releases sometimes get stabilised immediately. Ideally these releases >> would carry just the security fixes but that isn't always the case. >> > > I think we should consider both our packaging as well as upstream > issues, and I agree that for most packages 30 days in ~arch is enough > time to smoke out upstream issues. >
Agree. Arch testing is really more of a sanity test. I think there is some value in runtime testing, but perhaps it is a bit of a luxury compared to build testing. It isn't going to detect obscure bugs. The only way to do something like that is to have an extensive testing protocol for each package, and almost nobody can afford to do that let alone Gentoo. Upstream might be able to do it in some cases, though unfortunately not in our environment. To the extent that upstream supplies working automated tests we can try to leverage those. -- Rich
