Hi Julien, On Sat, Dec 17, 2016 at 11:03:12AM +0100, Julien Cristau wrote: > On Tue, Jun 28, 2016 at 12:38:15 +0200, Guido Günther wrote: > > > Hi Julien, > > On Tue, Jun 28, 2016 at 11:46:06AM +0200, Julien Cristau wrote: > > > On Sun, May 29, 2016 at 14:58:51 +0200, Guido Günther wrote: > > > > > > > Upstream has an internal test suite which we enabled for the package > > > > builds in nspr as well as nss (+ some autopkg smoke test in nss > > > > itself). Howver I don't know as to what extend ABI compatibility is > > > > tested upstream. Hopefully Mike (cc:) may have some input on this. > > > > > > > > In order get some ideas about ABI compatibility myself I ran > > > > abi-compliance-tester. The results for both NSS and NSPR are also > > > > attached. We would do that on all point release updates. > > > > > > > > Note that the only (as to my understanding) serious regression has been > > > > pointed out by Florian as well: > > > > > > > > https://lists.debian.org/debian-lts/2015/11/msg00037.html > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1260698 > > > > > > > > and it's unclear if this part of the ABI. The API break (removal of > > > > CERT_FindCertURLExtension) is bogus since the symbol was not exported. > > > > > > > I don't understand why you seem to be talking about ABI stability > > > issues. There are other kinds of bugs. > > > > The "other kind of bugs" were the reason why I enabled the test suites > > in nss/nspr and started to add some autpkg tests. What else would you > > expect? > > > > We could also add (and run) autopkg tests for reverse dependencies of > > nss/nspr over time. > > How many of the nss/nspr reverse dependencies in stable have meaningful > autopkgtests? Overall my confidence in our QA abilities for stable > updates is pretty low, so I'm not wild about routinely upgrading > libraries to major new versions. Regressions in stable point releases > that need follow-on fixes are a major PITA.
Some of the package have autopkgtests but at least the bigger ones I looked at (389-ds, ceph icedove and firefox) don't have coverage in that area but I think it's a chicken and egg problem. Some of the packages have internal test suites which could be used to gain confidence. And having icedove and firefox not switch to embedded copy soon after a release would also help to detect breackage. I'm not advocating the pick up _every_ new upstream versions by all means but some of the security fixes are not doable in years old versions and having stable-proposed-updates gives the packages at least some exposure that is not available for packages going via security master (like the recent nss 3.26). Cheers, -- Guido