Re: QA expectations (Was: Do we want to Require or Recommend DH)
On Wed, May 15, 2019 at 09:28:59AM +0100, Simon McVittie wrote: > Prior art: Ubuntu already does this, gating the transition with a version > of Debian's testing migration scripts that has been configured for a 0 day > delay for all urgencies. Yes. Colin Watson even had a talk about this in Vaumarcus. http://penta.debconf.org/dc13_schedule/events/1028.en.html Copying the strategy to Debian failed to gain traction thus far. I wonder whether we could do this opt-in (or maybe as some external service that forwards tested packages to the archive) anyhow. I have little clue about what would be required to implement it in the archive. > Ubuntu is more able to do this than Debian, because Ubuntu's slowest, > least reliable and least-well-supported architectures are faster, more > reliable and better-supported than Debian's. I'm not sure that we really > want to be waiting for important fixes (especially in large packages > like compilers, web browsers and the kernel) to build successfully on > mips(el), or requiring that their build-time tests have few enough race > conditions to be successful even on slower architectures, before they > can reach the part of the archive that developers use in practice? Given my experience with Multi-Arch: same skews in unstable, it does not appear to me that packages get stuck for that long. If they do, that's due to a FTBFS (which is exactly the thing we want to prevent from entering unstable). buildd speed is a concern for other reasons, so if an architecture fails to keep up for a longer time, it should likely be demoted to ports. Helmut
Re: QA expectations (Was: Do we want to Require or Recommend DH)
On Tue, 14 May 2019 at 18:19:47 +0200, Johannes Schauer wrote: > So maybe instead of creating unstable-proposed, stuff should move from > buildd-unstable to unstable only after it successfully passed all kinds of > automatable QA tests? Prior art: Ubuntu already does this, gating the transition with a version of Debian's testing migration scripts that has been configured for a 0 day delay for all urgencies. The down side of this is that families of packages occasionally get stuck in their incoming-equivalent because of versioned dependencies, and a transition is needed to get them into their testing-equivalent. Perhaps some Ubuntu developers could comment on the extent to which this is a practical problem? > It could also have other nice properties that currently only testing has, > like no Multi-Arch:same version skews because stuff could only move to > unstable > after it has been built on all arches. Ubuntu is more able to do this than Debian, because Ubuntu's slowest, least reliable and least-well-supported architectures are faster, more reliable and better-supported than Debian's. I'm not sure that we really want to be waiting for important fixes (especially in large packages like compilers, web browsers and the kernel) to build successfully on mips(el), or requiring that their build-time tests have few enough race conditions to be successful even on slower architectures, before they can reach the part of the archive that developers use in practice? smcv
Re: QA expectations (Was: Do we want to Require or Recommend DH)
On Tue, May 14, 2019 at 05:19:23PM +0200, Helmut Grohne wrote: > Let me briefly hijack the discussion for a side note. ;) > > On Tue, May 14, 2019 at 11:30:11AM +0200, Andreas Tille wrote: > > NMUers should do debdiff - no matter what change was done. And yes, it > > happened also to me in the past once or twice that I uploaded an empty > > package or package missing some files. I do not remember whether it was > > connected to a change to dh or not ... but mistakes happen. The fact > > that we might end up with broken NMUs is IMHO orthogonal to any cdbs to > > dh change. > > After a failure, we tend to bash uploaders: > * Why didn't they look at the debdiff? > * Why didn't they run piuparts? > * Why didn't they run lintian? Not running lintian before uploading to unstable shouldn't happen. > * Why didn't they run autopkgtest? > * Why didn't they do an arch-only/indep-only build? > * Why didn't they test their package? > * ... > > The things you have to remember before doing an upload are insane. >... "Check that your changes are working" is pretty basic and sane. If the only change was fixing a missing dependency, check that the built binary package actually has the correct dependency added. If you rewrote the whole build system, check that the built packages still have the same contents and still work. If you are trying to fix a build failure on an architecture, make a testbuild on a porterbox instead of 10 uploads to unstable. >... > People shouldn't have to remember all the QA. QA should just work and > QA should tell people about the (unusual) failures. >... QA is good for finding some kinds of common problems. But it is not a replacement for people being careful when making changes. > Helmut cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed
Re: QA expectations (Was: Do we want to Require or Recommend DH)
Quoting Holger Levsen (2019-05-14 17:38:15) > > Now one can turn this argument upside down. One can say: unstable is the QA > > area. Britney prevents testing migration on autopkgtest/piuparts/ missing > > binaries. In that case, we should simply stop filing such things in the BTS > > and stop doing manual QA on unstable. It should be ok to break unstable. > > But this is not going to work with transitions. Thus I still think we're > > doing it wrong and unstable isn't the place to do the QA we expect from > > everyone. > > have uploads go to unstable-proposed and then, after basic automatic QA > checks, go to unstable? (and then testing as usual today...) Doesn't a repository where all binary packages go before they are pushed into unstable already exist? deb http://incoming.debian.org/debian-buildd/ buildd-unstable main So maybe instead of creating unstable-proposed, stuff should move from buildd-unstable to unstable only after it successfully passed all kinds of automatable QA tests? Such an intermediate repository (be it unstable-proposed or buildd-unstable) could highly improve the quality of unstable and make sure that whatever lands in unstable will only have those bugs that are usually discovered by humans only. It could also have other nice properties that currently only testing has, like no Multi-Arch:same version skews because stuff could only move to unstable after it has been built on all arches. Thanks! cheers, josch signature.asc Description: signature
Re: QA expectations (Was: Do we want to Require or Recommend DH)
On Tue, May 14, 2019 at 05:19:23PM +0200, Helmut Grohne wrote: > The things you have to remember before doing an upload are insane. > Having humans remember all this crap is not a reasonable expectation. I > think our upload process is a bit like classical debhelper: You remember > to do all the things. We've seen the argument that the dh sequencer > sheds light on the unusual aspects of a package. I argue that this > should apply to QA as well. People shouldn't have to remember all the > QA. QA should just work and QA should tell people about the (unusual) > failures. agreed. > Now one can turn this argument upside down. One can say: unstable is the > QA area. Britney prevents testing migration on autopkgtest/piuparts/ > missing binaries. In that case, we should simply stop filing such things > in the BTS and stop doing manual QA on unstable. It should be ok to > break unstable. But this is not going to work with transitions. Thus I > still think we're doing it wrong and unstable isn't the place to do the > QA we expect from everyone. have uploads go to unstable-proposed and then, after basic automatic QA checks, go to unstable? (and then testing as usual today...) -- tschau, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
