Over the last few months, I've found myself struggling to find a simple way to describe our approach to QA to friends and colleagues. I reached out to lamby a week or so ago and he suggested I brought it to -devel.
I don't think Debian's approach to QA needs to look like that of a discrete software product or a large development consulting business or a small team of people sitting next to each other. Debian, being a universal operating system, is understood to have lots of software, lots of architectures, lots of use cases, lots of teams, etc. It does help to keep some analogies in mind, though. The three main concepts I'm struggling to convey and where I'd love this community's input are: (1) QA at Debian is a multi-stakeholder process* where ftp-master, QA, Release and other teams play a role but core accountability lies with the package maintainer, (2) The preferred vehicle for QA accountability in Debian is a bug report, and, (3) The QA "bar" is codified in Debian Policy but also in many other places (maybe not comprehensively) including for example the Release checklist I qualified "process" with an asterisk above as I'm not sure if we have documented a single QA process. It looks like there are places where Policy is enforced (e.g., ftp-master), places where testing automation is happening (e.g., lintian, piuparts, chroot/d-i runs in jenkins.d.n), campaigns (such as the recent NMUs for reproducible builds) and "final checkpoints" (e.g., release checklist) but I'm not sure if this is all mapped and documented as a single process (and, if so, if it's updated since a lot has happened in 10 years) Of course, for folks that live in a CI/CD environment where the build log and the stop light are the vehicles of accountability, the concept of a piuparts run happening after you've uploaded and getting a bug report that you then go address and "start over" is almost foreign to them. Also, that Policy codifies many sanity checks but not things that some organizations do like source code audits, etc., is also novel since many of those organizations have "single source of truth" docs that codify the "QA bar". Yet most people acknowledge Debian ships quality releases even without an approach to QA that "looks like" other use cases in the industry, so no prejudice there. Any reactions to educate me and help me formulate a more complete view would be appreciated. Thanks, bureado (not on this list) PS: it would also be interesting to analyze vs. other distros. Other distros are less universal, may or may not have highly distributed teams, may only test for a few use cases, etc., so it's clear it won't be apples to apples, but with things like OpenQA having some mindshare I think it's an interesting exercise too.
