Hi Julian > > As already noted, we really do need to move to a QA per-revision system > longer-term. > > As for qa-deployable, can you summarise the problems with using qa-ok? I > didn't see any hard issues in your email but then I might have been reading > it > wrong. You left me tantalisingly close to understanding why when you said: > >> marking a bug as >> qa-ok/fixed when it is not is misrepresenting the true state of the bug >> and has obvious downstream implications for process, deployment >> readiness and also metrics gathering." > > but it would be great to enumerate these implications! >
My reply to Aaron's recent email sums it up a bit. It's the fact that we seem to be confusing the meaning of the tag from a release management perspective vs a QA (verification) perspective. As a release manager you want to know if a revision is deployable. From a QA perspective, you want to know that the revision is verified as solving the defect identified by the bug and hasn't caused any regressions. These are overlapping but different concerns. Given the expected usage of the qa tags seems to be to indicate deployability rather than the QA meaning (for want of a better term), my contention is: - the tags are misnamed: "qa-ok" has a certain implied meaning - they are misdefined to match the implied QA meaning: eg the definitions of qa-ok and qa-bad on QAProcessContinuousRollouts talks about the QA aspect So if a revision is marked as qa-ok because it can be deployed, but doesn't actually fix the problem is was designed to fix, and given the *current* written definition of what qa-ok means on QAProcessContinuousRollouts, then we would be misrepresenting that the bug is fixed when it's not and so metrics gathering around number of open defects, regressions etc could/would be compromised depending on how such metrics were gathered. It could cause confusion also if someone ran a report "+bugs?field.tag=qa-ok" expecting to see bugs which were fixed. Read in conjunction with Aaron's and my discussion on this thread, I hope that makes more sense now :-) _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
