The current system for managing bugzilla priorities has a major problem,
in that it does not identify bugs that reasonably cannot be fixed before
the release.
The current set of priorities is in practice like this:
- P1: most wrong code bugs, and other absolutely blocking problems
- P2: problems worth a look on important platforms
- P3: uncategorized
- P4: problems worth a look on less important platforms
- P5: other
The problem with this set is that while P1 bugs will absolutely be fixed
before the release (and backported usually), P2 bugs are a one-catch-all
group for everything else that's worth looking at. It is impossible to
distinguish stuff that will probably be fixed before the release (and
presumably backported to all branches), and what instead requires new
stage1/stage2 material.
As a result, the release criteria we have are not really a measure of
the quality of the release, and especially are not really a measure of
the work being done towards a release.
I propose two solutions to this problem.
- The more conservative one is to use more aggressively the release
milestone field. Hard-to-fix bugs would be left as P2, but bumped to
the next major release at the beginning of stage 3.
Advantages: no need for churn in the bug database---very easy to implement
Disadvantages: the milestone field is not visible in search lists (maybe
this can be changed)?
- Alternatively, we could add a new priority "P--" for uncategorized
bugs, and split P2/P3 like this: P2 bugs will be fixed in stage 3/4, P3
bugs will most likely be postponed to stage 1/2.
Advantages: quicker impression from the bug searches, especially during
bug triage
Disadvantages: need to rethink bugzilla queries
I think any of these two approaches would provide a serious added value
to judging a release quality. Meeting the release criteria ("no more
than 50 P2 regressions") in the past included the release managers
downgrading bugs from P2 to P4, which is in my opinion cheating. In the
proposed scheme, this would be less necessary, because the release
criteria could take into account a broader view, such as respectively
for the two approaches:
- At most 60 P2 regressions, of which at most 15 should have release
milestone 4.4.0.
- No more than 15 P2 regressions and 45 P3 regressions.
Any opinions?
Paolo
