Jim Fulton wrote:
On Sep 12, 2006, at 8:40 AM, Martijn Faassen wrote:
[snip]
One problem we have is getting things to be tested. It hardly
motivates people to test for and report bugs if their reports
don't affect he release. I think we have a serious problem that
needs to be addressed. I don't think the right way to address it
is to release despite known serious bugs. Note that some
judgement goes into considering whether a bug is serious enough
to block a release. We don't block a release for just any bug.
Before a release, bug triaging needs to take place.
Which we do.
I know, but perhaps we need to be a bit more aggressive. Christian
Theune's point about empowerment to defer a bug may be useful. We may be
a bit too fearful sometimes to defer bugs right now.
I recall you saying we defer bugs too often and bugs never get
fixed, so we should stop doing any feature work until all bugs are
fixed, etc.
We should. We have yet to do this. As I mentioned in another note,
we should prevent new features at the beginning of a release cycle
until we've caught up on past bugs.
Trying to address old bugs was not responsible for the delays in the
3.3 release.
I think it contributed, though fully agreed it isn't the only source of
the delay.
I call that perfectionistic.
Then your idea of perfection and mine are far apart. Letting bugs
languish for months or even years is not acceptable. Ignoreing bugs
reported during beta testing, when we get too little testing to
begin with is unacceptable.
*Ignoring* bugs is unacceptable. Explicitly deferring a bug for months
is acceptable, as there are always more bugs than we can fix, and we
should fix the bugs of the highest priority. A low-priority bug may
therefore languish. I'm not saying we are doing this correctly *now*,
but such would be a reasonable process.
I think we have been blocking this release with a selection of bugs
that included quite a few that weren't absolutely critical.
Name a few.
Good point. Going through the bugs fixed over the last period makes them
seem all important to me or alternatively, lesser important bugs fixed
by the reporter themselves. Perhaps a hard-headed release manager who
will say "important schrimportant" and defers the issues anyway would've
been useful.
Still:
http://www.zope.org/Collectors/Zope3-dev/553
http://www.zope.org/Collectors/Zope3-dev/576
http://www.zope.org/Collectors/Zope3-dev/540
http://www.zope.org/Collectors/Zope3-dev/530
http://www.zope.org/Collectors/Zope3-dev/537
http://www.zope.org/Collectors/Zope3-dev/583
http://www.zope.org/Collectors/Zope3-dev/534
Some issues were only discovered way after we should've done the
release. We've been
fixing these as part of the release process. All of these issues appear
to be reported after mid-july or later, though. I think those are good
bugfix release candidates, and shouldn't have been blocking our 3.3
release (not all of them are marked critical, but these all have been
fixed):
http://www.zope.org/Collectors/Zope3-dev/677
http://www.zope.org/Collectors/Zope3-dev/675
http://www.zope.org/Collectors/Zope3-dev/676
http://www.zope.org/Collectors/Zope3-dev/679
http://www.zope.org/Collectors/Zope3-dev/674
http://www.zope.org/Collectors/Zope3-dev/683
http://www.zope.org/Collectors/Zope3-dev/681
http://www.zope.org/Collectors/Zope3-dev/683
http://www.zope.org/Collectors/Zope3-dev/682
http://www.zope.org/Collectors/Zope3-dev/680
http://www.zope.org/Collectors/Zope3-dev/673
http://www.zope.org/Collectors/Zope3-dev/690
http://www.zope.org/Collectors/Zope3-dev/696
http://www.zope.org/Collectors/Zope3-dev/691
http://www.zope.org/Collectors/Zope3-dev/697
Since these all were reported in july or later, we couldn't possibly
have fixed them if we'd have released in june. :)
I would suggest we triage bugs a lot more aggressively instead. I
say this as someone who spent a few afternoons staring at Zope 3
bugs trying to get them out of the way.
I've spent way more than a few afternoons.
Gratefully acknowledged.
I can think of a number of ways to approach this problem: 1. Do
less frequent releases.
I do not believe this helps a lot for bug fixes. If we have twice
the release period, people won't start fixing bugs twice the amount
of time in advance,
Right. This, alone is not a solution.
and we won't get twice the volunteers either.
No, but it will halve the work for the small existing group of
volunteers.
I do not believe this to be necessarily the case. The list of bugs fixed
that were reported after our supposed june release is one reason why I
think that. The other reason is that there'd be two times as much space
between bugfixing rounds, which will make bugs languish and people get
out of the habit of fixing them.
2. Feature freeze the trunk until the previous release has made
it to release candidate status
You mean don't branch the trunk (and thus let it be the release
branch) until the release has made it to release candidate status?
Yes. I think it is a shame to have to do this, but I think this is
now necessary.
I think this is a good idea.
I would go further. I would not unfreeze the trunk until until we've
cleaned up all open bugs, either by fixing them or rejecting them.
-1
3. Release less. I think it's time to start thinking of some
sort of "core" Zope 3 that we can manage with the very limited
number of volunteers we have now.
+1, though I wonder how much this has been blocking the release -
important bugs that could block releases don't tend to be in out of
the way components, one would think.
I spent a lot of time on crap that wasn't core at all.
So why were these critical issues? What happened during triaging?
4. Get more volunteers.
Getting a release out is *difficult* and the amount of volunteers
available, while important, is only partially related. More
volunteers won't speed it up tremendously much and can even slow
things down. Plone has many volunteers and has had much problems in
the past doing timely releases. Silva has had far less volunteers
and can do more agile releases, because there's less people to
manage.
And because it is much smaller.
Yes, smaller helps.
I think we're far from having too
many volunteers on Zope 3 maintenance. I'm not optimistic that we'll
get more volunteers, which is why I'm inclined to release less.
I'm coming from the perspective of trying to work best with the existing
volunteers as well.
These are just some ideas. But something has to give and I don't
think it should be responsible bug fixing prior to release.
Large Zope 3 projects have been working against 3.3 for many months
now. If there was any absolute showstopper in Zope 3.3, why have
these projects successfully transitioned?
It all depends on how large you want the audience to Zope 3 to be. A
small community on a limited number of platforms can work around
well-known problems.
I think such a larger audience needs to grow organically and has fairly
little to do with the bugfixing issues.
Who or what would have been hurt exactly had Zope 3.3 been released
in june?
Well, we would have been hurt badly as the first beta release, the
one made at around that time was badly broken.
Who is we? Why would we have been hurt? Even if such a badly broken beta
would've gone out (which I was not suggesting, obviously we do *some*
testing), we could've done a bugfix release.
There were some serious bugs.
Agreed.
We would have demonstrated pretty clearly that we don't care about
quality.
To whom? What if we'd have followed up with bugfix releases? What about
demonstrating we can release when we say we will?
I don't think it's Zope 3's reputation that would've been hurt, as
Zope 3.3 in june was not *that* buggy. Bugfix releases are also a
completely accepted practice.
Except that we don't do much of those and we put little effort into
the ones we do do.
Let's do more bugfix releases. I don't think they can be avoided anyway.
I still think our quality standards for a release have been too
high. Getting people to fix more bugs is good, sure, but perhaps we
should separate this at least somewhat from the release itself.
Sorry, I agree very much. I'd be willing to defer to a Zope 3
release manager, if there was one.
I'm not sure what 'sorry, I agree very much' means.
If we're going to do timebased releases, we should have a button
somewhere that says 'make a release', and a date on which the
button is pressed. If the release is botched, we can press the
button again for bugfix releases, and we have 6 months in which to
do a better job next time.
IMO, that would be a disaster.
In what way? To whom?
Regards,
Martijn
_______________________________________________
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
