On Wed, Aug 26, 2009 at 5:12 PM, Peter Eisentraut<pete...@gmx.net> wrote: > On ons, 2009-08-26 at 14:26 -0400, Robert Haas wrote: >> Sure, but an aimless mandate to do testing for 4 (or 8, or 12) months >> doesn't necessarily buy you much, either. I'm good at focused >> activity - but there was nothing focused about 8.4 beta that I could >> see. Maybe we need some kind of TestFest process. > > Yeah, exactly. I can't imagine end users would know what to do during > beta. Even assuming that you have release notes at the beginning of > beta, you can't expect people to go through every item and do a formal > test for it. Surely it's been tested before, else it would not be in > the release, right?
I would sure hope so. Testing features individually makes a whole lot more sense to me than testing the release as a whole. Just trying a bunch of random stuff and seeing if anything breaks is not a very productive activity. On the other hand, testing individual features is frequently very productive, but it's my understanding of the way PG does things that that is supposed to happen before the patch is committed. It appears to me that most of the really nasty bugs that have been found in 8.4.0 relate to one of the following three things, each of which seems to be related to multiple back-branch commits. 1. SEMI/ANTI join support. 2. running the bgwriter during recovery (infrastructure changes for recovery) 3. deadman switch Maybe some of these weren't tested well enough prior to commit? Or perhaps they're just more significant changes and therefore likely spots for rough edges. I think there is a lot of merit (as Andrew suggests) in running a production application on a beta version of the database just to see if anything funny happens. But insisting that all PG developers stop doing development to focus ONLY on that activity doesn't seem very reasonable: not everyone is well-placed to do that kind of experiment, or cares to do so. Conversely, there are many people who are NOT developers who ARE well-placed to beta test (for example, Kevin Grittner does a lot more testing than he does development, and I think there are few people on this mailing list who would argue that the quality of that testing is any less than kickass). ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers