Andrew Dunstan writes:
"Useful" is probably subjective. That list would at least be a good
place to start, though. What combinations of variables do you think we
would need?
First of all, I don't necessarily think that a large list of CPU/operation system combinations is going to help much. IIRC, this round of platform testing showed us two real problems, and both happened because the operating system version in question came out the previous day, so we could not have caught it. Much more problems arise when people use different versions of secondary packages, such as Tcl, Perl, Kerberos, Flex, Bison. So you would need to compile a large collection of these things. The problem again is that it is usually the brand-new or the odd intermediate version of such a tool that breaks things, so a "set and forget" build farm is not going to catch it. Another real source of problems are real systems. Weird combinations of packages, weird network setups, weird applications, custom kernels. These cannot be detected on out of the box setups. In fact, the regression tests might not detect them at all.
Hence the open-source community approach. Closed-source development teams can do all the above, with great effort. But by throwing out the code and have real people test them on real systems with real applications, you can do much better.
The fact that something doesn't find everything doesn't mean it is of no value. (Thinks of Scott Adams' nice example: "Your theory of gravity doesn't prove why there are no unicorns, so it is wrong." ;-) )
I don't believe there is a single "open source community" approach - open source projects all have differing ways of handling problems. At least 2 very significant open source projects I know of run build farms, notwithstanding that your objections should apply equally to them. Mozilla's is fairly centralised and very complex and heavy, but gives fairly immediate feedback if anything gets broken. Samba's is much lighter, distributed, and they still apparently see good value in it. (Samba uses a "torture test" - perhaps we need one of those in addition to the regression tests.)
Maybe it wouldn't be of great value to PostgreSQL. And maybe it would. I have an open mind about it. I don't think incompleteness is an argument against it, though.
cheers
andrew
---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
