In message <[EMAIL PROTECTED]>, "SteveB" wri
tes:
>With commercial software (well at least the places I worked) nothing
>could go out the door without a complete QA cycle performed on it.
Yes. This is why the open systems have "releases" every so often; a
release has been run through something more like a QA cycle. The QA
cycle is where the naive fools run "-current" believing it will have
"new features". :)
>Even the smallest of bug fixes couldn't be released without a QA
>cycle. A full QA cycle was time consuming and expensive, so fixes sat
>until there was a stack of them to QA'd as a group or they had to wait
>until next upgrade. That way we knew state of the product. Yes, the
>state of the product would include known bugs. The key was a known bug
>and a known documented bug was as valuable as a fix. Sure a bug is
>bad, but if it is documented you don't waste trying to make something
>work that is known to be broke.
But you can't *do* anything. Imagine a known bug "doesn't run on Pentium
or later systems". That's pretty much totally crippling now.
The important point is that you get the choice. You can run a stable release,
with known bugs, or you can run slightly less tested code which fixes them.
>So who is testing these fixes in open source world? Just seeing if
>the problem at hand is gone isn't real testing, even claiming
>thousands of people are now using it isn't enough. There can still be
>lurking potentially data destroying bugs lurking.
Yes. But that's just as true of a full QA cycle. Safety, in software,
is an analogue signal, not a digital one. My experience (and I admit,
I'm mostly from a NetBSD background) is that -current releases are
dramatically more reliable, and less buggy, than commercial software.
Testing, alone, does not catch bugs. *Analysis* does, and one of the
things the open source community shines at is having a fix *analyzed*
by a number of people.
>In the open source
>world is there a official QA process or group. Is there a FreeBSD
>test suite that releases go through. QA is unglamorous work, but
>needs to be done.
I don't know about the "official" process, but I will tell you that I'd
rather have my life depend on FreeBSD-current than on Windows NT, despite
the "QA cycle".
There are many ways to do effective QA.
-s
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
