On May 8, 2004, at 4:05 AM, Jim Jagielski wrote:
I don't consider us a "closely held ivory-tower QA" and I would say that if anyone knows of a talented pool of users would would like to test RCs, then we should have a mechanism to use them. That was the intent for the current/stable-testers list, but we've never really used that as we should have.
The problem is really 2 fold:
1. The tarballs were being mistakenly described as the official release. It's not released until we say so. I think it's our responsibility to ensure that people aren't mistakenly running pre-release s/w under the impression that it is release.
I agree that it was mistakenly described as an official release, but I think we're confusing the idea of an official release with the concept of quality. It's difficult for us to state the quality of a release when our license[1] clearly says that we make no guarantees or warranties. Even if we say it's perfect, it's still a "use at your own risk" type of thing.
2. That when all goes well, and the RC tarballs are approved,
they aren't changed at all... We are testing, really,
the accuracy of the tarball itself. This add some
complexity to the whole process.
I don't think it needs to add to the complexity. I see us wanting to do simple sanity checking on the tarball, but even that lends itself to scalability. Assuming the tarballs are labeled as release candidates, people should be free to do whatever they want with them. If the signatures don't check out, they'll tell us. If it doesn't build, they'll tell us. If it works out for a few days we'll bless it. When it breaks we'll fix it and roll another tarball.
I've been thinking over changing the 1.3 release process and us actually tagging a tree as RC, creating actual 1.3.x-rc1 tarballs and people testing that, and having those very, very open, but having the actual *release* tarballs somewhat closed (again, to test the viability of the tarball, not the code).
I think this would be a step in the right direction. I still don't see why any stage in the release process should be closed, though. We don't make any guarantees about any of our code at any time, so as long as we make it totally clear when we want sanity checking (testing a release candidate) or when we want normal bug testing then I think we may see much greater participation by our users in the QA process, and as a result we will all have much higher quality code.
-aaron
[1] - Excerpt from http://www.apache.org/LICENSE.txt
* THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE * DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE.
