2012/4/16 Andrea Pescetti <pesce...@apache.org> > On 15/04/2012 Rob Weir wrote: > >> On Sun, Apr 15, 2012 at 11:49 AM, Andrea Pescetti wrote: >> >>> Historically, OpenOffice.org produced a numbered Release Candidate >>> (OpenOffice.org 3.3 had ten, RC1 to RC10) that was made available to the >>> community exactly for the purpose of looking for unknown bugs. QA >>> activities >>> were planned for the RC phase and, after a few days of availability, a RC >>> was approved as final or rejected (and the Release Manager could, and at >>> times did, include fixes for previously known bugs that had been >>> accumulating and that were significant; none of them was a blocker in >>> itself, but each of them would have caused problems to users, so their >>> combined effect was blocking the release). >>> >> That is what we've been doing with the dev snapsots builds. Surely >> you've seen the QA work Lily has done with testing them? >> > > Sure, and it was great work. But those tests were run on versions that are > now quite outdated. Example: the spell check test asks you to verify that > no spell check is available and that dictionaries have been removed > accurately, while we all know that things are quite different in current > builds. >
How do you means "tests were run on versions that are now quite outdated"? We are doing regressiont testing continuously. Switch build do different feature regressiont esting. For RC build, I don't think the fulll regression is reasonable, more important, it is instalation, extenson, plus some small regressiont testing. The full regressionte testig should be done before one RC build is ready for vote. > > The traditional OpenOffice.org Quality Assurance process had two features > that I'd like to keep: > > 1) Full QA is run on what we release. We need to ensure that OpenOffice > works now, not that previous builds worked. > Agree if the "Full" here means test matrix on full platforms and full langauges. The "OpenOffice" you means here I understand is AOO. For AOO, the problem is we lost manual test cases, so general regression testing is done against AOO builds. I belive these testing results can be taken as evidence of what AOO works now. > > 2) The community at large is involved in testing, in a specific short > period just before the release. If "Release Candidate" has a different > meaning now, call it the "Pre-Release Phase", that will end with a "Release > Candidate" we can vote on rather confidently. We can't rely on volunteers > doing QA on a regular basis, but we can very easily find volunteers willing > to stress-test a "nearly final" version in a well defined timeframe (1-2 > weeks). > I agree you on this point ans it is what we do now. You may check the RC > build testing plan. QA volunteers will testing RC build for one week, > meanwhile, all volunteers can do stress-test against RC build before he/she > gives vote. > Lily > I would keep this as a basis for discussing how to structure QA activities > in future (after 3.4), if others agree. > > Regards, > Andrea. >
