http://dev.laptop.org/ticket/8106 makes an interesting, relatively easy-to-follow (even for nontechnical folks) case study on how testing works right now, since improving QA processes for the next release cycle has been a topic of discussion lately.
Note that the snags we hit were minor and easily fixable, but took a while to resolve - how can we fix this for next round? Here goes. Mel's Ideal scenario (vs what happened): 1. There'd be a test case attached to the ticket saying "Do this! If it passes, you've proved the feature works." (Reality: No test case. No problem - I wrote one up. Getting good testcases to happen is something the QA team should do in any case.) 2. But wait. To write the test case, we needed to find an out of date content bundle that we could upgrade. Ideally, there would be a list of available content bundles, you could find different versions of each content bundle, and install an old one. (Reality: I hunted around in vain; SJ, bless him, finally stepped in and gave me direct URLs to 2 bundle files, simultaneously noting that there was not, in fact, a good way to find that kind of download/history information yet.) 3. Ok, great - we've installed the old version of the library bundle, we run the software updater, and we see a clear difference in behav (Reality: the fault was mine for making a test case that didn't include the last verification steps for "is this bug fixed?" I'd expected some sort of visible indication on the screen saying "your content bundles are updating." There wasn't a message like that, but after a few minutes of confusion, I realized I should check the library.info file manually. This indicated that the bundle had indeed been upgraded. I filed http://dev.laptop.org/ticket/8805 suggesting that we make this visible to users next time.) 4. Close bug. (Reality: Close bug. Yay!) -- Mel Chua QA/Support Engineer [EMAIL PROTECTED] _______________________________________________ Testing mailing list [email protected] http://lists.laptop.org/listinfo/testing
