On Tue, May 7, 2013 at 3:42 PM, janI <j...@apache.org> wrote: > On 7 May 2013 21:31, Rob Weir <robw...@apache.org> wrote: > >> On Tue, May 7, 2013 at 11:46 AM, V Stuart Foote <vstuart.fo...@utsa.edu> >> wrote: >> > Shenfeng, Yuzhen, Liuping, Rob >> > >> > Took a moment and browsed through the TestLink test cases for the Full >> Regression and earlier test plans against the trunk and sidebar branch. >> > >> > Unless I completely missed seeing it, one noticeable omission from the >> test cases in general is a lack keyboard based navigation and use of >> accelerators. Would expect that to be a significant component of >> functional testing. >> > >> > Only testing with mouse and menu based usage is unfortunately a way to >> miss the framework needed for accessibility and Assistive Technology >> support and impact all the supported OS builds. Not just the work on the >> IA2 branch. >> > >> >> So what would you recommend? Is this something you can help with, on >> the test definition or execution side? >> >> My naive approach would be, "Make sure you can perform all test cases >> with just the keyboard". Or is there a more targeted way to do this? >> A regression test (as opposed to a full functional test) tries to be >> smart about coverage and focuses on the areas that have changed in the >> most recent version. In other words, it puts the effort where the >> risk is greatest. >> > > Sounds like a good approach to me "just with the keyboard". > > I hope that for all automated testing we only use "full functional test". > With human resources involved regression testing is understandable. >
The regression test is a manual test, not an automated one. Automated tests are "smoke tests" and have even less coverage than the regression tests. So automation, at least what we have now, is not going to solve this problem. > >> >> So, given what we know about AOO 3.4.1 keyboard support, and knowing >> what was changed in AOO 4.0, my guess would be the area to focus on >> would be the sidebar, yes? >> >> Translations are another area where I bet there is risk introduced. >> It is always possible for an accelerator collision to be introduced >> accidentally during translation. And the translations come in rather >> late in the process, after most testing. I wonder if we have any >> automated ways of detecting collisions? >> > > if/when we have keyboard automated testing, it would be an easy job to > write a script that maps the keyboard and accellerators to any language. > That would allow us to run the same tests in any language. > I was hoping for something closer to present capabilities. For example, accelerators are defined in resource files, yes? It should be possible to analyze the resources files directly to see if there is a collision within any menu item. In theory. > Based on my experience from international software, menus and accellerators > are common translation problems. > Yes. > > >> >> Any other areas that would make sense to focus on? >> > > Installation with a keyboard only, or automated installation > I didn't think the install UI changed in 4.0, did it? When I say "focus" I'm reasoning like this: 1) No product of the size and complexity of AOO is able to be fully tested, in any formal sense (line coverage, path coverage, etc.) 2) So we rely on a good sample of possible test cases that give good overall coverage, combined with more focused testing on areas where we think there is higher risk because of recent code changes. 3) There are many areas that are important to specific users: performance, running on a server, users with bidirectional writing scripts, users with East Asian writing styles, users on older versions of Windows versus newest versions of Linux, etc., And keyboard use as well. We can't do every test in every combination. (What about Windows 8 Hebrew users without a mouse?) So we need to figure out the best combination of tests that give good overall coverage. 4) To make the problem tractable areas that have not changed in AOO 4.0 will get less test coverage. Not the same as no coverage, since bugs can have strange interactions. But lacking 100% automated test suites with perfect coverage, infinite volunteers or infinite time, we need to figure out where to spend the time to find the most critical bugs. Where are those bugs? I'm thinking they are not in the install UI. But who knows... -Rob > rgds > Jan I. > >> >> Regards, >> >> -Rob >> >> >> > Stuart >> > San Antonio, Texas >> > >> > >> > ________________________________________ >> > From: Shenfeng Liu [liush...@gmail.com] >> > Sent: Tuesday, May 07, 2013 8:28 AM >> > To: qa@openoffice.apache.org; d...@openoffice.apache.org >> > Subject: Start AOO 4.0 Full Regression Test >> > >> > Yu Zhen & Liu Ping, >> > Thanks very much for your help to prepare for the test guidance! >> > >> > Now we have the latest dev snapshot build here: >> > >> https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds#DevelopmentSnapshotBuilds-AOOSnapshotfullsets >> > And we have the "AOO 4.0 Full Regression Test" test plan created in >> > Testlink: http://aootesting.adfinis-sygroup.org/index.php . >> > And we have test guidance here: >> > >> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Full+Regression+Test+Guidance >> > and here: >> > >> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+Defect+Verification+Guidance >> > >> > So I suggest we start the AOO 4.0 Full Regression Test now !!!!! >> > >> > The test focus of this test circle include the following: >> > (1) Basic Function Verification >> > (2) MS Interoperability Test >> > (3) 2nd phase of sidebar testing >> > (4) Defect verification >> > >> > We will add more test cases (e.g. the Upgrade Check) in later circle. >> > >> > We already got some volunteers replied in mail list. Thanks very much! >> > Will assign test cases in Testlink and send mail to you one by one. >> > And looking forward more people to join and help the testing! >> > >> > - Shenfeng (Simon) >> > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org >> > For additional commands, e-mail: dev-h...@openoffice.apache.org >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: qa-unsubscr...@openoffice.apache.org >> For additional commands, e-mail: qa-h...@openoffice.apache.org >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: qa-unsubscr...@openoffice.apache.org For additional commands, e-mail: qa-h...@openoffice.apache.org
