What we need to figure out in the short term is what test case storage system we are going to use, as well as what sort of test execution planning & reporting system we are going to use. While I've since gotten a Nitrate install (which does both) to mostly work so I can discuss it at FUDcon, we do not know when Fedora is going to formally put theirs online and what it will look like.
In general I know what test cases I want. But I don't know what system to put the test cases in. These test cases are not necessarily unit test cases; for instance there may be more than one way to execute a collaboration test, and the comments/system has to note that with the test run. And the number of test cases which need to be put somewhere is potentially large; if we create just 5 test cases for the ~40 G1G1 XO-1.5 activities that's ~200 test cases & items to be run, not including any for the base system or enhancements. Moving them all later could be a pain. It looks like the current testcase Wiki template that Fedora is using is a subset of ours (although many Fedora test cases don't use it). Therefore presuming they make a converter to handle that, they likely can handle our existing test case template similarly. Nitrate and other TCMS's I've looked at use an inline HTML editor instead of Wiki code, so a converter would have to be used or lots of manual work to go back and forth. I'm a bit more concerned about where to put the test execution plan, as in some ways that controls how the test cases should be written. To simplify a wiki-based approach, we could pair it with another project with a "developer's/tester's" database of all activities' functionality, and how this varies over time. The advantage of this is that it means I have to create relatively few wiki pages, and the database probably could be an alternative view of the activities.sugarlabs.org data with a few extra database columns. Every activity basically needs the same sort of things tested: Open/close, save/open journal entries, supported MIME types, collaboration, etc. So a standard test case with a database containing the specifics makes sense. The advantage is we don't have to worry about creating lots of Wiki pages, and we have accurate documentation on how activities work outside of the test case/run system(s). The problem with this two-system approach is that I don't think it ports to any other test case management system cleanly if at all. On 01/26/11 20:33, Samuel Klein wrote: > On Wed, Jan 26, 2011 at 8:23 PM, S Page <[email protected]> wrote: >> I'm in violent agreement with everything you say :) >> >> *If* you have useful information in w.l.o pages, other pages can query >> and show it. That's all. >> >> I don't know anyone else who's managed test results in a wiki. > > Ditto. Syncing with Fedora's process make sense to me. > _______________________________________________ Testing mailing list [email protected] http://lists.laptop.org/listinfo/testing
