On Thu, 2016-12-01 at 14:25 +0100, Josef Skladanka wrote: > On Wed, Nov 30, 2016 at 6:29 PM, Adam Williamson <adamw...@fedoraproject.org > > wrote: > > On Wed, 2016-11-30 at 18:20 +0100, Josef Skladanka wrote: > > > I would try not to go the third way, because that is really prone to > > > > erros > > > IMO, and I'm not sure that "per context" is always right. So for me, the > > > "TCMS" part of the data, should be: > > > 1) testcases (with required fields/types of the fields in the "result > > > response" > > > 2) testplans - which testcases, possibly organized into groups. Maybe > > > > even > > > dependencies + saying "I need testcase X to pass, Y can be pass or warn, > > > > Z > > > can be whatever when A passes, for the testplan to pass" > > > > > > But this is fairly complex thing, to be honest, and it would be the first > > > and only useable TCMS in the world (from my point of view). > > > > I have rather different opinions, actually...but I'm not working on > > this right now and I'd rather have something concrete to discuss than > > just opinions :) > > > > We should obviously set goals properly, before diving into implementation > > details :) I'm interested in what you have in mind, since I've been > thinking about this particular kind of thing for the last few years, and it > really depends on what you expect of the system.
Well, the biggest point where I differ is that I think your 'third way' is kind of unavoidable. For all kinds of reasons. We re-use test cases between package update testing, Test Days, and release validation testing, for instance; some tests are more or less unique to some specific process, but certainly not all of them. The desired test environments may be significantly different in these different cases. We also have secondary arch teams using release validation processes similar to the primary arch process: they use many of the same test cases, but the desired test environments are of course not the same. Of course, in a non-wiki based system you could plausibly argue that a test case could be stored along with *all* of its possible environments, and then the configuration for a specific test event could include the information as to which environments are relevant and/or required for that test event. But at that point I think you're rather splitting hairs... In my original vision of 'relval NG' the test environment wouldn't actually exist at all, BTW. I was hoping we could simply list test cases, and the user could choose the image they were testing, and the image would serve as the 'test environment'. But on second thought that's unsustainable as there are things like BIOS vs. UEFI where we may want to run the same test on the same image and consider it a different result. The only way we could stick to my original vision there would be to present 'same test, different environment' as another row in the UI, kinda like we do for 'two-dimensional test tables' in Wikitcms; it's not actually horrible UI, but I don't think we'd want to pretend in the backend that these were two completely different. I mean, we could. Ultimately a 'test case' is going to be a database row with a URL and a numeric ID. We don't *have* to say the URL key is unique. ;) There are really all kinds of ways you can structure it, but I think fundamentally they'd all boil down to the same inherent level of complexity; some of them might be demonstrably worse than others (like...sticking them all in wikicode and parsing wiki table syntax to figure out when you have different 'test instances' for the same test case! that sounds like a *really bad* way to do it!) Er. I'm rambling, aren't I? One reason I actually tend to prefer just sitting down and writing something to trying to plan it all out comprehensively is that when I just sit here and try to think out planning questions I get very long-winded and fuzzy and chase off down all possible paths. Just writing a damn thing is usually quite quick and crystallizes a lot of the questions wonderfully... -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ qa-devel mailing list -- qa-devel@lists.fedoraproject.org To unsubscribe send an email to qa-devel-le...@lists.fedoraproject.org
