On 11/06/2012 05:57 AM, Lucas Meneghel Rodrigues wrote:
On 11/05/2012 02:36 PM, Vimal Patel wrote:
Requesting a Change to report a TestCase ID from ScanResults.py or the
Results in General
No problem, but I don't fully understand what the workflow would be. I'm
guessing:
* There's a Test Case Management System (TCMS), say, Testopia, with an
XMLRPC API interface that you can use to notify things like 'test 1234
PASS'.
* After the autotest job you need to parse results and report to the TCMS.
Provided that this is my correct understanding, we have options:
1) Formal API to report to such systems. May take a while to model and
get 'right'.
2) Do it Ad-Hoc, in which case it's fairly straight forward and doesn't
require any change on our side.
We (spice-qa team) require a way to map our TestCaseIDs in TCMS with
the Tests in KVM-Autotest.
After a conversation with Chris Evich, a good way to get this
accomplished is through using a test_case_id parameter and reporting
this in the results including scanresults.py.
Maybe generate a dict with test case names and test case ids. Something
like:
t_case_mapping = {'kvm.spice.foo1': 1234
'kvm.spice.bar2': 1235}
And during results parsing, if kvm.spice.foo1 PASS, call the XMLRPC API
in the test case management. I'm guessing your script does that.
As for creating a formal autotest API, I'd say, why not? But it might
take a while to work on that, and given the current TODO list, this
would have reduced priority.
Currently, the spiceqa team created a script using the results from
scanresults.py to parse through to get the ID & results to update TCMS.
Could you guys share the script?
As we discussed on IRC, I'm a very little bit worried about having all
those test_case_id parameters all over the upstream Cartesian config.
Eventhough I suggested this option :S Really it would only be
meaningful to us, and just "noise" for everyone else.
However, I do think others may appreciate some kind of standardized
parameter / API for the purpose of tracking virt. autotest case
association with test case management system. So this is a good problem
for us to be solving, I'll bet others have this need as well (or will
eventually).
I like the translation-dictionary idea, maybe we could incorporate that
into some kind of official script (similar to scanresults) that would
interface with your update script? (i.e. an option in between hard and
easy) :D
--
Chris Evich, RHCA, RHCE, RHCDS, RHCSS
Quality Assurance Engineer
e-mail: cevich + `@' + redhat.com o: 1-888-RED-HAT1 x44214
_______________________________________________
Autotest-kernel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/autotest-kernel
