If a subtest crashed it and all following subtests have the result notrun. Only the test as a whole has the result crash. This commit propagates the crash to the correct subtest.
Note that if the crash happened after the last subtest it is silently ignored. --- I like this approach. Personally, I would have added a NULL terminated array of char* to config that defaults to NULL and emitted the JSON structure in PIGLIT_GL_TEST_CONFIG_END, but that's just details. Also this needs a little more work in the python framework: As is crashes don't show up in the result summary (regardless of whether the crash happens before or after the first call to piglit_report_subtest_result). If we use an OrderedDict instead of a regular dict in patch [3/9] and as the container of Subtests (framework/results.py:44) we could deduce the crashing subtest (if the order is preserved between piglit_enumerate_subtests and test execution) with this patch. Note that this would differ from the previous behavior in that if a test crashes before a call to piglit_report_subtest_result (as is common in CL tests, I am told) the first subtest is marked as crashed and the whole test doesn't show up in the summary. I'll gladly help with the adaption of the remaining tests if needed. framework/results.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/framework/results.py b/framework/results.py index 99dd3735b..2e1b832fa 100644 --- a/framework/results.py +++ b/framework/results.py @@ -69,6 +69,22 @@ class Subtests(collections.MutableMapping): res['__type__'] = 'Subtests' return res + def set_crash(self): + """Find the first subtest marked as notrun and mark it as crashed. + + If a subtest crashed it and all following subtests have the result + notrun. Only the test as a whole has the result crash. + This function propagates the crash to the correct subtest. + Note that if the crash happened after the last subtest it is silently + ignored. + """ + if status.CRASH in six.itervalues(self): + return + for key, result in six.iteritems(self): + if result == status.NOTRUN: + self[key] = status.CRASH + break + @classmethod def from_dict(cls, dict_): if '__type__' in dict_: @@ -232,6 +248,8 @@ class TestResult(object): # Set special instances if 'subtests' in dict_: inst.subtests = Subtests.from_dict(dict_['subtests']) + if inst.result == status.CRASH: + inst.subtests.set_crash() if 'time' in dict_: inst.time = TimeAttribute.from_dict(dict_['time']) -- 2.15.1 _______________________________________________ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit