There are a few things with this patch that complicate my use of junit:
1) the stdout for the parent test is in the stored in the testsuite tag
in the junit xml. However, I can't access this data from the
jenkins ui. If a subtest fails, I won't be able to write up a bug
from the Jenkins UI.
2) the subtests reference the parent test by descriptive name (eg "I am
a subtest of imageAtomicMin"), but the user needs to find the test
in the junit hierarchy
(piglit.spec.arb_shader_image_load_store.atomicity)
3) each subtest's testcase tag lacks the classname attribute, which
lets me identify the test uniquely. To use this format, I would
have to check if each test is in a suite, then append the suite's
name.
one option for making this usable is to keep the structure flat (no
nesting of suites). The parent test stays the same as today, and the
subtests are simply added as subsequent tests at the same level. The
stdout for a subtest refers to the parent test in stdout, so the user
can find it in the jenkins UI.
It's unfortunate to have to hack around the details of how Jenkins
displays tests. Maybe there is a better option...
-Mark
baker.dyla...@gmail.com writes:
> From: Dylan Baker <baker.dyla...@gmail.com>
>
> This series changes the way the JUnit backend generates results for
> tests with subtests. Rather than generating them as a single test and
> ignoring the subtests it generates them as a testsuite, and then adds
> each subtest as a test. This allows subtests to be represented nicely
> and to avoid duplicating large amounts of data in the xml file.
>
> Dylan Baker (2):
> framework/backends/junit.py: refactor _write into several helpers.
> framework/backends/junit.py: Handle tests with subtests as testsuite
> elements
>
> framework/backends/junit.py | 263
> +++++++++++++++++++++++---------
> framework/tests/junit_backends_tests.py | 206 ++++++++++++++++++++++++-
> 2 files changed, 393 insertions(+), 76 deletions(-)
>
> --
> 2.6.3
>
> _______________________________________________
> Piglit mailing list
> Piglit@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit
_______________________________________________
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit
