Re: [Piglit] [PATCH v3 0/4] Add support for subtests in JUnit as nested testsuite elements

[Mark Janes]

Before --junit-subtests is made the default, I need to be able to use
subtest names in:

 * --include-tests and --exclude-tests filters
 * [expected-failures] and [expected-crashes] sections of config files.

For filtering subtests, it is acceptable for include/exclude to apply to
the parent test.  I don't need to run a single subtest.

Series is

Reviewed-by: Mark Janes 
Tested-by: Mark Janes 

Dylan Baker  writes:

> This series refactors the JUnit backend so that the _write attribute is
> actually a callable class, instead of a method. This gives greater
> flexibility to handle subtests in the JUnit, or continue ignoring them.
>
> The reason to ignore them is that some xUnit implementations don't
> handle them properly (the JUnit plugin for Jenkins does not, for
> example, while the xUnit plugin does). Since piglit has a more than one
> consumer of the JUnit backend, it makes sense to be able to select the
> new backend or not.
>
> To enable this features add the --junit-subtests switch to the command
> line (obviously when using the junit backend), and watch magic happen.
> Each test with subtests will be recorded as a testsuite element, and the
> stderr and stdout will be attached to the testsuite rather than the
> testcase. However, when using the xUnit plugin for Jenkins it is
> rendered correctly.
>
> Changes in v2:
>  - Add error message explaining what happened if result != pass.
>  - Merge two patches together to maintain behavior for bisecting.
>  - Add skipped tag to tests without subtests (when using junit-subtests).
>  - Add classname to subtests, like is done for full tests.
>  - Add additional unittests.
>  - Call junit_escape on the testname as well as the classname.
>  - Reword the help message for the junit-subtests switch.
>  - Add a new patch which fixes a bug masking status changes if the test 
> becomes
>skip.
>
> Changes in v3:
>  - Fix classname in subtests to include the test name.
>  - Fix spelling errors in commit messages.
>
> Dylan Baker (4):
>   framework/backends/junit.py: Split _write into a separate class.
>   framework/backends/junit.py: Add a writer class that handles subtests
>   framework: add command line switch to enable junit subtests
>   framework/backends/junit: Don't let skip hide status changes
>
>  framework/backends/junit.py| 352 ++
>  framework/programs/run.py  |   8 +-
>  unittests/framework/backends/test_junit.py | 192 +++-
>  3 files changed, 414 insertions(+), 138 deletions(-)
>
> -- 
> git-series 0.8.7
> ___
> Piglit mailing list
> Piglit@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/piglit
___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

[Piglit] [PATCH v3 0/4] Add support for subtests in JUnit as nested testsuite elements

[Dylan Baker]

This series refactors the JUnit backend so that the _write attribute is
actually a callable class, instead of a method. This gives greater
flexibility to handle subtests in the JUnit, or continue ignoring them.

The reason to ignore them is that some xUnit implementations don't
handle them properly (the JUnit plugin for Jenkins does not, for
example, while the xUnit plugin does). Since piglit has a more than one
consumer of the JUnit backend, it makes sense to be able to select the
new backend or not.

To enable this features add the --junit-subtests switch to the command
line (obviously when using the junit backend), and watch magic happen.
Each test with subtests will be recorded as a testsuite element, and the
stderr and stdout will be attached to the testsuite rather than the
testcase. However, when using the xUnit plugin for Jenkins it is
rendered correctly.

Changes in v2:
 - Add error message explaining what happened if result != pass.
 - Merge two patches together to maintain behavior for bisecting.
 - Add skipped tag to tests without subtests (when using junit-subtests).
 - Add classname to subtests, like is done for full tests.
 - Add additional unittests.
 - Call junit_escape on the testname as well as the classname.
 - Reword the help message for the junit-subtests switch.
 - Add a new patch which fixes a bug masking status changes if the test becomes
   skip.

Changes in v3:
 - Fix classname in subtests to include the test name.
 - Fix spelling errors in commit messages.

Dylan Baker (4):
  framework/backends/junit.py: Split _write into a separate class.
  framework/backends/junit.py: Add a writer class that handles subtests
  framework: add command line switch to enable junit subtests
  framework/backends/junit: Don't let skip hide status changes

 framework/backends/junit.py| 352 ++
 framework/programs/run.py  |   8 +-
 unittests/framework/backends/test_junit.py | 192 +++-
 3 files changed, 414 insertions(+), 138 deletions(-)

-- 
git-series 0.8.7
___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit