[
https://issues.apache.org/jira/browse/SUREFIRE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Geoff Soutter updated SUREFIRE-2147:
------------------------------------
Description:
I created a really simple example of JUnit5 DynamicContainer/DynamicTest
{code}
package samplepackage;
...
public class SampleTest {
@TestFactory
Collection<DynamicContainer> createDynamicTests() {
List<DynamicTest> testList = new ArrayList<>();
testList.add(DynamicTest.dynamicTest("test1", new MyExecutable()));
testList.add(DynamicTest.dynamicTest("test2", new MyExecutable()));
List<DynamicContainer> root = new ArrayList<>();
root.add(DynamicContainer.dynamicContainer("aFolder", testList));
return root;
}
private static class MyExecutable implements Executable {
@Override
public void execute() throws Throwable {
}
}
}
{code}
When I run this in IDEA. I get the following structure
* SampleTest
** createDynamicTests()
*** aFolder
**** test1
**** test2
Here we can see that dynamic tests are represented by IDEA as nested children
of the test method. We can open and close the "virtual packages" created as
usual to explore in the dynamically created test results. It is not a "flat
list".
Using surefire 3.0.0-m7, with the example "phrased" reporter configuration
enabled, it creates contents in surefire-reports like so:
{code}
<testcase name="aFolder createDynamicTests() test1"
classname="samplepackage.SampleTest" time="0.02"/>
<testcase name="aFolder createDynamicTests() test2"
classname="samplepackage.SampleTest" time="0.001"/>
{code}
Here all the nested synthetic tests are created as the child of a single class
in the HTML report. The folder name is injected into the "method" name. That
is, from a reporting perspective, they are created as a flat list.
This doesn't scale nicely as the number of tests in the dynamically created
"folder" structure grows. We can't view the different DynamicContainers
separately in the rendered HTML reports. We can't get history on any particular
DynamicContainer in the rendered HTML reports.
I think what we want here is that dynamic tests in folders should have those
folders structured as synthetic packages. This allows us the benefits of having
the nested structure modelled correctly in the HTML report. I think we could do
that by moving the method name and folder name from name= into classname=
{code}
<testcase name="test1"
classname="samplepackage.SampleTest.createDynamicTests().aFolder.Test"
time="0.02"/>
<testcase name="test2"
classname="samplepackage.SampleTest.createDynamicTests().aFolder.Test"
time="0.001"/>
{code}
A compromise I have made here is that I have injected ".Test" at the end as it
allows the entire DynamicContainer "folder" structure to be visible in the
virtual package name - DynamicTests don't have a "class" concept.
was:
I created a really simple example of JUnit5 DynamicContainer/DynamicTest
{code}
package samplepackage;
...
public class SampleTest {
@TestFactory
Collection<DynamicContainer> createDynamicTests() {
List<DynamicTest> testList = new ArrayList<>();
testList.add(DynamicTest.dynamicTest("test1", new MyExecutable()));
testList.add(DynamicTest.dynamicTest("test2", new MyExecutable()));
List<DynamicContainer> root = new ArrayList<>();
root.add(DynamicContainer.dynamicContainer("aFolder", testList));
return root;
}
private static class MyExecutable implements Executable {
@Override
public void execute() throws Throwable {
}
}
}
{code}
When I run this in IDEA. I get the following structure
* SampleTest
** createDynamicTests()
*** aFolder
**** test1
**** test2
Here we can see that dynamic tests are represented by IDEA as nested children
of the test method. We can open and close the "virtual packages" created as
usual to explore in the dynamically created test results. It is not a "flat
list".
Using surefire 3.0.0-m7, with the example "phrased" reporter configuration
enabled, it creates contents in surefire-reports like so:
{code}
<testcase name="aFolder createDynamicTests() test1"
classname="samplepackage.SampleTest" time="0.02"/>
<testcase name="aFolder createDynamicTests() test2"
classname="samplepackage.SampleTest" time="0.001"/>
{code}
Here all the nested synthetic tests are created as the child of a single class
in the HTML report. The folder name is injected into the "method" name. That
is, from a reporting perspective, they are created as a flat list.
This doesn't scale nicely as the number of tests in the dynamically created
"folder" structure grows. We can't view the different DynamicContainers
separately in the rendered HTML reports. We can't get history on any particular
DynamicContainer in the rendered HTML reports.
I think what we want here is that dynamic tests in folders should have those
folders structured as synthetic packages. This allows us the benefits of having
the nested structure modelled correctly in the HTML report. I think we could do
that by moving the method name and folder name from name= into classname=
<testcase name="test1"
classname="samplepackage.SampleTest.createDynamicTests().aFolder.Test"
time="0.02"/>
<testcase name="test2"
classname="samplepackage.SampleTest.createDynamicTests().aFolder.Test"
time="0.001"/>
A compromise I have made here is that I have injected ".Test" at the end as it
allows the entire DynamicContainer "folder" structure to be visible in the
virtual package name - DynamicTests don't have a "class" concept.
> Surefire report for JUnit5 DynamicContainer/DynamicTest doesn't work with
> HTML generation
> -----------------------------------------------------------------------------------------
>
> Key: SUREFIRE-2147
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2147
> Project: Maven Surefire
> Issue Type: Bug
> Reporter: Geoff Soutter
> Priority: Major
>
> I created a really simple example of JUnit5 DynamicContainer/DynamicTest
> {code}
> package samplepackage;
> ...
> public class SampleTest {
> @TestFactory
> Collection<DynamicContainer> createDynamicTests() {
> List<DynamicTest> testList = new ArrayList<>();
> testList.add(DynamicTest.dynamicTest("test1", new MyExecutable()));
> testList.add(DynamicTest.dynamicTest("test2", new MyExecutable()));
> List<DynamicContainer> root = new ArrayList<>();
> root.add(DynamicContainer.dynamicContainer("aFolder", testList));
> return root;
> }
> private static class MyExecutable implements Executable {
> @Override
> public void execute() throws Throwable {
> }
> }
> }
> {code}
> When I run this in IDEA. I get the following structure
> * SampleTest
> ** createDynamicTests()
> *** aFolder
> **** test1
> **** test2
> Here we can see that dynamic tests are represented by IDEA as nested children
> of the test method. We can open and close the "virtual packages" created as
> usual to explore in the dynamically created test results. It is not a "flat
> list".
> Using surefire 3.0.0-m7, with the example "phrased" reporter configuration
> enabled, it creates contents in surefire-reports like so:
> {code}
> <testcase name="aFolder createDynamicTests() test1"
> classname="samplepackage.SampleTest" time="0.02"/>
> <testcase name="aFolder createDynamicTests() test2"
> classname="samplepackage.SampleTest" time="0.001"/>
> {code}
> Here all the nested synthetic tests are created as the child of a single
> class in the HTML report. The folder name is injected into the "method" name.
> That is, from a reporting perspective, they are created as a flat list.
> This doesn't scale nicely as the number of tests in the dynamically created
> "folder" structure grows. We can't view the different DynamicContainers
> separately in the rendered HTML reports. We can't get history on any
> particular DynamicContainer in the rendered HTML reports.
> I think what we want here is that dynamic tests in folders should have those
> folders structured as synthetic packages. This allows us the benefits of
> having the nested structure modelled correctly in the HTML report. I think we
> could do that by moving the method name and folder name from name= into
> classname=
> {code}
> <testcase name="test1"
> classname="samplepackage.SampleTest.createDynamicTests().aFolder.Test"
> time="0.02"/>
> <testcase name="test2"
> classname="samplepackage.SampleTest.createDynamicTests().aFolder.Test"
> time="0.001"/>
> {code}
> A compromise I have made here is that I have injected ".Test" at the end as
> it allows the entire DynamicContainer "folder" structure to be visible in the
> virtual package name - DynamicTests don't have a "class" concept.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
