Dmitry Baev created SUREFIRE-1345:
-
Summary: Report parser doesn't support flaky and retries
Key: SUREFIRE-1345
URL: https://issues.apache.org/jira/browse/SUREFIRE-1345
Project: Maven Surefire
Issue Type: Bug
Reporter: Dmitry Baev
TestSuiteXmlParser (from
`org.apache.maven.surefire:surefire-report-parser:2.19.1`)
doesn't support changes described in
https://github.com/apache/maven-surefire/blob/master/maven-surefire-plugin/src/site/apt/examples/rerun-failing-tests.apt.vm
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142750#comment-15142750
]
Dmitry Baev commented on SUREFIRE-1225:
---
Ok
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>Assignee: Tibor Digana
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141029#comment-15141029
]
Dmitry Baev commented on SUREFIRE-1225:
---
But it will run {{testWithParameters2}} and {{testWithParameters3}} as well...
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140999#comment-15140999
]
Dmitry Baev commented on SUREFIRE-1225:
---
I know that JUnit need to have an unique Description for each test case and
already read the docs as well. My point is that I don't like to care about
JUnit internal test case descriptions at all. I kinda have a test cases in my
source code and I like to run the test cases by method name (at least while
test case is a method in my source code).
Why am I think so:
In my case I build a distributed test runner, so I need to get the list of test
cases to run and then create a tasks to execute. Each task can contains one or
few test cases to run. The problem is that I don't is it a JUnit parameterized
test or it is TestNg test or whatever. In general I like to simply provide to
Surefire a command like "could you please run the test methods for me". And I
expect that surefire will decide how to run this methods by yourself. I don't
have a JUnit context, so I can't get the real test cases descriptions. An
example, some users can use an JUNit extension that will change the policy of
naming test cases in Descriptions.
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141259#comment-15141259
]
Dmitry Baev commented on SUREFIRE-1225:
---
Thanks! It works. What about special goal to get the list of test cases?
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141193#comment-15141193
]
Dmitry Baev commented on SUREFIRE-1225:
---
Yep, but as I say above I don't know is it a JUnit test, is it a parameterized
one etc. And using {{ParametersRunnerFactory}} you can override the name of
test in description as well (in general, all custom runners can change the test
description)
BTW maybe we should add the ability to get the list of test cases from Surefire
plugin? Like a separate goal that prints a json with all the test it going to
run? In such case I will use the Surefire plugin to get the list of tests (and
it helps to solve the problem with custom runners that can change the test
description for me).
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141199#comment-15141199
]
Dmitry Baev commented on SUREFIRE-1225:
---
And I found the separate problem:
If you add testng to classpath
{code:xml}
org.testng
testng
6.9.10
test
{code}
the test method filtering not working any more:
{{clean test -Dtest=com.github.baev.MyTest#adsasdasd}}
runs all the tests in {{MyTest}}. Is it a known issue? If not I can create a
new one
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141277#comment-15141277
]
Dmitry Baev commented on SUREFIRE-1225:
---
In general I can do it. The question is if it sounds good for you
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141310#comment-15141310
]
Dmitry Baev commented on SUREFIRE-1225:
---
{quote}Pls try to use TestNG 6.9.7 - 6.9.9.{quote}
The same
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141292#comment-15141292
]
Dmitry Baev edited comment on SUREFIRE-1225 at 2/10/16 5:49 PM:
There is the sample project to play around
https://github.com/baev/surefire-testng-junit-bug
{quote}What happens if you run test -Dtest=MyTest#adsasdasd without the
package?{quote}
It will run all the methods inside {{MyTest}}
{quote}I guess adsasdasd is the exact method name.{quote}
Nope, there is only {{testWithParameters}} method.
{quote}If you use TestNG suites this filtering I guess is not applicable{quote}
Nope, there is only one JUnit test, and both junit and testng deps.
was (Author: baev):
There is the sample project to play around
https://github.com/baev/surefire-testng-junit-bug
{{quote}}What happens if you run test -Dtest=MyTest#adsasdasd without the
package?{{quote}}
It will run all the methods inside {{MyTest}}
{{quote}}I guess adsasdasd is the exact method name.{{quote}}
Nope, there is only {{testWithParameters}} method.
{{quote}}If you use TestNG suites this filtering I guess is not
applicable{{quote}}
Nope, there is only one JUnit test, and both junit and testng deps.
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141320#comment-15141320
]
Dmitry Baev commented on SUREFIRE-1225:
---
Should I create an issue?
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141292#comment-15141292
]
Dmitry Baev commented on SUREFIRE-1225:
---
There is the sample project to play around
https://github.com/baev/surefire-testng-junit-bug
{{quote}}What happens if you run test -Dtest=MyTest#adsasdasd without the
package?{{quote}}
It will run all the methods inside {{MyTest}}
{{quote}}I guess adsasdasd is the exact method name.{{quote}}
Nope, there is only {{testWithParameters}} method.
{{quote}}If you use TestNG suites this filtering I guess is not
applicable{{quote}}
Nope, there is only one JUnit test, and both junit and testng deps.
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Baev updated SUREFIRE-1225:
--
Description:
The sample test to reproduce the problem:
{code:java|title=com/github/baev/MyTest.java}
@RunWith(Parameterized.class)
public class MyTest {
@Parameterized.Parameters
public static Collection
[
https://issues.apache.org/jira/browse/SUREFIRE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Baev updated SUREFIRE-1225:
--
Description:
The sample test to reproduce the problem:
{code:java|title=com/github/baev/MyTest.java}
@RunWith(Parameterized.class)
public class MyTest {
@Parameterized.Parameters
public static Collection data() {
return Arrays.asList(
new Object[]{},
new Object[]{}
);
}
@Test
public void testWithParameters() throws Exception {
}
}
{code}
and then run
{{clean test -Dtest=com.github.baev.MyTest#testWithParameters}}
The output is
{{Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on
project surefire-base-test-bug: No tests were executed!}}
Is it expected?
was:
The sample test to reproduce the problem:
{code:java|title=com/github/baev/MyTest.java}
@RunWith(Parameterized.class)
public class MyTest {
@Parameterized.Parameters
public static Collection data() {
return Arrays.asList(
new Object[]{},
new Object[]{}
);
}
@Test
public void testWithParameters() throws Exception {
}
}
{code}
and then run
{{clean test -Dtest=com.github.baev.MyTest#testWithParameters}}
The output is
{{Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on
project surefire-base-test-bug: No tests were executed! }}
Is it expected?
> Can't run a single parameterized test
> -
>
> Key: SUREFIRE-1225
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19.1
>Reporter: Dmitry Baev
>
> The sample test to reproduce the problem:
> {code:java|title=com/github/baev/MyTest.java}
> @RunWith(Parameterized.class)
> public class MyTest {
> @Parameterized.Parameters
> public static Collection data() {
> return Arrays.asList(
> new Object[]{},
> new Object[]{}
> );
> }
> @Test
> public void testWithParameters() throws Exception {
> }
> }
> {code}
> and then run
> {{clean test -Dtest=com.github.baev.MyTest#testWithParameters}}
> The output is
> {{Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on
> project surefire-base-test-bug: No tests were executed!}}
> Is it expected?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Dmitry Baev created SUREFIRE-1225:
-
Summary: Can't run a single parameterized test
Key: SUREFIRE-1225
URL: https://issues.apache.org/jira/browse/SUREFIRE-1225
Project: Maven Surefire
Issue Type: Bug
Affects Versions: 2.19.1
Reporter: Dmitry Baev
The sample test to reproduce the problem:
{code:java|title=com/github/baev/MyTest.java}
@RunWith(Parameterized.class)
public class MyTest {
@Parameterized.Parameters
public static Collection data() {
return Arrays.asList(
new Object[]{},
new Object[]{}
);
}
@Test
public void testWithParameters() throws Exception {
}
}
{code}
and then run
{{clean test -Dtest=com.github.baev.MyTest#testWithParameters}}
{{Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on
project surefire-base-test-bug: No tests were executed! }}
Is it expected?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[
https://issues.apache.org/jira/browse/SUREFIRE-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Baev updated SUREFIRE-1200:
--
Description:
Since 2.19 I can't run a single test by full name anymore:
{code:java}
package my.tests;
import org.junit.Test;
public class MyTest {
@Test
public void shouldRun() throws Exception {
}
}
{code}
And then command
{code: bash}
mvn clean test -Dtest=my.tests.MyTest
{code}
Doesn't run any tests.
BTW -Dtest=MyTest works fine and -Dtest=my/tests/MyTest works fine.
was:
Since 2.19 I can't run a single test by full name anymore:
{code:java}
package my.tests;
import org.junit.Test;
public class MyTest {
@Test
public void shouldRun() throws Exception {
}
}
{code}
And then command
{code: bash}
mvn clean test -Dtest=my.tests.MyTest
{code}
Doesn't run any tests.
BTW -Dtest=MyTest works fine
> Could not run single test by full name
> --
>
> Key: SUREFIRE-1200
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1200
> Project: Maven Surefire
> Issue Type: Bug
>Affects Versions: 2.19
>Reporter: Dmitry Baev
>
> Since 2.19 I can't run a single test by full name anymore:
> {code:java}
> package my.tests;
> import org.junit.Test;
> public class MyTest {
> @Test
> public void shouldRun() throws Exception {
> }
> }
> {code}
> And then command
> {code: bash}
> mvn clean test -Dtest=my.tests.MyTest
> {code}
> Doesn't run any tests.
> BTW -Dtest=MyTest works fine and -Dtest=my/tests/MyTest works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Dmitry Baev created SUREFIRE-1200:
-
Summary: Could not run single test by full name
Key: SUREFIRE-1200
URL: https://issues.apache.org/jira/browse/SUREFIRE-1200
Project: Maven Surefire
Issue Type: Bug
Affects Versions: 2.19
Reporter: Dmitry Baev
Since 2.19 I can't run a single test by full name anymore:
{code:java}
package my.tests;
import org.junit.Test;
public class MyTest {
@Test
public void shouldRun() throws Exception {
}
}
{code}
And then command
{code: bash}
mvn clean test -Dtest=my.tests.MyTest
{code}
Doesn't run any tests.
BTW -Dtest=MyTest works fine
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)