Re: Assumption fail treated as unexcepted exception?!
MG>below
From: Markus KARG
Sent: Tuesday, August 18, 2020 8:28 AM
To: 'Maven Developers List'
Subject: AW: Assumption fail treated as unexcepted exception?!
Martin,
sorry was busy otherwise, had no time to check with JUnit 4. Will try ASAP and
report here.
In fact I do not understand what you want me to ask the JUnit authors. I do not
want to use JUnit 3 at all (I did not actively decide for any version), so if
there is something to ask them, please tell me *what*?
MG>we need to understand if the statement if Junit AssumptionViolatedException
is thrown
MG>the intended behaviour is to NOT fail the testcase that wraps
AssumptionViolatedException
MG>AssumptionViolatedException source is displayed below
package org.junit;
import org.hamcrest.Matcher;
/**
* An exception class used to implement assumptions (state in which a
given test
* is meaningful and should or should not be executed). A test for which an
assumption
* fails should not generate a test case failure.
*
* @see org.junit.Assume
* @since 4.12
*/
@SuppressWarnings("deprecation")
public class AssumptionViolatedException extends
org.junit.internal.AssumptionViolatedException {
private static final long serialVersionUID = 1L;
/**
* An assumption exception with the given actual value and a
matcher describing
* the expectation that failed.
*/
public AssumptionViolatedException(T actual, Matcher matcher) {
super(actual, matcher);
}
/**
* An assumption exception with a message with the given actual
value and a
* matcher describing the expectation that failed.
*/
public AssumptionViolatedException(String message, T expected,
Matcher matcher) {
super(message, expected, matcher);
}
/**
* An assumption exception with the given message only.
*/
public AssumptionViolatedException(String message) {
super(message);
}
/**
* An assumption exception with the given message and a cause.
*/
public AssumptionViolatedException(String assumption, Throwable t) {
super(assumption, t);
}
}
MG>if the above junit statement is true then when you throw
AssumptionViolatedException
MG>the testcase that wraps AssumptionViolatedException should
NOT FAIL the testcase
MG>If true I will admit this junit behaviour seems counter-intuitive
MG>thanks
MG>martin
thanks
-Markus
-Ursprüngliche Nachricht-
Von: Martin Gainty [mailto:mgai...@hotmail.com]
Gesendet: Dienstag, 18. August 2020 11:28
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
have you tried with junit4?
if you need to stick with older junit then you can implement your
AssumptionViolatedException to extend BuildFailureException
any feedback from junit author on this behaviour?
gruss
~martin~
From: Markus KARG
Sent: Monday, August 17, 2020 1:53 PM
To: 'Maven Developers List'
Subject: AW: Assumption fail treated as unexcepted exception?!
That doesn't make it any better, and it is no excuse that we do not simply
catch that particular exception inside the Surefire plugin.
So did we simply forget to implement this is the Surefire plugin (so I can add
it), did we not implement it deliberately (so I shall not add it), or is it
broken (so I shall fix it)?
-Markus
-Ursprüngliche Nachricht-
Von: Tibor Digana [mailto:tibordig...@apache.org]
Gesendet: Montag, 17. August 2020 18:37
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
This is the outcome in IDEA:
"Tests failed: 1, passed: 0"
So the behavior is the same with Maven.
On Mon, Aug 17, 2020 at 6:35 PM Tibor Digana wrote:
> Hi Markus,
>
> It is a specific problem related to the JUnit library because the test
> fails in IntelliJ IDEA and in Maven as well.
>
> The JUnit4 assumptions fail with yellow markers in IDEA but here the
> JUnit3' TestCase fails in red as a typical error or failure.
> And Maven fails this test as follows but i think this behavior starts in
> the junit library itself.
>
> [INFO] Running test.TestAssumptions
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 0.028 s <<< FAILURE! - in test.TestAssumptions
> [ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR!
> org.junit.AssumptionViolatedException: got: , expected: is
> at test/test.TestAssumptions.testABC(TestAssumptions.java:9)
>
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: ,
> expected: is [INFO]
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD FAILURE
AW: Assumption fail treated as unexcepted exception?!
Martin,
sorry was busy otherwise, had no time to check with JUnit 4. Will try ASAP and
report here.
In fact I do not understand what you want me to ask the JUnit authors. I do not
want to use JUnit 3 at all (I did not actively decide for any version), so if
there is something to ask them, please tell me *what*?
thanks
-Markus
-Ursprüngliche Nachricht-
Von: Martin Gainty [mailto:mgai...@hotmail.com]
Gesendet: Dienstag, 18. August 2020 11:28
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
have you tried with junit4?
if you need to stick with older junit then you can implement your
AssumptionViolatedException to extend BuildFailureException
any feedback from junit author on this behaviour?
gruss
~martin~
From: Markus KARG
Sent: Monday, August 17, 2020 1:53 PM
To: 'Maven Developers List'
Subject: AW: Assumption fail treated as unexcepted exception?!
That doesn't make it any better, and it is no excuse that we do not simply
catch that particular exception inside the Surefire plugin.
So did we simply forget to implement this is the Surefire plugin (so I can add
it), did we not implement it deliberately (so I shall not add it), or is it
broken (so I shall fix it)?
-Markus
-Ursprüngliche Nachricht-
Von: Tibor Digana [mailto:tibordig...@apache.org]
Gesendet: Montag, 17. August 2020 18:37
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
This is the outcome in IDEA:
"Tests failed: 1, passed: 0"
So the behavior is the same with Maven.
On Mon, Aug 17, 2020 at 6:35 PM Tibor Digana wrote:
> Hi Markus,
>
> It is a specific problem related to the JUnit library because the test
> fails in IntelliJ IDEA and in Maven as well.
>
> The JUnit4 assumptions fail with yellow markers in IDEA but here the
> JUnit3' TestCase fails in red as a typical error or failure.
> And Maven fails this test as follows but i think this behavior starts in
> the junit library itself.
>
> [INFO] Running test.TestAssumptions
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 0.028 s <<< FAILURE! - in test.TestAssumptions
> [ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR!
> org.junit.AssumptionViolatedException: got: , expected: is
> at test/test.TestAssumptions.testABC(TestAssumptions.java:9)
>
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: ,
> expected: is [INFO]
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD FAILURE
>
> On Mon, Aug 17, 2020 at 3:07 PM Markus KARG
> wrote:
>
>> Martin,
>>
>> I don't understand your question.
>>
>> You posted the Javadocs, and they tell it clearly: "A test for which an
>> assumption fails should NOT generate a test case failure.".
>>
>> Also Assume's Javadocs
>> (https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are pretty
>> clear about it: "... A failed assumption does NOT mean the code is broken,
>> but that the test provides no useful information. Assume basically means
>> "don't RUN this test if these conditions don't apply". The default JUnit
>> runner SKIPS tests with failing assumptions..."
>>
>> So do you really want me to ask the JUnit people whether they really MEAN
>> what they clearly wrote...?
>>
>> -Markus
>>
>> -Ursprüngliche Nachricht-
>> Von: Martin Gainty [mailto:mgai...@hotmail.com]
>> Gesendet: Montag, 17. August 2020 12:27
>> An: Maven Developers List
>> Betreff: Re: Assumption fail treated as unexcepted exception?!
>>
>>
>>/**
>> * Call to assume that actual satisfies the condition
>> specified by matcher.
>> * If not, the test halts and is ignored.
>> * Example:
>> * :
>> * assumeThat(1, is(1)); // passes
>> * foo(); // will execute
>> * assumeThat(0, is(1)); // assumption failure! test halts
>> * int x = 1 / 0; // will never execute
>> *
>> *
>> * @param the static type accepted by the matcher (this can flag
>> obvious compile-time problems such as {@code assumeThat(1, is("a"))}
>> * @param actual the computed value being compared
>> * @param matcher an expression, built of {@link Matcher}s, specifying
>> allowed values
>> * @see org.hamcrest.CoreMatchers
>> * @see org.junit.matchers.JUnitMatchers
>> */
>> p
Re: Assumption fail treated as unexcepted exception?!
Think the answer had been written already but in case it is not clear - it looks at least to me: it works as expected. JUnit 3 does not support assumptions - even when executed with JUnit 4 launcher - exactly as JUnt Jupiter mimics JUnit 4 behavior when executed with vintage engine. Assumptions are JUnit >= 4.4 so if you want to use an assumption write a JUnit 4 test and not a JUnit 3 one. Hope it helps. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <https://www.packtpub.com/application-development/java-ee-8-high-performance> Le mar. 18 août 2020 à 13:59, Martin Gainty a écrit : > have you tried with junit4? > > if you need to stick with older junit then you can implement your > AssumptionViolatedException to extend BuildFailureException > > any feedback from junit author on this behaviour? > > gruss > ~martin~ > > > > From: Markus KARG > Sent: Monday, August 17, 2020 1:53 PM > To: 'Maven Developers List' > Subject: AW: Assumption fail treated as unexcepted exception?! > > That doesn't make it any better, and it is no excuse that we do not simply > catch that particular exception inside the Surefire plugin. > So did we simply forget to implement this is the Surefire plugin (so I can > add it), did we not implement it deliberately (so I shall not add it), or > is it broken (so I shall fix it)? > -Markus > > -Ursprüngliche Nachricht- > Von: Tibor Digana [mailto:tibordig...@apache.org] > Gesendet: Montag, 17. August 2020 18:37 > An: Maven Developers List > Betreff: Re: Assumption fail treated as unexcepted exception?! > > This is the outcome in IDEA: > > "Tests failed: 1, passed: 0" > > So the behavior is the same with Maven. > > On Mon, Aug 17, 2020 at 6:35 PM Tibor Digana > wrote: > > > Hi Markus, > > > > It is a specific problem related to the JUnit library because the test > > fails in IntelliJ IDEA and in Maven as well. > > > > The JUnit4 assumptions fail with yellow markers in IDEA but here the > > JUnit3' TestCase fails in red as a typical error or failure. > > And Maven fails this test as follows but i think this behavior starts in > > the junit library itself. > > > > [INFO] Running test.TestAssumptions > > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > > 0.028 s <<< FAILURE! - in test.TestAssumptions > > [ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR! > > org.junit.AssumptionViolatedException: got: , expected: is > > at test/test.TestAssumptions.testABC(TestAssumptions.java:9) > > > > [INFO] > > [INFO] Results: > > [INFO] > > [ERROR] Errors: > > [ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: , > > expected: is > [INFO] > > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 > > [INFO] > > [INFO] > > > > [INFO] BUILD FAILURE > > > > On Mon, Aug 17, 2020 at 3:07 PM Markus KARG > > wrote: > > > >> Martin, > >> > >> I don't understand your question. > >> > >> You posted the Javadocs, and they tell it clearly: "A test for which an > >> assumption fails should NOT generate a test case failure.". > >> > >> Also Assume's Javadocs > >> (https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are > pretty > >> clear about it: "... A failed assumption does NOT mean the code is > broken, > >> but that the test provides no useful information. Assume basically means > >> "don't RUN this test if these conditions don't apply". The default JUnit > >> runner SKIPS tests with failing assumptions..." > >> > >> So do you really want me to ask the JUnit people whether they really > MEAN > >> what they clearly wrote...? > >> > >> -Markus > >> > >> -Ursprüngliche Nachricht- > >> Von: Martin Gainty [mailto:mgai...@hotmail.com] > >> Gesendet: Montag, 17. August 2020 12:27 > >> An: Maven Developers List > >> Betreff: Re: Assumption fail treated as unexcepted exception?! > >> > >> > >>/** > >> * Call to assume that actual satisfies the condition > >> specified by matcher. > >> * If not,
Re: Assumption fail treated as unexcepted exception?!
have you tried with junit4?
if you need to stick with older junit then you can implement your
AssumptionViolatedException to extend BuildFailureException
any feedback from junit author on this behaviour?
gruss
~martin~
From: Markus KARG
Sent: Monday, August 17, 2020 1:53 PM
To: 'Maven Developers List'
Subject: AW: Assumption fail treated as unexcepted exception?!
That doesn't make it any better, and it is no excuse that we do not simply
catch that particular exception inside the Surefire plugin.
So did we simply forget to implement this is the Surefire plugin (so I can add
it), did we not implement it deliberately (so I shall not add it), or is it
broken (so I shall fix it)?
-Markus
-Ursprüngliche Nachricht-
Von: Tibor Digana [mailto:tibordig...@apache.org]
Gesendet: Montag, 17. August 2020 18:37
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
This is the outcome in IDEA:
"Tests failed: 1, passed: 0"
So the behavior is the same with Maven.
On Mon, Aug 17, 2020 at 6:35 PM Tibor Digana wrote:
> Hi Markus,
>
> It is a specific problem related to the JUnit library because the test
> fails in IntelliJ IDEA and in Maven as well.
>
> The JUnit4 assumptions fail with yellow markers in IDEA but here the
> JUnit3' TestCase fails in red as a typical error or failure.
> And Maven fails this test as follows but i think this behavior starts in
> the junit library itself.
>
> [INFO] Running test.TestAssumptions
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 0.028 s <<< FAILURE! - in test.TestAssumptions
> [ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR!
> org.junit.AssumptionViolatedException: got: , expected: is
> at test/test.TestAssumptions.testABC(TestAssumptions.java:9)
>
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: ,
> expected: is [INFO]
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD FAILURE
>
> On Mon, Aug 17, 2020 at 3:07 PM Markus KARG
> wrote:
>
>> Martin,
>>
>> I don't understand your question.
>>
>> You posted the Javadocs, and they tell it clearly: "A test for which an
>> assumption fails should NOT generate a test case failure.".
>>
>> Also Assume's Javadocs
>> (https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are pretty
>> clear about it: "... A failed assumption does NOT mean the code is broken,
>> but that the test provides no useful information. Assume basically means
>> "don't RUN this test if these conditions don't apply". The default JUnit
>> runner SKIPS tests with failing assumptions..."
>>
>> So do you really want me to ask the JUnit people whether they really MEAN
>> what they clearly wrote...?
>>
>> -Markus
>>
>> -Ursprüngliche Nachricht-
>> Von: Martin Gainty [mailto:mgai...@hotmail.com]
>> Gesendet: Montag, 17. August 2020 12:27
>> An: Maven Developers List
>> Betreff: Re: Assumption fail treated as unexcepted exception?!
>>
>>
>>/**
>> * Call to assume that actual satisfies the condition
>> specified by matcher.
>> * If not, the test halts and is ignored.
>> * Example:
>> * :
>> * assumeThat(1, is(1)); // passes
>> * foo(); // will execute
>> * assumeThat(0, is(1)); // assumption failure! test halts
>> * int x = 1 / 0; // will never execute
>> *
>> *
>> * @param the static type accepted by the matcher (this can flag
>> obvious compile-time problems such as {@code assumeThat(1, is("a"))}
>> * @param actual the computed value being compared
>> * @param matcher an expression, built of {@link Matcher}s, specifying
>> allowed values
>> * @see org.hamcrest.CoreMatchers
>> * @see org.junit.matchers.JUnitMatchers
>> */
>> public static void assumeThat(T actual, Matcher matcher) {
>> if (!matcher.matches(actual)) {
>> throw new AssumptionViolatedException(actual, matcher);
>> }
>> }
>>
>> //does AssumptionViolatedException ever produce test case failure?
>>
>> /**
>> * An exception class used to implement assumptions (state in
>> which a
>> given test
>> * is meaningful and should or should not be executed). A test for which
>> an
>> assumpt
Re: Assumption fail treated as unexcepted exception?!
Markus, I was not talking about excuse. I only said that the JUnit library
has a bug regarding the old junit3 style. As far as I know the situation
with this library, it is very unlikely that it would be fixed.
The Surefire cannot do anything about it once the events are not fired from
JUnit3 TestCase.
As Piotr said you have to use the annotations and not to extend the class
TestCase. This would work for sure.
T
On Mon, Aug 17, 2020 at 7:58 PM Markus KARG wrote:
> That doesn't make it any better, and it is no excuse that we do not simply
> catch that particular exception inside the Surefire plugin.
> So did we simply forget to implement this is the Surefire plugin (so I can
> add it), did we not implement it deliberately (so I shall not add it), or
> is it broken (so I shall fix it)?
> -Markus
>
> -Ursprüngliche Nachricht-
> Von: Tibor Digana [mailto:tibordig...@apache.org]
> Gesendet: Montag, 17. August 2020 18:37
> An: Maven Developers List
> Betreff: Re: Assumption fail treated as unexcepted exception?!
>
> This is the outcome in IDEA:
>
> "Tests failed: 1, passed: 0"
>
> So the behavior is the same with Maven.
>
> On Mon, Aug 17, 2020 at 6:35 PM Tibor Digana
> wrote:
>
> > Hi Markus,
> >
> > It is a specific problem related to the JUnit library because the test
> > fails in IntelliJ IDEA and in Maven as well.
> >
> > The JUnit4 assumptions fail with yellow markers in IDEA but here the
> > JUnit3' TestCase fails in red as a typical error or failure.
> > And Maven fails this test as follows but i think this behavior starts in
> > the junit library itself.
> >
> > [INFO] Running test.TestAssumptions
> > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > 0.028 s <<< FAILURE! - in test.TestAssumptions
> > [ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR!
> > org.junit.AssumptionViolatedException: got: , expected: is
> > at test/test.TestAssumptions.testABC(TestAssumptions.java:9)
> >
> > [INFO]
> > [INFO] Results:
> > [INFO]
> > [ERROR] Errors:
> > [ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: ,
> > expected: is > [INFO]
> > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> > [INFO]
> > [INFO]
> >
> > [INFO] BUILD FAILURE
> >
> > On Mon, Aug 17, 2020 at 3:07 PM Markus KARG
> > wrote:
> >
> >> Martin,
> >>
> >> I don't understand your question.
> >>
> >> You posted the Javadocs, and they tell it clearly: "A test for which an
> >> assumption fails should NOT generate a test case failure.".
> >>
> >> Also Assume's Javadocs
> >> (https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are
> pretty
> >> clear about it: "... A failed assumption does NOT mean the code is
> broken,
> >> but that the test provides no useful information. Assume basically means
> >> "don't RUN this test if these conditions don't apply". The default JUnit
> >> runner SKIPS tests with failing assumptions..."
> >>
> >> So do you really want me to ask the JUnit people whether they really
> MEAN
> >> what they clearly wrote...?
> >>
> >> -Markus
> >>
> >> -Ursprüngliche Nachricht-
> >> Von: Martin Gainty [mailto:mgai...@hotmail.com]
> >> Gesendet: Montag, 17. August 2020 12:27
> >> An: Maven Developers List
> >> Betreff: Re: Assumption fail treated as unexcepted exception?!
> >>
> >>
> >>/**
> >> * Call to assume that actual satisfies the condition
> >> specified by matcher.
> >> * If not, the test halts and is ignored.
> >> * Example:
> >> * :
> >> * assumeThat(1, is(1)); // passes
> >> * foo(); // will execute
> >> * assumeThat(0, is(1)); // assumption failure! test halts
> >> * int x = 1 / 0; // will never execute
> >> *
> >> *
> >> * @param the static type accepted by the matcher (this can flag
> >> obvious compile-time problems such as {@code assumeThat(1, is("a"))}
> >> * @param actual the computed value being compared
> >> * @param matcher an expression, built of {@link Matcher}s,
> specifying
> >> allowed values
> >> * @see org.hamcrest.CoreMatchers
> >> * @see org.junit.matchers.JUnitMatchers
>
Re: Assumption fail treated as unexcepted exception?!
What could be fixed is PlexusTestCase (https://github.com/eclipse/sisu.plexus/tree/master/org.eclipse.sisu.plexus) that TestCopyMojo inherits from. Migrate it to JUnit 4, and one day it will reach maven-dependency-plugin via maven-plugin-testing-harness. It's that simple. OR Upgrade TestCopyMojo to JUnit 4: 1. Add explicit runner to TestCopyMojo 2. Add (missing now) @Test-s and @Before etc 3. upgrade surefire plugin (currently used 2.22.0 does not mark not-assumed as skipped) and you'll get: [WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 2.432 s - in org.apache.maven.plugins.dependency.fromConfiguration.TestCopyMojo with 62 63 in xml surefire report. (https://github.com/pzygielo/maven-dependency-plugin/commit/353ca210c720278b42e4dca1e7f3f3c34c439c9a, ran with `mvn test -Dtest=TestCopyMojo`). Given all existing test methods meet their goals. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
AW: Assumption fail treated as unexcepted exception?!
That doesn't make it any better, and it is no excuse that we do not simply
catch that particular exception inside the Surefire plugin.
So did we simply forget to implement this is the Surefire plugin (so I can add
it), did we not implement it deliberately (so I shall not add it), or is it
broken (so I shall fix it)?
-Markus
-Ursprüngliche Nachricht-
Von: Tibor Digana [mailto:tibordig...@apache.org]
Gesendet: Montag, 17. August 2020 18:37
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
This is the outcome in IDEA:
"Tests failed: 1, passed: 0"
So the behavior is the same with Maven.
On Mon, Aug 17, 2020 at 6:35 PM Tibor Digana wrote:
> Hi Markus,
>
> It is a specific problem related to the JUnit library because the test
> fails in IntelliJ IDEA and in Maven as well.
>
> The JUnit4 assumptions fail with yellow markers in IDEA but here the
> JUnit3' TestCase fails in red as a typical error or failure.
> And Maven fails this test as follows but i think this behavior starts in
> the junit library itself.
>
> [INFO] Running test.TestAssumptions
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 0.028 s <<< FAILURE! - in test.TestAssumptions
> [ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR!
> org.junit.AssumptionViolatedException: got: , expected: is
> at test/test.TestAssumptions.testABC(TestAssumptions.java:9)
>
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: ,
> expected: is [INFO]
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD FAILURE
>
> On Mon, Aug 17, 2020 at 3:07 PM Markus KARG
> wrote:
>
>> Martin,
>>
>> I don't understand your question.
>>
>> You posted the Javadocs, and they tell it clearly: "A test for which an
>> assumption fails should NOT generate a test case failure.".
>>
>> Also Assume's Javadocs
>> (https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are pretty
>> clear about it: "... A failed assumption does NOT mean the code is broken,
>> but that the test provides no useful information. Assume basically means
>> "don't RUN this test if these conditions don't apply". The default JUnit
>> runner SKIPS tests with failing assumptions..."
>>
>> So do you really want me to ask the JUnit people whether they really MEAN
>> what they clearly wrote...?
>>
>> -Markus
>>
>> -Ursprüngliche Nachricht-
>> Von: Martin Gainty [mailto:mgai...@hotmail.com]
>> Gesendet: Montag, 17. August 2020 12:27
>> An: Maven Developers List
>> Betreff: Re: Assumption fail treated as unexcepted exception?!
>>
>>
>>/**
>> * Call to assume that actual satisfies the condition
>> specified by matcher.
>> * If not, the test halts and is ignored.
>> * Example:
>> * :
>> * assumeThat(1, is(1)); // passes
>> * foo(); // will execute
>> * assumeThat(0, is(1)); // assumption failure! test halts
>> * int x = 1 / 0; // will never execute
>> *
>> *
>> * @param the static type accepted by the matcher (this can flag
>> obvious compile-time problems such as {@code assumeThat(1, is("a"))}
>> * @param actual the computed value being compared
>> * @param matcher an expression, built of {@link Matcher}s, specifying
>> allowed values
>> * @see org.hamcrest.CoreMatchers
>> * @see org.junit.matchers.JUnitMatchers
>> */
>> public static void assumeThat(T actual, Matcher matcher) {
>> if (!matcher.matches(actual)) {
>> throw new AssumptionViolatedException(actual, matcher);
>> }
>> }
>>
>> //does AssumptionViolatedException ever produce test case failure?
>>
>> /**
>> * An exception class used to implement assumptions (state in
>> which a
>> given test
>> * is meaningful and should or should not be executed). A test for which
>> an
>> assumption
>> * fails should not generate a test case failure.
>> *
>> * @see org.junit.Assume
>> * @since 4.12
>> */
>> @SuppressWarnings("deprecation")
>> public class AssumptionViolatedException extends
>> org.junit.internal.AssumptionViolatedException {
>> private static final long serialVersionUID = 1L;
>>
>> /**
>>
Re: Assumption fail treated as unexcepted exception?!
This is the outcome in IDEA:
"Tests failed: 1, passed: 0"
So the behavior is the same with Maven.
On Mon, Aug 17, 2020 at 6:35 PM Tibor Digana wrote:
> Hi Markus,
>
> It is a specific problem related to the JUnit library because the test
> fails in IntelliJ IDEA and in Maven as well.
>
> The JUnit4 assumptions fail with yellow markers in IDEA but here the
> JUnit3' TestCase fails in red as a typical error or failure.
> And Maven fails this test as follows but i think this behavior starts in
> the junit library itself.
>
> [INFO] Running test.TestAssumptions
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 0.028 s <<< FAILURE! - in test.TestAssumptions
> [ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR!
> org.junit.AssumptionViolatedException: got: , expected: is
> at test/test.TestAssumptions.testABC(TestAssumptions.java:9)
>
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: ,
> expected: is [INFO]
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD FAILURE
>
> On Mon, Aug 17, 2020 at 3:07 PM Markus KARG
> wrote:
>
>> Martin,
>>
>> I don't understand your question.
>>
>> You posted the Javadocs, and they tell it clearly: "A test for which an
>> assumption fails should NOT generate a test case failure.".
>>
>> Also Assume's Javadocs
>> (https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are pretty
>> clear about it: "... A failed assumption does NOT mean the code is broken,
>> but that the test provides no useful information. Assume basically means
>> "don't RUN this test if these conditions don't apply". The default JUnit
>> runner SKIPS tests with failing assumptions..."
>>
>> So do you really want me to ask the JUnit people whether they really MEAN
>> what they clearly wrote...?
>>
>> -Markus
>>
>> -Ursprüngliche Nachricht-
>> Von: Martin Gainty [mailto:mgai...@hotmail.com]
>> Gesendet: Montag, 17. August 2020 12:27
>> An: Maven Developers List
>> Betreff: Re: Assumption fail treated as unexcepted exception?!
>>
>>
>>/**
>> * Call to assume that actual satisfies the condition
>> specified by matcher.
>> * If not, the test halts and is ignored.
>> * Example:
>> * :
>> * assumeThat(1, is(1)); // passes
>> * foo(); // will execute
>> * assumeThat(0, is(1)); // assumption failure! test halts
>> * int x = 1 / 0; // will never execute
>> *
>> *
>> * @param the static type accepted by the matcher (this can flag
>> obvious compile-time problems such as {@code assumeThat(1, is("a"))}
>> * @param actual the computed value being compared
>> * @param matcher an expression, built of {@link Matcher}s, specifying
>> allowed values
>> * @see org.hamcrest.CoreMatchers
>> * @see org.junit.matchers.JUnitMatchers
>> */
>> public static void assumeThat(T actual, Matcher matcher) {
>> if (!matcher.matches(actual)) {
>> throw new AssumptionViolatedException(actual, matcher);
>> }
>> }
>>
>> //does AssumptionViolatedException ever produce test case failure?
>>
>> /**
>> * An exception class used to implement assumptions (state in
>> which a
>> given test
>> * is meaningful and should or should not be executed). A test for which
>> an
>> assumption
>> * fails should not generate a test case failure.
>> *
>> * @see org.junit.Assume
>> * @since 4.12
>> */
>> @SuppressWarnings("deprecation")
>> public class AssumptionViolatedException extends
>> org.junit.internal.AssumptionViolatedException {
>> private static final long serialVersionUID = 1L;
>>
>> /**
>> * An assumption exception with the given actual value and a
>> matcher describing
>> * the expectation that failed.
>> */
>> public AssumptionViolatedException(T actual, Matcher matcher) {
>> super(actual, matcher);
>> }
>>
>> /**
>> * An assumption exception with a message with the given actual
>> value and a
>> * matcher describing the expectation that failed.
>> */
>> public AssumptionViolatedException(String message,
Re: Assumption fail treated as unexcepted exception?!
Hi Markus,
It is a specific problem related to the JUnit library because the test
fails in IntelliJ IDEA and in Maven as well.
The JUnit4 assumptions fail with yellow markers in IDEA but here the
JUnit3' TestCase fails in red as a typical error or failure.
And Maven fails this test as follows but i think this behavior starts in
the junit library itself.
[INFO] Running test.TestAssumptions
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
0.028 s <<< FAILURE! - in test.TestAssumptions
[ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR!
org.junit.AssumptionViolatedException: got: , expected: is
at test/test.TestAssumptions.testABC(TestAssumptions.java:9)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: ,
expected: is wrote:
> Martin,
>
> I don't understand your question.
>
> You posted the Javadocs, and they tell it clearly: "A test for which an
> assumption fails should NOT generate a test case failure.".
>
> Also Assume's Javadocs
> (https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are pretty
> clear about it: "... A failed assumption does NOT mean the code is broken,
> but that the test provides no useful information. Assume basically means
> "don't RUN this test if these conditions don't apply". The default JUnit
> runner SKIPS tests with failing assumptions..."
>
> So do you really want me to ask the JUnit people whether they really MEAN
> what they clearly wrote...?
>
> -Markus
>
> -Ursprüngliche Nachricht-
> Von: Martin Gainty [mailto:mgai...@hotmail.com]
> Gesendet: Montag, 17. August 2020 12:27
> An: Maven Developers List
> Betreff: Re: Assumption fail treated as unexcepted exception?!
>
>
>/**
> * Call to assume that actual satisfies the condition
> specified by matcher.
> * If not, the test halts and is ignored.
> * Example:
> * :
> * assumeThat(1, is(1)); // passes
> * foo(); // will execute
> * assumeThat(0, is(1)); // assumption failure! test halts
> * int x = 1 / 0; // will never execute
> *
> *
> * @param the static type accepted by the matcher (this can flag
> obvious compile-time problems such as {@code assumeThat(1, is("a"))}
> * @param actual the computed value being compared
> * @param matcher an expression, built of {@link Matcher}s, specifying
> allowed values
> * @see org.hamcrest.CoreMatchers
> * @see org.junit.matchers.JUnitMatchers
> */
> public static void assumeThat(T actual, Matcher matcher) {
> if (!matcher.matches(actual)) {
> throw new AssumptionViolatedException(actual, matcher);
> }
> }
>
> //does AssumptionViolatedException ever produce test case failure?
>
> /**
> * An exception class used to implement assumptions (state in which
> a
> given test
> * is meaningful and should or should not be executed). A test for which an
> assumption
> * fails should not generate a test case failure.
> *
> * @see org.junit.Assume
> * @since 4.12
> */
> @SuppressWarnings("deprecation")
> public class AssumptionViolatedException extends
> org.junit.internal.AssumptionViolatedException {
> private static final long serialVersionUID = 1L;
>
> /**
> * An assumption exception with the given actual value and a
> matcher describing
> * the expectation that failed.
> */
> public AssumptionViolatedException(T actual, Matcher matcher) {
> super(actual, matcher);
> }
>
> /**
> * An assumption exception with a message with the given actual
> value and a
> * matcher describing the expectation that failed.
> */
> public AssumptionViolatedException(String message, T expected,
> Matcher matcher) {
> super(message, expected, matcher);
> }
>
> /**
> * An assumption exception with the given message only.
> */
> public AssumptionViolatedException(String message) {
> super(message);
> }
>
> /**
> * An assumption exception with the given message and a cause.
> */
> public AssumptionViolatedException(String assumption, Throwable t) {
> super(assumption, t);
> }
> }
>
> //will base class throw FAILURE for TestCase?
> /**
> * An exception class used to implement assumptions (state in which
> a
> given test
> * is meaningful and should or should not be executed). A test for which an
> assumption
> * fails should not generate a test case failure.
> *
> * @see org.junit.Assume
&g
AW: Assumption fail treated as unexcepted exception?!
Martin,
I don't understand your question.
You posted the Javadocs, and they tell it clearly: "A test for which an
assumption fails should NOT generate a test case failure.".
Also Assume's Javadocs
(https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are pretty
clear about it: "... A failed assumption does NOT mean the code is broken,
but that the test provides no useful information. Assume basically means
"don't RUN this test if these conditions don't apply". The default JUnit
runner SKIPS tests with failing assumptions..."
So do you really want me to ask the JUnit people whether they really MEAN
what they clearly wrote...?
-Markus
-Ursprüngliche Nachricht-
Von: Martin Gainty [mailto:mgai...@hotmail.com]
Gesendet: Montag, 17. August 2020 12:27
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
/**
* Call to assume that actual satisfies the condition
specified by matcher.
* If not, the test halts and is ignored.
* Example:
* :
* assumeThat(1, is(1)); // passes
* foo(); // will execute
* assumeThat(0, is(1)); // assumption failure! test halts
* int x = 1 / 0; // will never execute
*
*
* @param the static type accepted by the matcher (this can flag
obvious compile-time problems such as {@code assumeThat(1, is("a"))}
* @param actual the computed value being compared
* @param matcher an expression, built of {@link Matcher}s, specifying
allowed values
* @see org.hamcrest.CoreMatchers
* @see org.junit.matchers.JUnitMatchers
*/
public static void assumeThat(T actual, Matcher matcher) {
if (!matcher.matches(actual)) {
throw new AssumptionViolatedException(actual, matcher);
}
}
//does AssumptionViolatedException ever produce test case failure?
/**
* An exception class used to implement assumptions (state in which a
given test
* is meaningful and should or should not be executed). A test for which an
assumption
* fails should not generate a test case failure.
*
* @see org.junit.Assume
* @since 4.12
*/
@SuppressWarnings("deprecation")
public class AssumptionViolatedException extends
org.junit.internal.AssumptionViolatedException {
private static final long serialVersionUID = 1L;
/**
* An assumption exception with the given actual value and a
matcher describing
* the expectation that failed.
*/
public AssumptionViolatedException(T actual, Matcher matcher) {
super(actual, matcher);
}
/**
* An assumption exception with a message with the given actual
value and a
* matcher describing the expectation that failed.
*/
public AssumptionViolatedException(String message, T expected,
Matcher matcher) {
super(message, expected, matcher);
}
/**
* An assumption exception with the given message only.
*/
public AssumptionViolatedException(String message) {
super(message);
}
/**
* An assumption exception with the given message and a cause.
*/
public AssumptionViolatedException(String assumption, Throwable t) {
super(assumption, t);
}
}
//will base class throw FAILURE for TestCase?
/**
* An exception class used to implement assumptions (state in which a
given test
* is meaningful and should or should not be executed). A test for which an
assumption
* fails should not generate a test case failure.
*
* @see org.junit.Assume
*/
public class AssumptionViolatedException extends RuntimeException implements
SelfDescribing {
private static final long serialVersionUID = 2L;
/*
* We have to use the f prefix until the next major release to ensure
* serialization compatibility.
* See https://github.com/junit-team/junit/issues/976
*/
private final String fAssumption;
private final boolean fValueMatcher;
private final Object fValue;
private final Matcher fMatcher;
/**
* @deprecated Please use {@link org.junit.AssumptionViolatedException}
instead.
*/
@Deprecated
public AssumptionViolatedException(String assumption, boolean hasValue,
Object value, Matcher matcher) {
this.fAssumption = assumption;
this.fValue = value;
this.fMatcher = matcher;
this.fValueMatcher = hasValue;
if (value instanceof Throwable) {
initCause((Throwable) value);
}
}
/**
* An assumption exception with the given value (String or
* Throwable) and an additional failing {@link Matcher}.
*
* @deprecated Please use {@link org.junit.AssumptionViolatedException}
instead.
*/
@Deprecated
public AssumptionViolatedException(Object value, Matcher matcher) {
this(null, true, value, matcher);
}
/**
* An assumption exception with the given value (String or
* Throwable) and an additiona
AW: Assumption fail treated as unexcepted exception?!
So the bug is that Maven Dependency Plugin's test suite uses the JUnit 4 library for test-compile, but actually executes the tests using JUnit 3, right? -Markus -Ursprüngliche Nachricht- Von: Tibor Digana [mailto:tibordig...@apache.org] Gesendet: Montag, 17. August 2020 14:09 An: Maven Developers List Betreff: Re: Assumption fail treated as unexcepted exception?! Yes Piotr, this is another example why Markus has these unpleasant experiences with JUnit3. The code with JUnit4 annotation should be fine. On Mon, Aug 17, 2020 at 1:34 PM Piotr Żygieło wrote: > Please note, that TestCopyMojo is considered as JUnit3 TestCase > ( > https://github.com/junit-team/junit4/blob/3a5c6b4d08f408c8ca6a8e0bae71a9bc5a8f97e8/src/main/java/org/junit/internal/builders/JUnit3Builder.java#L17 > ). > So perhaps JUnit4+ features are not to be expected in such a case? > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
AW: Assumption fail treated as unexcepted exception?!
Just to make clear: *I* did not add *anything* to make it go wrong -- I just
execute the existing Maven Dependency Plugin's test suite into which I added an
assumption. Neither did I write that plugin nor its tests nor its POM. Just to
clarify that.
-Markus
-Ursprüngliche Nachricht-
Von: Tibor Digana [mailto:tibordig...@apache.org]
Gesendet: Montag, 17. August 2020 13:27
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
yes it should be marked Skipped.
Many factors may turn this. As for instance I have these experiences with
Powermock which always fails the test in IntelliJ IDEA even if the
assumption fails.
Check it out with almost empty POM and a trivial test and compare the
outcome with the one you have attached. And then you will maybe see more.
@Test
public void test() {
assumeTrue(false);
}
This code should work with JUnit 4.12 and whatever plugin version.
Turn this code to your specifics one by one and you will see what you have
added necessarily has changed the behavior.
T
On Sun, Aug 16, 2020 at 1:40 PM Markus KARG wrote:
> Guys,
>
>
>
> I'm stuck with working on a new feature due to this, so I hope you can help
> me quickly:
>
>
>
> JUnit knows assumptions, assertions and exceptions. Failing assertions will
> FAIL tests (hence will fail maven builds). Failing assumptions will SKIP
> tests (hence will pass maven builds). Exceptions will ERROR tests (hence
> will break maven builds). Unfortunately today I noticed that assumptions
> actually ERROR test (hence fail builds) in Maven!
>
>
>
> (I simply added another test to maven dependency plugin which contains an
> always-failing assumption to proof the claim)
>
>
>
> [INFO] Results:
>
> [INFO]
>
> [ERROR] Errors:
>
> [ERROR] TestCopyMojo.proofClaim:274 ┐ AssumptionViolated always skip
>
> [INFO]
>
> [ERROR] Tests run: 257, Failures: 0, Errors: 1, Skipped: 0
>
>
>
> It seems maven treats JUnit assumptions just like JUnit exceptions! :-(
>
>
>
> Can someone tell my why that happens? Do I have to set some option tell
> Maven it shall SKIP instead of ERROR on failing assumptions?!
>
>
>
> Thanks in advance!
>
> -Markus Karg
>
>
>
>
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
Re: Assumption fail treated as unexcepted exception?!
Yes Piotr, this is another example why Markus has these unpleasant experiences with JUnit3. The code with JUnit4 annotation should be fine. On Mon, Aug 17, 2020 at 1:34 PM Piotr Żygieło wrote: > Please note, that TestCopyMojo is considered as JUnit3 TestCase > ( > https://github.com/junit-team/junit4/blob/3a5c6b4d08f408c8ca6a8e0bae71a9bc5a8f97e8/src/main/java/org/junit/internal/builders/JUnit3Builder.java#L17 > ). > So perhaps JUnit4+ features are not to be expected in such a case? > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
Re: Assumption fail treated as unexcepted exception?!
Please note, that TestCopyMojo is considered as JUnit3 TestCase (https://github.com/junit-team/junit4/blob/3a5c6b4d08f408c8ca6a8e0bae71a9bc5a8f97e8/src/main/java/org/junit/internal/builders/JUnit3Builder.java#L17). So perhaps JUnit4+ features are not to be expected in such a case? - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Assumption fail treated as unexcepted exception?!
yes it should be marked Skipped.
Many factors may turn this. As for instance I have these experiences with
Powermock which always fails the test in IntelliJ IDEA even if the
assumption fails.
Check it out with almost empty POM and a trivial test and compare the
outcome with the one you have attached. And then you will maybe see more.
@Test
public void test() {
assumeTrue(false);
}
This code should work with JUnit 4.12 and whatever plugin version.
Turn this code to your specifics one by one and you will see what you have
added necessarily has changed the behavior.
T
On Sun, Aug 16, 2020 at 1:40 PM Markus KARG wrote:
> Guys,
>
>
>
> I'm stuck with working on a new feature due to this, so I hope you can help
> me quickly:
>
>
>
> JUnit knows assumptions, assertions and exceptions. Failing assertions will
> FAIL tests (hence will fail maven builds). Failing assumptions will SKIP
> tests (hence will pass maven builds). Exceptions will ERROR tests (hence
> will break maven builds). Unfortunately today I noticed that assumptions
> actually ERROR test (hence fail builds) in Maven!
>
>
>
> (I simply added another test to maven dependency plugin which contains an
> always-failing assumption to proof the claim)
>
>
>
> [INFO] Results:
>
> [INFO]
>
> [ERROR] Errors:
>
> [ERROR] TestCopyMojo.proofClaim:274 ┐ AssumptionViolated always skip
>
> [INFO]
>
> [ERROR] Tests run: 257, Failures: 0, Errors: 1, Skipped: 0
>
>
>
> It seems maven treats JUnit assumptions just like JUnit exceptions! :-(
>
>
>
> Can someone tell my why that happens? Do I have to set some option tell
> Maven it shall SKIP instead of ERROR on failing assumptions?!
>
>
>
> Thanks in advance!
>
> -Markus Karg
>
>
>
>
Re: Assumption fail treated as unexcepted exception?!
initCause(e);
}
@Override
public String getMessage() {
return StringDescription.asString(this);
}
public void describeTo(Description description) {
if (fAssumption != null) {
description.appendText(fAssumption);
}
if (fValueMatcher) {
// a value was passed in when this instance was constructed; print
it
if (fAssumption != null) {
description.appendText(": ");
}
description.appendText("got: ");
description.appendValue(fValue);
if (fMatcher != null) {
description.appendText(", expected: ");
description.appendDescriptionOf(fMatcher);
}
}
}
}
//at least with junit 4.12 if AssumptionViolatedException will not FAIL the
test-case
//AssumptionViolatedException falls between the exception-handlers to produce
ERROR
can you verify with junit author(s) AssumptionViolatedException does not FAIL
test-case is intended behaviour?
martin
From: Markus KARG
Sent: Sunday, August 16, 2020 1:39 PM
To: 'Maven Developers List'
Subject: AW: Assumption fail treated as unexcepted exception?!
The debug output is quite huge, so I won't put it here. What in particular
shall I lookup inside of that? The environment is:
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
2019-04-04T21:00:29+02:00)
Maven home: C:\Program Files\apache-maven-3.6.1
Java version: 1.8.0_212, vendor: Azul Systems, Inc., runtime: C:\Program
Files\zulu8.38.0.13-ca-jdk8.0.212-win_x64\jre
Default locale: de_DE, platform encoding: Cp1252
OS name: "windows 8.1", version: "6.3", arch: "amd64", family: "windows"
...
[DEBUG] Goal:
org.apache.maven.plugins:maven-surefire-plugin:2.22.0:test (default-test)
[DEBUG] Style: Regular
[DEBUG] Configuration:
${maven.test.additionalClasspath}
-Xmx384m
${childDelegation}
${maven.test.dependency.excludes}
${maven.surefire.debug}
${dependenciesToScan}
${disableXmlReport}
${enableAssertions}
${surefire.encoding}
true
${excludedGroups}
${surefire.excludesFile}
${surefire.failIfNoSpecifiedTests}
${failIfNoTests}
${forkCount}
${forkMode}
${surefire.exitTimeout}
${surefire.timeout}
${groups}
${surefire.includesFile}
${junitArtifactName}
${junitPlatformArti
factName}
${jvm}
${objectFactory}
${parallel}
${parallelOptimized}
${surefire.parallel.forcedTimeout}
${surefire.parallel.timeout}
${perCoreThreadCount}
${plugin.artifactMap}
${surefire.printSummary}
${project.artifactMap}
${maven.test.redirectTestOutputToFile}
${surefire.reportFormat}
${surefire.reportNameSuffix}
${surefire.rerunFailingTestsCount}
${reuseForks}
${surefire.runOrder}
${surefire.shutdown}
${maven.test.skip}
${surefire.skipAfterFailureCount}
${maven.test.skip.exec}
${skipTests}
${surefire.suiteXmlFiles}
C:\Program Files\apache-maven-3.6.1
${tempDir}
${test}
${maven.test.failure.ignore}
${testNGArtifactName}
${threadCount}
${threadCountClasses}
${threadCountMethods}
${threadCountSuites}
${trimStackTrace}
${surefire.useFile}
${surefire.useManifestOnlyJar}
${surefire.useSystemClassLoader}
${useUnlimitedThreads}
${basedir}
...
[ERROR]
testProofClaim(org.apache.maven.plugins.dependency.fromConfiguration.TestCop
yMojo) Time elapsed: 0.146 s <<< ERROR!
org.junit.AssumptionViolatedException: always skip
at
org.apache.maven.plugins.dependency.fromConfiguration.TestCopyMojo.testProof
Claim(TestCopyMojo.java:76)
...
IIUC, Maven debug doesn't contain any really specific information besides
what I aready assumed: Surefire treats org.junit.AssumptionViolatedException
as an error but not as a request to skip the test.
This can be reproduced with this really simple test case:
https://github.com/mkarg/maven-dependency-plugin/commit/af257d7987fc41ac4377
4d2dca60b201979d11a2 - as you can see, I just ask JUnit 4.13 to fail the
assumptions, and JUnit CORRECTLY throws AssumptionViolatedException - but
Maven seems to not specifically deal with it.
-Markus
-Ursprüngliche Nachricht-
Von: Martin Gainty [mailto:mgai...@hotmail.com]
Gesendet: Sonntag, 16. August 2020 14:24
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
MG>below
________________
From: Markus KARG
Sent: Sunday, August 16, 2020 7:40 AM
To: dev@maven.apache.org
Subject: Assumption fail treated as unexcepted exception?!
Guys,
I'm stuck with working on a new feature due to this, so I hope you can help
me quickly:
JUnit knows assumptions, assertions and exceptions. Failing assertions will
FAIL tests (hence will fail maven builds). Failing assumptions will SKIP
tests (hence will pass maven builds). Excepti
Re: Assumption fail treated as unexcepted exception?!
I see TestCopyMojo runs with JUnit38ClassRunner. Is it Assumptions[Assume since 4.4, AssumptionViolatedException since 4.12]-proof? Piotrek - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
AW: Assumption fail treated as unexcepted exception?!
Please see my answer to Martin Gainty, it proofs there is no NPE but
AssumptionViolatedException. It is a wrong assumption that all thrown
exceptions lead to test fail. It is JUnit's normal behavior that
Assume.assumeTrue(false) will throw AssumptionViolatedException, so that one
explicitly needs special treatment to skip the test, as this is the intention
of that JUnit API.
-Markus
-Ursprüngliche Nachricht-
Von: John Patrick [mailto:nhoj.patr...@gmail.com]
Gesendet: Sonntag, 16. August 2020 14:47
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
If the code working out the assumption fails and causes a throwable,
either Runtime or Checked, I would expect the test to fail.
If you have a line something like below, and that causes a test error,
then that feels wrong as it's a valid assumption.
Assumptions.assumeFalse("ABC".equals("ABC"), "message");
But if you had below, which would throw an NPE, then that should error
as it's a badly coded assumption logic.
String IS_NULL = null;
Assumptions.assumeFalse(IS_NULL.equals("ABC"), "message");
So without seeing the test involved it's hard to tell if it's correct or not.
John
On Sun, 16 Aug 2020 at 13:31, Martin Gainty wrote:
>
> MG>below
>
>
> From: Markus KARG
> Sent: Sunday, August 16, 2020 7:40 AM
> To: dev@maven.apache.org
> Subject: Assumption fail treated as unexcepted exception?!
>
> Guys,
>
>
>
> I'm stuck with working on a new feature due to this, so I hope you can help
> me quickly:
>
>
>
> JUnit knows assumptions, assertions and exceptions. Failing assertions will
> FAIL tests (hence will fail maven builds). Failing assumptions will SKIP
> tests (hence will pass maven builds). Exceptions will ERROR tests (hence
> will break maven builds). Unfortunately today I noticed that assumptions
> actually ERROR test (hence fail builds) in Maven!
>
>
>
> (I simply added another test to maven dependency plugin which contains an
> always-failing assumption to proof the claim)
>
>
>
> [INFO] Results:
>
> [INFO]
>
> [ERROR] Errors:
>
> [ERROR] TestCopyMojo.proofClaim:274 ┐ AssumptionViolated always skip
>
> MG>get maven environment and debug information
> MG>mvn -e -X
> MG>also I know Junit 5.4.2 needs Hamcrest to be on classpath
> MG>can you tell us which version of Junit you are using?
>
> -Markus Karg
> ~gruss~
> ~martin~
>
>
>
>
>
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
AW: Assumption fail treated as unexcepted exception?!
The debug output is quite huge, so I won't put it here. What in particular
shall I lookup inside of that? The environment is:
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
2019-04-04T21:00:29+02:00)
Maven home: C:\Program Files\apache-maven-3.6.1
Java version: 1.8.0_212, vendor: Azul Systems, Inc., runtime: C:\Program
Files\zulu8.38.0.13-ca-jdk8.0.212-win_x64\jre
Default locale: de_DE, platform encoding: Cp1252
OS name: "windows 8.1", version: "6.3", arch: "amd64", family: "windows"
...
[DEBUG] Goal:
org.apache.maven.plugins:maven-surefire-plugin:2.22.0:test (default-test)
[DEBUG] Style: Regular
[DEBUG] Configuration:
${maven.test.additionalClasspath}
-Xmx384m
${childDelegation}
${maven.test.dependency.excludes}
${maven.surefire.debug}
${dependenciesToScan}
${disableXmlReport}
${enableAssertions}
${surefire.encoding}
true
${excludedGroups}
${surefire.excludesFile}
${surefire.failIfNoSpecifiedTests}
${failIfNoTests}
${forkCount}
${forkMode}
${surefire.exitTimeout}
${surefire.timeout}
${groups}
${surefire.includesFile}
${junitArtifactName}
${junitPlatformArti
factName}
${jvm}
${objectFactory}
${parallel}
${parallelOptimized}
${surefire.parallel.forcedTimeout}
${surefire.parallel.timeout}
${perCoreThreadCount}
${plugin.artifactMap}
${surefire.printSummary}
${project.artifactMap}
${maven.test.redirectTestOutputToFile}
${surefire.reportFormat}
${surefire.reportNameSuffix}
${surefire.rerunFailingTestsCount}
${reuseForks}
${surefire.runOrder}
${surefire.shutdown}
${maven.test.skip}
${surefire.skipAfterFailureCount}
${maven.test.skip.exec}
${skipTests}
${surefire.suiteXmlFiles}
C:\Program Files\apache-maven-3.6.1
${tempDir}
${test}
${maven.test.failure.ignore}
${testNGArtifactName}
${threadCount}
${threadCountClasses}
${threadCountMethods}
${threadCountSuites}
${trimStackTrace}
${surefire.useFile}
${surefire.useManifestOnlyJar}
${surefire.useSystemClassLoader}
${useUnlimitedThreads}
${basedir}
...
[ERROR]
testProofClaim(org.apache.maven.plugins.dependency.fromConfiguration.TestCop
yMojo) Time elapsed: 0.146 s <<< ERROR!
org.junit.AssumptionViolatedException: always skip
at
org.apache.maven.plugins.dependency.fromConfiguration.TestCopyMojo.testProof
Claim(TestCopyMojo.java:76)
...
IIUC, Maven debug doesn't contain any really specific information besides
what I aready assumed: Surefire treats org.junit.AssumptionViolatedException
as an error but not as a request to skip the test.
This can be reproduced with this really simple test case:
https://github.com/mkarg/maven-dependency-plugin/commit/af257d7987fc41ac4377
4d2dca60b201979d11a2 - as you can see, I just ask JUnit 4.13 to fail the
assumptions, and JUnit CORRECTLY throws AssumptionViolatedException - but
Maven seems to not specifically deal with it.
-Markus
-Ursprüngliche Nachricht-
Von: Martin Gainty [mailto:mgai...@hotmail.com]
Gesendet: Sonntag, 16. August 2020 14:24
An: Maven Developers List
Betreff: Re: Assumption fail treated as unexcepted exception?!
MG>below
From: Markus KARG
Sent: Sunday, August 16, 2020 7:40 AM
To: dev@maven.apache.org
Subject: Assumption fail treated as unexcepted exception?!
Guys,
I'm stuck with working on a new feature due to this, so I hope you can help
me quickly:
JUnit knows assumptions, assertions and exceptions. Failing assertions will
FAIL tests (hence will fail maven builds). Failing assumptions will SKIP
tests (hence will pass maven builds). Exceptions will ERROR tests (hence
will break maven builds). Unfortunately today I noticed that assumptions
actually ERROR test (hence fail builds) in Maven!
(I simply added another test to maven dependency plugin which contains an
always-failing assumption to proof the claim)
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestCopyMojo.proofClaim:274 ┐ AssumptionViolated always skip
MG>get maven environment and debug information
MG>mvn -e -X
MG>also I know Junit 5.4.2 needs Hamcrest to be on classpath
MG>can you tell us which version of Junit you are using?
-Markus Karg
~gruss~
~martin~
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
Re: Assumption fail treated as unexcepted exception?!
If the code working out the assumption fails and causes a throwable,
either Runtime or Checked, I would expect the test to fail.
If you have a line something like below, and that causes a test error,
then that feels wrong as it's a valid assumption.
Assumptions.assumeFalse("ABC".equals("ABC"), "message");
But if you had below, which would throw an NPE, then that should error
as it's a badly coded assumption logic.
String IS_NULL = null;
Assumptions.assumeFalse(IS_NULL.equals("ABC"), "message");
So without seeing the test involved it's hard to tell if it's correct or not.
John
On Sun, 16 Aug 2020 at 13:31, Martin Gainty wrote:
>
> MG>below
>
>
> From: Markus KARG
> Sent: Sunday, August 16, 2020 7:40 AM
> To: dev@maven.apache.org
> Subject: Assumption fail treated as unexcepted exception?!
>
> Guys,
>
>
>
> I'm stuck with working on a new feature due to this, so I hope you can help
> me quickly:
>
>
>
> JUnit knows assumptions, assertions and exceptions. Failing assertions will
> FAIL tests (hence will fail maven builds). Failing assumptions will SKIP
> tests (hence will pass maven builds). Exceptions will ERROR tests (hence
> will break maven builds). Unfortunately today I noticed that assumptions
> actually ERROR test (hence fail builds) in Maven!
>
>
>
> (I simply added another test to maven dependency plugin which contains an
> always-failing assumption to proof the claim)
>
>
>
> [INFO] Results:
>
> [INFO]
>
> [ERROR] Errors:
>
> [ERROR] TestCopyMojo.proofClaim:274 ┐ AssumptionViolated always skip
>
> MG>get maven environment and debug information
> MG>mvn -e -X
> MG>also I know Junit 5.4.2 needs Hamcrest to be on classpath
> MG>can you tell us which version of Junit you are using?
>
> -Markus Karg
> ~gruss~
> ~martin~
>
>
>
>
>
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
Re: Assumption fail treated as unexcepted exception?!
MG>below From: Markus KARG Sent: Sunday, August 16, 2020 7:40 AM To: dev@maven.apache.org Subject: Assumption fail treated as unexcepted exception?! Guys, I'm stuck with working on a new feature due to this, so I hope you can help me quickly: JUnit knows assumptions, assertions and exceptions. Failing assertions will FAIL tests (hence will fail maven builds). Failing assumptions will SKIP tests (hence will pass maven builds). Exceptions will ERROR tests (hence will break maven builds). Unfortunately today I noticed that assumptions actually ERROR test (hence fail builds) in Maven! (I simply added another test to maven dependency plugin which contains an always-failing assumption to proof the claim) [INFO] Results: [INFO] [ERROR] Errors: [ERROR] TestCopyMojo.proofClaim:274 ┐ AssumptionViolated always skip MG>get maven environment and debug information MG>mvn -e -X MG>also I know Junit 5.4.2 needs Hamcrest to be on classpath MG>can you tell us which version of Junit you are using? -Markus Karg ~gruss~ ~martin~
AW: Assumption fail treated as unexcepted exception?!
Happens with surefire 2.22.0 (but also with 3.0.0-M5, just tried it). Just added the assumption, did not otherwise touch any config properties, but maybe maven dependency plugin's existing build does that (haven't read the complete source). Any clue which property to search for which could produce that? Thanks -Markus -Ursprüngliche Nachricht- Von: Falko Modler [mailto:f.mod...@gmx.net] Gesendet: Sonntag, 16. August 2020 13:53 An: dev@maven.apache.org Betreff: Assumption fail treated as unexcepted exception?! Which surefire plugin version? Have you set any config properties? Cheers, Falko - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Assumption fail treated as unexcepted exception?!
Which surefire plugin version? Have you set any config properties? Cheers, Falko - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Assumption fail treated as unexcepted exception?!
Guys, I'm stuck with working on a new feature due to this, so I hope you can help me quickly: JUnit knows assumptions, assertions and exceptions. Failing assertions will FAIL tests (hence will fail maven builds). Failing assumptions will SKIP tests (hence will pass maven builds). Exceptions will ERROR tests (hence will break maven builds). Unfortunately today I noticed that assumptions actually ERROR test (hence fail builds) in Maven! (I simply added another test to maven dependency plugin which contains an always-failing assumption to proof the claim) [INFO] Results: [INFO] [ERROR] Errors: [ERROR] TestCopyMojo.proofClaim:274 ┐ AssumptionViolated always skip [INFO] [ERROR] Tests run: 257, Failures: 0, Errors: 1, Skipped: 0 It seems maven treats JUnit assumptions just like JUnit exceptions! :-( Can someone tell my why that happens? Do I have to set some option tell Maven it shall SKIP instead of ERROR on failing assumptions?! Thanks in advance! -Markus Karg
