This is the outcome in IDEA: "Tests failed: 1, passed: 0"
So the behavior is the same with Maven. On Mon, Aug 17, 2020 at 6:35 PM Tibor Digana <tibordig...@apache.org> wrote: > Hi Markus, > > It is a specific problem related to the JUnit library because the test > fails in IntelliJ IDEA and in Maven as well. > > The JUnit4 assumptions fail with yellow markers in IDEA but here the > JUnit3' TestCase fails in red as a typical error or failure. > And Maven fails this test as follows but i think this behavior starts in > the junit library itself. > > [INFO] Running test.TestAssumptions > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 0.028 s <<< FAILURE! - in test.TestAssumptions > [ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR! > org.junit.AssumptionViolatedException: got: <false>, expected: is <true> > at test/test.TestAssumptions.testABC(TestAssumptions.java:9) > > [INFO] > [INFO] Results: > [INFO] > [ERROR] Errors: > [ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: <false>, > expected: is <tru... > [INFO] > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 > [INFO] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > > On Mon, Aug 17, 2020 at 3:07 PM Markus KARG <mar...@headcrashing.eu> > wrote: > >> Martin, >> >> I don't understand your question. >> >> You posted the Javadocs, and they tell it clearly: "A test for which an >> assumption fails should NOT generate a test case failure.". >> >> Also Assume's Javadocs >> (https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are pretty >> clear about it: "... A failed assumption does NOT mean the code is broken, >> but that the test provides no useful information. Assume basically means >> "don't RUN this test if these conditions don't apply". The default JUnit >> runner SKIPS tests with failing assumptions..." >> >> So do you really want me to ask the JUnit people whether they really MEAN >> what they clearly wrote...? >> >> -Markus >> >> -----Ursprüngliche Nachricht----- >> Von: Martin Gainty [mailto:mgai...@hotmail.com] >> Gesendet: Montag, 17. August 2020 12:27 >> An: Maven Developers List >> Betreff: Re: Assumption fail treated as unexcepted exception?! >> >> >> /** >> * Call to assume that <code>actual</code> satisfies the condition >> specified by <code>matcher</code>. >> * If not, the test halts and is ignored. >> * Example: >> * <pre>: >> * assumeThat(1, is(1)); // passes >> * foo(); // will execute >> * assumeThat(0, is(1)); // assumption failure! test halts >> * int x = 1 / 0; // will never execute >> * </pre> >> * >> * @param <T> the static type accepted by the matcher (this can flag >> obvious compile-time problems such as {@code assumeThat(1, is("a"))} >> * @param actual the computed value being compared >> * @param matcher an expression, built of {@link Matcher}s, specifying >> allowed values >> * @see org.hamcrest.CoreMatchers >> * @see org.junit.matchers.JUnitMatchers >> */ >> public static <T> void assumeThat(T actual, Matcher<T> matcher) { >> if (!matcher.matches(actual)) { >> throw new AssumptionViolatedException(actual, matcher); >> } >> } >> >> //does AssumptionViolatedException ever produce test case failure? >> >> /** >> * An exception class used to implement <i>assumptions</i> (state in >> which a >> given test >> * is meaningful and should or should not be executed). A test for which >> an >> assumption >> * fails should not generate a test case failure. >> * >> * @see org.junit.Assume >> * @since 4.12 >> */ >> @SuppressWarnings("deprecation") >> public class AssumptionViolatedException extends >> org.junit.internal.AssumptionViolatedException { >> private static final long serialVersionUID = 1L; >> >> /** >> * An assumption exception with the given <i>actual</i> value and a >> <i>matcher</i> describing >> * the expectation that failed. >> */ >> public <T> AssumptionViolatedException(T actual, Matcher<T> matcher) { >> super(actual, matcher); >> } >> >> /** >> * An assumption exception with a message with the given <i>actual</i> >> value and a >> * <i>matcher</i> describing the expectation that failed. >> */ >> public <T> AssumptionViolatedException(String message, T expected, >> Matcher<T> matcher) { >> super(message, expected, matcher); >> } >> >> /** >> * An assumption exception with the given message only. >> */ >> public AssumptionViolatedException(String message) { >> super(message); >> } >> >> /** >> * An assumption exception with the given message and a cause. >> */ >> public AssumptionViolatedException(String assumption, Throwable t) { >> super(assumption, t); >> } >> } >> >> //will base class throw FAILURE for TestCase? >> /** >> * An exception class used to implement <i>assumptions</i> (state in >> which a >> given test >> * is meaningful and should or should not be executed). A test for which >> an >> assumption >> * fails should not generate a test case failure. >> * >> * @see org.junit.Assume >> */ >> public class AssumptionViolatedException extends RuntimeException >> implements >> SelfDescribing { >> private static final long serialVersionUID = 2L; >> >> /* >> * We have to use the f prefix until the next major release to ensure >> * serialization compatibility. >> * See https://github.com/junit-team/junit/issues/976 >> */ >> private final String fAssumption; >> private final boolean fValueMatcher; >> private final Object fValue; >> private final Matcher<?> fMatcher; >> >> /** >> * @deprecated Please use {@link >> org.junit.AssumptionViolatedException} >> instead. >> */ >> @Deprecated >> public AssumptionViolatedException(String assumption, boolean >> hasValue, >> Object value, Matcher<?> matcher) { >> this.fAssumption = assumption; >> this.fValue = value; >> this.fMatcher = matcher; >> this.fValueMatcher = hasValue; >> >> if (value instanceof Throwable) { >> initCause((Throwable) value); >> } >> } >> >> /** >> * An assumption exception with the given <i>value</i> (String or >> * Throwable) and an additional failing {@link Matcher}. >> * >> * @deprecated Please use {@link >> org.junit.AssumptionViolatedException} >> instead. >> */ >> @Deprecated >> public AssumptionViolatedException(Object value, Matcher<?> matcher) { >> this(null, true, value, matcher); >> } >> >> /** >> * An assumption exception with the given <i>value</i> (String or >> * Throwable) and an additional failing {@link Matcher}. >> * >> * @deprecated Please use {@link >> org.junit.AssumptionViolatedException} >> instead. >> */ >> @Deprecated >> public AssumptionViolatedException(String assumption, Object value, >> Matcher<?> matcher) { >> this(assumption, true, value, matcher); >> } >> >> /** >> * An assumption exception with the given message only. >> * >> * @deprecated Please use {@link >> org.junit.AssumptionViolatedException} >> instead. >> */ >> @Deprecated >> public AssumptionViolatedException(String assumption) { >> this(assumption, false, null, null); >> } >> >> /** >> * An assumption exception with the given message and a cause. >> * >> * @deprecated Please use {@link >> org.junit.AssumptionViolatedException} >> instead. >> */ >> @Deprecated >> public AssumptionViolatedException(String assumption, Throwable e) { >> this(assumption, false, null, null); >> initCause(e); >> } >> >> @Override >> public String getMessage() { >> return StringDescription.asString(this); >> } >> >> public void describeTo(Description description) { >> if (fAssumption != null) { >> description.appendText(fAssumption); >> } >> >> if (fValueMatcher) { >> // a value was passed in when this instance was constructed; >> print it >> if (fAssumption != null) { >> description.appendText(": "); >> } >> >> description.appendText("got: "); >> description.appendValue(fValue); >> >> if (fMatcher != null) { >> description.appendText(", expected: "); >> description.appendDescriptionOf(fMatcher); >> } >> } >> } >> } >> >> //at least with junit 4.12 if AssumptionViolatedException will not FAIL >> the >> test-case >> //AssumptionViolatedException falls between the exception-handlers to >> produce ERROR >> >> can you verify with junit author(s) AssumptionViolatedException does not >> FAIL test-case is intended behaviour? >> martin >> >> >> >> >> >> >> ________________________________ >> From: Markus KARG <mar...@headcrashing.eu> >> Sent: Sunday, August 16, 2020 1:39 PM >> To: 'Maven Developers List' <dev@maven.apache.org> >> Subject: AW: Assumption fail treated as unexcepted exception?! >> >> The debug output is quite huge, so I won't put it here. What in particular >> shall I lookup inside of that? The environment is: >> >> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; >> 2019-04-04T21:00:29+02:00) >> Maven home: C:\Program Files\apache-maven-3.6.1 >> Java version: 1.8.0_212, vendor: Azul Systems, Inc., runtime: C:\Program >> Files\zulu8.38.0.13-ca-jdk8.0.212-win_x64\jre >> Default locale: de_DE, platform encoding: Cp1252 >> OS name: "windows 8.1", version: "6.3", arch: "amd64", family: "windows" >> ... >> [DEBUG] Goal: >> org.apache.maven.plugins:maven-surefire-plugin:2.22.0:test (default-test) >> [DEBUG] Style: Regular >> [DEBUG] Configuration: <?xml version="1.0" encoding="UTF-8"?> >> <configuration> >> >> >> <additionalClasspathElements>${maven.test.additionalClasspath}</additionalCl >> asspathElements> >> <argLine>-Xmx384m</argLine> >> <basedir default-value="${basedir}"/> >> <childDelegation >> default-value="false">${childDelegation}</childDelegation> >> <classesDirectory default-value="${project.build.outputDirectory}"/> >> >> >> <classpathDependencyExcludes>${maven.test.dependency.excludes}</classpathDep >> endencyExcludes> >> <debugForkedProcess>${maven.surefire.debug}</debugForkedProcess> >> <dependenciesToScan>${dependenciesToScan}</dependenciesToScan> >> <disableXmlReport >> default-value="false">${disableXmlReport}</disableXmlReport> >> <enableAssertions >> default-value="true">${enableAssertions}</enableAssertions> >> <encoding >> >> default-value="${project.reporting.outputEncoding}">${surefire.encoding}</en >> coding> >> <environmentVariables> >> <JENKINS_MAVEN_AGENT_DISABLED>true</JENKINS_MAVEN_AGENT_DISABLED> >> </environmentVariables> >> <excludedGroups>${excludedGroups}</excludedGroups> >> <excludesFile>${surefire.excludesFile}</excludesFile> >> >> >> <failIfNoSpecifiedTests>${surefire.failIfNoSpecifiedTests}</failIfNoSpecifie >> dTests> >> <failIfNoTests>${failIfNoTests}</failIfNoTests> >> <forkCount default-value="1">${forkCount}</forkCount> >> <forkMode default-value="once">${forkMode}</forkMode> >> <forkedProcessExitTimeoutInSeconds >> >> default-value="30">${surefire.exitTimeout}</forkedProcessExitTimeoutInSecond >> s> >> >> >> <forkedProcessTimeoutInSeconds>${surefire.timeout}</forkedProcessTimeoutInSe >> conds> >> <groups>${groups}</groups> >> <includesFile>${surefire.includesFile}</includesFile> >> <junitArtifactName >> default-value="junit:junit">${junitArtifactName}</junitArtifactName> >> <junitPlatformArtifactName >> >> default-value="org.junit.platform:junit-platform-engine">${junitPlatformArti >> factName}</junitPlatformArtifactName> >> <jvm>${jvm}</jvm> >> <localRepository default-value="${localRepository}"/> >> <objectFactory>${objectFactory}</objectFactory> >> <parallel>${parallel}</parallel> >> <parallelMavenExecution default-value="${session.parallel}"/> >> <parallelOptimized >> default-value="true">${parallelOptimized}</parallelOptimized> >> >> >> <parallelTestsTimeoutForcedInSeconds>${surefire.parallel.forcedTimeout}</par >> allelTestsTimeoutForcedInSeconds> >> >> >> <parallelTestsTimeoutInSeconds>${surefire.parallel.timeout}</parallelTestsTi >> meoutInSeconds> >> <perCoreThreadCount >> default-value="true">${perCoreThreadCount}</perCoreThreadCount> >> <pluginArtifactMap>${plugin.artifactMap}</pluginArtifactMap> >> <pluginDescriptor default-value="${plugin}"/> >> <printSummary >> default-value="true">${surefire.printSummary}</printSummary> >> <projectArtifactMap>${project.artifactMap}</projectArtifactMap> >> <projectBuildDirectory default-value="${project.build.directory}"/> >> <redirectTestOutputToFile >> >> default-value="false">${maven.test.redirectTestOutputToFile}</redirectTestOu >> tputToFile> >> <remoteRepositories >> default-value="${project.pluginArtifactRepositories}"/> >> <reportFormat >> default-value="brief">${surefire.reportFormat}</reportFormat> >> <reportNameSuffix >> default-value="">${surefire.reportNameSuffix}</reportNameSuffix> >> <reportsDirectory >> default-value="${project.build.directory}/surefire-reports"/> >> <rerunFailingTestsCount >> >> default-value="0">${surefire.rerunFailingTestsCount}</rerunFailingTestsCount >> > >> <reuseForks default-value="true">${reuseForks}</reuseForks> >> <runOrder default-value="filesystem">${surefire.runOrder}</runOrder> >> <shutdown default-value="testset">${surefire.shutdown}</shutdown> >> <skip default-value="false">${maven.test.skip}</skip> >> <skipAfterFailureCount >> >> default-value="0">${surefire.skipAfterFailureCount}</skipAfterFailureCount> >> <skipExec>${maven.test.skip.exec}</skipExec> >> <skipTests default-value="false">${skipTests}</skipTests> >> <suiteXmlFiles>${surefire.suiteXmlFiles}</suiteXmlFiles> >> <systemPropertyVariables> >> <maven.home>C:\Program Files\apache-maven-3.6.1</maven.home> >> </systemPropertyVariables> >> <tempDir default-value="surefire">${tempDir}</tempDir> >> <test>${test}</test> >> <testClassesDirectory >> default-value="${project.build.testOutputDirectory}"/> >> <testFailureIgnore >> default-value="false">${maven.test.failure.ignore}</testFailureIgnore> >> <testNGArtifactName >> >> default-value="org.testng:testng">${testNGArtifactName}</testNGArtifactName> >> <testSourceDirectory >> default-value="${project.build.testSourceDirectory}"/> >> <threadCount>${threadCount}</threadCount> >> <threadCountClasses >> default-value="0">${threadCountClasses}</threadCountClasses> >> <threadCountMethods >> default-value="0">${threadCountMethods}</threadCountMethods> >> <threadCountSuites >> default-value="0">${threadCountSuites}</threadCountSuites> >> <trimStackTrace default-value="true">${trimStackTrace}</trimStackTrace> >> <useFile default-value="true">${surefire.useFile}</useFile> >> <useManifestOnlyJar >> default-value="true">${surefire.useManifestOnlyJar}</useManifestOnlyJar> >> <useSystemClassLoader >> >> default-value="true">${surefire.useSystemClassLoader}</useSystemClassLoader> >> <useUnlimitedThreads >> default-value="false">${useUnlimitedThreads}</useUnlimitedThreads> >> <workingDirectory>${basedir}</workingDirectory> >> <project default-value="${project}"/> >> <session default-value="${session}"/> >> </configuration> >> ... >> [ERROR] >> >> testProofClaim(org.apache.maven.plugins.dependency.fromConfiguration.TestCop >> yMojo) Time elapsed: 0.146 s <<< ERROR! >> org.junit.AssumptionViolatedException: always skip >> at >> >> org.apache.maven.plugins.dependency.fromConfiguration.TestCopyMojo.testProof >> Claim(TestCopyMojo.java:76) >> ... >> >> IIUC, Maven debug doesn't contain any really specific information besides >> what I aready assumed: Surefire treats >> org.junit.AssumptionViolatedException >> as an error but not as a request to skip the test. >> >> This can be reproduced with this really simple test case: >> >> https://github.com/mkarg/maven-dependency-plugin/commit/af257d7987fc41ac4377 >> 4d2dca60b201979d11a2 >> <https://github.com/mkarg/maven-dependency-plugin/commit/af257d7987fc41ac43774d2dca60b201979d11a2> >> - as you can see, I just ask JUnit 4.13 to fail the >> assumptions, and JUnit CORRECTLY throws AssumptionViolatedException - but >> Maven seems to not specifically deal with it. >> >> -Markus >> >> >> -----Ursprüngliche Nachricht----- >> Von: Martin Gainty [mailto:mgai...@hotmail.com] >> Gesendet: Sonntag, 16. August 2020 14:24 >> An: Maven Developers List >> Betreff: Re: Assumption fail treated as unexcepted exception?! >> >> MG>below >> >> ________________________________ >> From: Markus KARG <mar...@headcrashing.eu> >> Sent: Sunday, August 16, 2020 7:40 AM >> To: dev@maven.apache.org <dev@maven.apache.org> >> Subject: Assumption fail treated as unexcepted exception?! >> >> Guys, >> >> >> >> I'm stuck with working on a new feature due to this, so I hope you can >> help >> me quickly: >> >> >> >> JUnit knows assumptions, assertions and exceptions. Failing assertions >> will >> FAIL tests (hence will fail maven builds). Failing assumptions will SKIP >> tests (hence will pass maven builds). Exceptions will ERROR tests (hence >> will break maven builds). Unfortunately today I noticed that assumptions >> actually ERROR test (hence fail builds) in Maven! >> >> >> >> (I simply added another test to maven dependency plugin which contains an >> always-failing assumption to proof the claim) >> >> >> >> [INFO] Results: >> >> [INFO] >> >> [ERROR] Errors: >> >> [ERROR] TestCopyMojo.proofClaim:274 ┐ AssumptionViolated always skip >> >> MG>get maven environment and debug information >> MG>mvn -e -X >> MG>also I know Junit 5.4.2 needs Hamcrest to be on classpath >> MG>can you tell us which version of Junit you are using? >> >> -Markus Karg >> ~gruss~ >> ~martin~ >> >> >> >> >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> >>
