Hi Markus, It is a specific problem related to the JUnit library because the test fails in IntelliJ IDEA and in Maven as well.
The JUnit4 assumptions fail with yellow markers in IDEA but here the JUnit3' TestCase fails in red as a typical error or failure. And Maven fails this test as follows but i think this behavior starts in the junit library itself. [INFO] Running test.TestAssumptions [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.028 s <<< FAILURE! - in test.TestAssumptions [ERROR] test.TestAssumptions.testABC Time elapsed: 0.01 s <<< ERROR! org.junit.AssumptionViolatedException: got: <false>, expected: is <true> at test/test.TestAssumptions.testABC(TestAssumptions.java:9) [INFO] [INFO] Results: [INFO] [ERROR] Errors: [ERROR] TestAssumptions.testABC:9 » AssumptionViolated got: <false>, expected: is <tru... [INFO] [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE On Mon, Aug 17, 2020 at 3:07 PM Markus KARG <mar...@headcrashing.eu> wrote: > Martin, > > I don't understand your question. > > You posted the Javadocs, and they tell it clearly: "A test for which an > assumption fails should NOT generate a test case failure.". > > Also Assume's Javadocs > (https://junit.org/junit4/javadoc/4.12/org/junit/Assume.html) are pretty > clear about it: "... A failed assumption does NOT mean the code is broken, > but that the test provides no useful information. Assume basically means > "don't RUN this test if these conditions don't apply". The default JUnit > runner SKIPS tests with failing assumptions..." > > So do you really want me to ask the JUnit people whether they really MEAN > what they clearly wrote...? > > -Markus > > -----Ursprüngliche Nachricht----- > Von: Martin Gainty [mailto:mgai...@hotmail.com] > Gesendet: Montag, 17. August 2020 12:27 > An: Maven Developers List > Betreff: Re: Assumption fail treated as unexcepted exception?! > > > /** > * Call to assume that <code>actual</code> satisfies the condition > specified by <code>matcher</code>. > * If not, the test halts and is ignored. > * Example: > * <pre>: > * assumeThat(1, is(1)); // passes > * foo(); // will execute > * assumeThat(0, is(1)); // assumption failure! test halts > * int x = 1 / 0; // will never execute > * </pre> > * > * @param <T> the static type accepted by the matcher (this can flag > obvious compile-time problems such as {@code assumeThat(1, is("a"))} > * @param actual the computed value being compared > * @param matcher an expression, built of {@link Matcher}s, specifying > allowed values > * @see org.hamcrest.CoreMatchers > * @see org.junit.matchers.JUnitMatchers > */ > public static <T> void assumeThat(T actual, Matcher<T> matcher) { > if (!matcher.matches(actual)) { > throw new AssumptionViolatedException(actual, matcher); > } > } > > //does AssumptionViolatedException ever produce test case failure? > > /** > * An exception class used to implement <i>assumptions</i> (state in which > a > given test > * is meaningful and should or should not be executed). A test for which an > assumption > * fails should not generate a test case failure. > * > * @see org.junit.Assume > * @since 4.12 > */ > @SuppressWarnings("deprecation") > public class AssumptionViolatedException extends > org.junit.internal.AssumptionViolatedException { > private static final long serialVersionUID = 1L; > > /** > * An assumption exception with the given <i>actual</i> value and a > <i>matcher</i> describing > * the expectation that failed. > */ > public <T> AssumptionViolatedException(T actual, Matcher<T> matcher) { > super(actual, matcher); > } > > /** > * An assumption exception with a message with the given <i>actual</i> > value and a > * <i>matcher</i> describing the expectation that failed. > */ > public <T> AssumptionViolatedException(String message, T expected, > Matcher<T> matcher) { > super(message, expected, matcher); > } > > /** > * An assumption exception with the given message only. > */ > public AssumptionViolatedException(String message) { > super(message); > } > > /** > * An assumption exception with the given message and a cause. > */ > public AssumptionViolatedException(String assumption, Throwable t) { > super(assumption, t); > } > } > > //will base class throw FAILURE for TestCase? > /** > * An exception class used to implement <i>assumptions</i> (state in which > a > given test > * is meaningful and should or should not be executed). A test for which an > assumption > * fails should not generate a test case failure. > * > * @see org.junit.Assume > */ > public class AssumptionViolatedException extends RuntimeException > implements > SelfDescribing { > private static final long serialVersionUID = 2L; > > /* > * We have to use the f prefix until the next major release to ensure > * serialization compatibility. > * See https://github.com/junit-team/junit/issues/976 > */ > private final String fAssumption; > private final boolean fValueMatcher; > private final Object fValue; > private final Matcher<?> fMatcher; > > /** > * @deprecated Please use {@link org.junit.AssumptionViolatedException} > instead. > */ > @Deprecated > public AssumptionViolatedException(String assumption, boolean hasValue, > Object value, Matcher<?> matcher) { > this.fAssumption = assumption; > this.fValue = value; > this.fMatcher = matcher; > this.fValueMatcher = hasValue; > > if (value instanceof Throwable) { > initCause((Throwable) value); > } > } > > /** > * An assumption exception with the given <i>value</i> (String or > * Throwable) and an additional failing {@link Matcher}. > * > * @deprecated Please use {@link org.junit.AssumptionViolatedException} > instead. > */ > @Deprecated > public AssumptionViolatedException(Object value, Matcher<?> matcher) { > this(null, true, value, matcher); > } > > /** > * An assumption exception with the given <i>value</i> (String or > * Throwable) and an additional failing {@link Matcher}. > * > * @deprecated Please use {@link org.junit.AssumptionViolatedException} > instead. > */ > @Deprecated > public AssumptionViolatedException(String assumption, Object value, > Matcher<?> matcher) { > this(assumption, true, value, matcher); > } > > /** > * An assumption exception with the given message only. > * > * @deprecated Please use {@link org.junit.AssumptionViolatedException} > instead. > */ > @Deprecated > public AssumptionViolatedException(String assumption) { > this(assumption, false, null, null); > } > > /** > * An assumption exception with the given message and a cause. > * > * @deprecated Please use {@link org.junit.AssumptionViolatedException} > instead. > */ > @Deprecated > public AssumptionViolatedException(String assumption, Throwable e) { > this(assumption, false, null, null); > initCause(e); > } > > @Override > public String getMessage() { > return StringDescription.asString(this); > } > > public void describeTo(Description description) { > if (fAssumption != null) { > description.appendText(fAssumption); > } > > if (fValueMatcher) { > // a value was passed in when this instance was constructed; > print it > if (fAssumption != null) { > description.appendText(": "); > } > > description.appendText("got: "); > description.appendValue(fValue); > > if (fMatcher != null) { > description.appendText(", expected: "); > description.appendDescriptionOf(fMatcher); > } > } > } > } > > //at least with junit 4.12 if AssumptionViolatedException will not FAIL the > test-case > //AssumptionViolatedException falls between the exception-handlers to > produce ERROR > > can you verify with junit author(s) AssumptionViolatedException does not > FAIL test-case is intended behaviour? > martin > > > > > > > ________________________________ > From: Markus KARG <mar...@headcrashing.eu> > Sent: Sunday, August 16, 2020 1:39 PM > To: 'Maven Developers List' <dev@maven.apache.org> > Subject: AW: Assumption fail treated as unexcepted exception?! > > The debug output is quite huge, so I won't put it here. What in particular > shall I lookup inside of that? The environment is: > > Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; > 2019-04-04T21:00:29+02:00) > Maven home: C:\Program Files\apache-maven-3.6.1 > Java version: 1.8.0_212, vendor: Azul Systems, Inc., runtime: C:\Program > Files\zulu8.38.0.13-ca-jdk8.0.212-win_x64\jre > Default locale: de_DE, platform encoding: Cp1252 > OS name: "windows 8.1", version: "6.3", arch: "amd64", family: "windows" > ... > [DEBUG] Goal: > org.apache.maven.plugins:maven-surefire-plugin:2.22.0:test (default-test) > [DEBUG] Style: Regular > [DEBUG] Configuration: <?xml version="1.0" encoding="UTF-8"?> > <configuration> > > > <additionalClasspathElements>${maven.test.additionalClasspath}</additionalCl > asspathElements> > <argLine>-Xmx384m</argLine> > <basedir default-value="${basedir}"/> > <childDelegation > default-value="false">${childDelegation}</childDelegation> > <classesDirectory default-value="${project.build.outputDirectory}"/> > > > <classpathDependencyExcludes>${maven.test.dependency.excludes}</classpathDep > endencyExcludes> > <debugForkedProcess>${maven.surefire.debug}</debugForkedProcess> > <dependenciesToScan>${dependenciesToScan}</dependenciesToScan> > <disableXmlReport > default-value="false">${disableXmlReport}</disableXmlReport> > <enableAssertions > default-value="true">${enableAssertions}</enableAssertions> > <encoding > > default-value="${project.reporting.outputEncoding}">${surefire.encoding}</en > coding> > <environmentVariables> > <JENKINS_MAVEN_AGENT_DISABLED>true</JENKINS_MAVEN_AGENT_DISABLED> > </environmentVariables> > <excludedGroups>${excludedGroups}</excludedGroups> > <excludesFile>${surefire.excludesFile}</excludesFile> > > > <failIfNoSpecifiedTests>${surefire.failIfNoSpecifiedTests}</failIfNoSpecifie > dTests> > <failIfNoTests>${failIfNoTests}</failIfNoTests> > <forkCount default-value="1">${forkCount}</forkCount> > <forkMode default-value="once">${forkMode}</forkMode> > <forkedProcessExitTimeoutInSeconds > > default-value="30">${surefire.exitTimeout}</forkedProcessExitTimeoutInSecond > s> > > > <forkedProcessTimeoutInSeconds>${surefire.timeout}</forkedProcessTimeoutInSe > conds> > <groups>${groups}</groups> > <includesFile>${surefire.includesFile}</includesFile> > <junitArtifactName > default-value="junit:junit">${junitArtifactName}</junitArtifactName> > <junitPlatformArtifactName > > default-value="org.junit.platform:junit-platform-engine">${junitPlatformArti > factName}</junitPlatformArtifactName> > <jvm>${jvm}</jvm> > <localRepository default-value="${localRepository}"/> > <objectFactory>${objectFactory}</objectFactory> > <parallel>${parallel}</parallel> > <parallelMavenExecution default-value="${session.parallel}"/> > <parallelOptimized > default-value="true">${parallelOptimized}</parallelOptimized> > > > <parallelTestsTimeoutForcedInSeconds>${surefire.parallel.forcedTimeout}</par > allelTestsTimeoutForcedInSeconds> > > > <parallelTestsTimeoutInSeconds>${surefire.parallel.timeout}</parallelTestsTi > meoutInSeconds> > <perCoreThreadCount > default-value="true">${perCoreThreadCount}</perCoreThreadCount> > <pluginArtifactMap>${plugin.artifactMap}</pluginArtifactMap> > <pluginDescriptor default-value="${plugin}"/> > <printSummary > default-value="true">${surefire.printSummary}</printSummary> > <projectArtifactMap>${project.artifactMap}</projectArtifactMap> > <projectBuildDirectory default-value="${project.build.directory}"/> > <redirectTestOutputToFile > > default-value="false">${maven.test.redirectTestOutputToFile}</redirectTestOu > tputToFile> > <remoteRepositories > default-value="${project.pluginArtifactRepositories}"/> > <reportFormat > default-value="brief">${surefire.reportFormat}</reportFormat> > <reportNameSuffix > default-value="">${surefire.reportNameSuffix}</reportNameSuffix> > <reportsDirectory > default-value="${project.build.directory}/surefire-reports"/> > <rerunFailingTestsCount > > default-value="0">${surefire.rerunFailingTestsCount}</rerunFailingTestsCount > > > <reuseForks default-value="true">${reuseForks}</reuseForks> > <runOrder default-value="filesystem">${surefire.runOrder}</runOrder> > <shutdown default-value="testset">${surefire.shutdown}</shutdown> > <skip default-value="false">${maven.test.skip}</skip> > <skipAfterFailureCount > default-value="0">${surefire.skipAfterFailureCount}</skipAfterFailureCount> > <skipExec>${maven.test.skip.exec}</skipExec> > <skipTests default-value="false">${skipTests}</skipTests> > <suiteXmlFiles>${surefire.suiteXmlFiles}</suiteXmlFiles> > <systemPropertyVariables> > <maven.home>C:\Program Files\apache-maven-3.6.1</maven.home> > </systemPropertyVariables> > <tempDir default-value="surefire">${tempDir}</tempDir> > <test>${test}</test> > <testClassesDirectory > default-value="${project.build.testOutputDirectory}"/> > <testFailureIgnore > default-value="false">${maven.test.failure.ignore}</testFailureIgnore> > <testNGArtifactName > > default-value="org.testng:testng">${testNGArtifactName}</testNGArtifactName> > <testSourceDirectory > default-value="${project.build.testSourceDirectory}"/> > <threadCount>${threadCount}</threadCount> > <threadCountClasses > default-value="0">${threadCountClasses}</threadCountClasses> > <threadCountMethods > default-value="0">${threadCountMethods}</threadCountMethods> > <threadCountSuites > default-value="0">${threadCountSuites}</threadCountSuites> > <trimStackTrace default-value="true">${trimStackTrace}</trimStackTrace> > <useFile default-value="true">${surefire.useFile}</useFile> > <useManifestOnlyJar > default-value="true">${surefire.useManifestOnlyJar}</useManifestOnlyJar> > <useSystemClassLoader > > default-value="true">${surefire.useSystemClassLoader}</useSystemClassLoader> > <useUnlimitedThreads > default-value="false">${useUnlimitedThreads}</useUnlimitedThreads> > <workingDirectory>${basedir}</workingDirectory> > <project default-value="${project}"/> > <session default-value="${session}"/> > </configuration> > ... > [ERROR] > > testProofClaim(org.apache.maven.plugins.dependency.fromConfiguration.TestCop > yMojo) Time elapsed: 0.146 s <<< ERROR! > org.junit.AssumptionViolatedException: always skip > at > > org.apache.maven.plugins.dependency.fromConfiguration.TestCopyMojo.testProof > Claim(TestCopyMojo.java:76) > ... > > IIUC, Maven debug doesn't contain any really specific information besides > what I aready assumed: Surefire treats > org.junit.AssumptionViolatedException > as an error but not as a request to skip the test. > > This can be reproduced with this really simple test case: > > https://github.com/mkarg/maven-dependency-plugin/commit/af257d7987fc41ac4377 > 4d2dca60b201979d11a2 > <https://github.com/mkarg/maven-dependency-plugin/commit/af257d7987fc41ac43774d2dca60b201979d11a2> > - as you can see, I just ask JUnit 4.13 to fail the > assumptions, and JUnit CORRECTLY throws AssumptionViolatedException - but > Maven seems to not specifically deal with it. > > -Markus > > > -----Ursprüngliche Nachricht----- > Von: Martin Gainty [mailto:mgai...@hotmail.com] > Gesendet: Sonntag, 16. August 2020 14:24 > An: Maven Developers List > Betreff: Re: Assumption fail treated as unexcepted exception?! > > MG>below > > ________________________________ > From: Markus KARG <mar...@headcrashing.eu> > Sent: Sunday, August 16, 2020 7:40 AM > To: dev@maven.apache.org <dev@maven.apache.org> > Subject: Assumption fail treated as unexcepted exception?! > > Guys, > > > > I'm stuck with working on a new feature due to this, so I hope you can help > me quickly: > > > > JUnit knows assumptions, assertions and exceptions. Failing assertions will > FAIL tests (hence will fail maven builds). Failing assumptions will SKIP > tests (hence will pass maven builds). Exceptions will ERROR tests (hence > will break maven builds). Unfortunately today I noticed that assumptions > actually ERROR test (hence fail builds) in Maven! > > > > (I simply added another test to maven dependency plugin which contains an > always-failing assumption to proof the claim) > > > > [INFO] Results: > > [INFO] > > [ERROR] Errors: > > [ERROR] TestCopyMojo.proofClaim:274 ┐ AssumptionViolated always skip > > MG>get maven environment and debug information > MG>mvn -e -X > MG>also I know Junit 5.4.2 needs Hamcrest to be on classpath > MG>can you tell us which version of Junit you are using? > > -Markus Karg > ~gruss~ > ~martin~ > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
