Great news that it fails! Thank you Tibor.
I note it emits a different exception than with 2.22.2 or running in IDE.
2.22.2 emits an IllegalStateException with root
NoSuchBeanDefinitionException but per below M5 emits NoClassDefFoundError.
Is this ok because you changed something locally or is this a concern?
On Sat, Jun 13, 2020 at 12:47 PM Tibor Digana <tibordig...@apache.org>
wrote:
> We have changed the resolution of junit5 engines 2 times.
> Now I have got the result with 3.0.0-M5. I think it is what you expect:
>
> [INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @
> surefire-no-build-fail ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running com.jeffjensen.surefire.ApplicationTest
> 19:43:18.789 [main] DEBUG org.springframework.test.context.BootstrapUtils -
> Instantiating CacheAwareContextLoaderDelegate from class
>
> [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate]
> 19:43:18.821 [main] DEBUG org.springframework.test.context.BootstrapUtils -
> Instantiating BootstrapContext using constructor [public
>
> org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)]
> 19:43:18.910 [main] DEBUG org.springframework.test.context.BootstrapUtils -
> Instantiating TestContextBootstrapper for test class
> [com.jeffjensen.surefire.ApplicationTest] from class
>
> [org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper]
> 19:43:18.944 [main] INFO
>
> org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper
> - Neither @ContextConfiguration nor @ContextHierarchy found for test class
> [com.jeffjensen.surefire.ApplicationTest], using SpringBootContextLoader
> 19:43:18.951 [main] DEBUG
> org.springframework.test.context.support.AbstractContextLoader - Did not
> detect default resource location for test class
> [com.jeffjensen.surefire.ApplicationTest]: class path resource
> [com/jeffjensen/surefire/ApplicationTest-context.xml] does not exist
> 19:43:18.952 [main] DEBUG
> org.springframework.test.context.support.AbstractContextLoader - Did not
> detect default resource location for test class
> [com.jeffjensen.surefire.ApplicationTest]: class path resource
> [com/jeffjensen/surefire/ApplicationTestContext.groovy] does not exist
> 19:43:18.952 [main] INFO
> org.springframework.test.context.support.AbstractContextLoader - Could not
> detect default resource locations for test class
> [com.jeffjensen.surefire.ApplicationTest]: no resource found for suffixes
> {-context.xml, Context.groovy}.
> 19:43:18.953 [main] INFO
> org.springframework.test.context.support.AnnotationConfigContextLoaderUtils
> - Could not detect default configuration classes for test class
> [com.jeffjensen.surefire.ApplicationTest]: ApplicationTest does not declare
> any static, non-private, non-final, nested classes annotated with
> @Configuration.
> 19:43:19.024 [main] DEBUG org.springframework.test.context.BootstrapUtils -
> Instantiating CacheAwareContextLoaderDelegate from class
>
> [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate]
> 19:43:19.024 [main] DEBUG org.springframework.test.context.BootstrapUtils -
> Instantiating BootstrapContext using constructor [public
>
> org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)]
> 19:43:19.025 [main] DEBUG org.springframework.test.context.BootstrapUtils -
> Instantiating TestContextBootstrapper for test class
> [com.jeffjensen.surefire.ApplicationTest] from class
>
> [org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper]
> 19:43:19.026 [main] INFO
>
> org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper
> - Neither @ContextConfiguration nor @ContextHierarchy found for test class
> [com.jeffjensen.surefire.ApplicationTest], using SpringBootContextLoader
> 19:43:19.027 [main] DEBUG
> org.springframework.test.context.support.AbstractContextLoader - Did not
> detect default resource location for test class
> [com.jeffjensen.surefire.ApplicationTest]: class path resource
> [com/jeffjensen/surefire/ApplicationTest-context.xml] does not exist
> 19:43:19.028 [main] DEBUG
> org.springframework.test.context.support.AbstractContextLoader - Did not
> detect default resource location for test class
> [com.jeffjensen.surefire.ApplicationTest]: class path resource
> [com/jeffjensen/surefire/ApplicationTestContext.groovy] does not exist
> 19:43:19.029 [main] INFO
> org.springframework.test.context.support.AbstractContextLoader - Could not
> detect default resource locations for test class
> [com.jeffjensen.surefire.ApplicationTest]: no resource found for suffixes
> {-context.xml, Context.groovy}.
> 19:43:19.029 [main] INFO
> org.springframework.test.context.support.AnnotationConfigContextLoaderUtils
> - Could not detect default configuration classes for test class
> [com.jeffjensen.surefire.ApplicationTest]: ApplicationTest does not declare
> any static, non-private, non-final, nested classes annotated with
> @Configuration.
> 19:43:19.035 [main] DEBUG org.springframework.test.context.BootstrapUtils -
> Instantiating CacheAwareContextLoaderDelegate from class
>
> [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate]
> 19:43:19.035 [main] DEBUG org.springframework.test.context.BootstrapUtils -
> Instantiating BootstrapContext using constructor [public
>
> org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)]
> 19:43:19.037 [main] DEBUG org.springframework.test.context.BootstrapUtils -
> Instantiating TestContextBootstrapper for test class
> [com.jeffjensen.surefire.ApplicationTest] from class
>
> [org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper]
> 19:43:19.038 [main] INFO
>
> org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper
> - Neither @ContextConfiguration nor @ContextHierarchy found for test class
> [com.jeffjensen.surefire.ApplicationTest], using SpringBootContextLoader
> 19:43:19.040 [main] DEBUG
> org.springframework.test.context.support.AbstractContextLoader - Did not
> detect default resource location for test class
> [com.jeffjensen.surefire.ApplicationTest]: class path resource
> [com/jeffjensen/surefire/ApplicationTest-context.xml] does not exist
> 19:43:19.040 [main] DEBUG
> org.springframework.test.context.support.AbstractContextLoader - Did not
> detect default resource location for test class
> [com.jeffjensen.surefire.ApplicationTest]: class path resource
> [com/jeffjensen/surefire/ApplicationTestContext.groovy] does not exist
> 19:43:19.040 [main] INFO
> org.springframework.test.context.support.AbstractContextLoader - Could not
> detect default resource locations for test class
> [com.jeffjensen.surefire.ApplicationTest]: no resource found for suffixes
> {-context.xml, Context.groovy}.
> 19:43:19.041 [main] INFO
> org.springframework.test.context.support.AnnotationConfigContextLoaderUtils
> - Could not detect default configuration classes for test class
> [com.jeffjensen.surefire.ApplicationTest]: ApplicationTest does not declare
> any static, non-private, non-final, nested classes annotated with
> @Configuration.
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 0.417 s <<< FAILURE! - in com.jeffjensen.surefire.ApplicationTest
> [ERROR] com.jeffjensen.surefire.ApplicationTest Time elapsed: 0.417 s <<<
> ERROR!
> java.lang.NoClassDefFoundError:
> org/springframework/web/bind/annotation/ControllerAdvice
> Caused by: java.lang.ClassNotFoundException:
> org.springframework.web.bind.annotation.ControllerAdvice
>
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR] ApplicationTest » NoClassDefFound
> org/springframework/web/bind/annotation/Cont...
> [INFO]
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> [INFO]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 6.723 s
> [INFO] Finished at: 2020-06-13T19:43:19+02:00
> [INFO]
> ------------------------------------------------------------------------
>
>
>
> On Sat, Jun 13, 2020 at 6:56 PM Jeff Jensen <jjen...@apache.org> wrote:
>
> > This reproduces the issue:
> > https://github.com/jeffjensen/maven-surefire-3.0.0-m4-no-build-fail
> >
> > Please let me know if I can help further.
> >
> >
> > On Sat, Jun 13, 2020 at 9:09 AM Tibor Digaňa
> > <tibor.dig...@googlemail.com.invalid> wrote:
> >
> > > yes, the reproducible project would be great to have.
> > > Thx
> > >
> > > On Sat, Jun 13, 2020 at 4:04 PM Jeff Jensen <jjen...@apache.org>
> wrote:
> > >
> > > > Thank you Enrico and Tibor for verifying it is a probable new issue.
> > > >
> > > > I cannot upload this project (customer product/private repo) but will
> > > make
> > > > a project that reproduces it and reply again.
> > > >
> > > >
> > > > On Sat, Jun 13, 2020 at 8:24 AM Enrico Olivelli <eolive...@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Il Sab 13 Giu 2020, 15:07 Jeff Jensen <jjen...@apache.org> ha
> > scritto:
> > > > >
> > > > > > Hi Enrico,
> > > > > > Thanks for the reply.
> > > > > >
> > > > > > Surefire 2.22.2 correctly fails the build. This is with Java 8
> and
> > > > JUnit
> > > > > > 5.
> > > > > >
> > > > >
> > > > >
> > > > > This sounds interesting.
> > > > >
> > > > > It looks like a big bug to me and it is worth a JIRA
> > > > >
> > > > > It would help a lot if you could attach a simple reproducer for the
> > > > > problem.
> > > > > If you know surefire codebase it would also help a lot to create
> the
> > > > > reproducer as an integration test
> > > > >
> > > > > Thanks
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > > >
> > > > > > On Sat, Jun 13, 2020 at 12:41 AM Enrico Olivelli <
> > > eolive...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Jeff
> > > > > > >
> > > > > > > Il Sab 13 Giu 2020, 00:15 Jeff Jensen <
> > > > > jeffjen...@upstairstechnology.com
> > > > > > >
> > > > > > > ha scritto:
> > > > > > >
> > > > > > > > I looked for this issue in JIRA but haven't found anything
> yet.
> > > > > Anyone
> > > > > > > > know if this has been reported and/or fixed?
> > > > > > > >
> > > > > > > > Our scenario is a failure occurs at startup of tests but
> > Surefire
> > > > > > doesn't
> > > > > > > > fail the build. Specifically, Spring controller tests aren't
> > > > failing
> > > > > > > when
> > > > > > > > there is a Spring configuration problem.
> > > > > > > >
> > > > > > > > Spring top-level thrown exception is:
> > > > > > > > java.lang.IllegalStateException: Failed to load
> > > > ApplicationContext
> > > > > > > >
> > > > > > > > but Surefire doesn't report fail. Surefire output results
> is:
> > > > > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0,
> > > > > > > >
> > > > > > > > Running the tests in an IDE correctly fail for the setup
> > problem.
> > > > > > > >
> > > > > > >
> > > > > > > Which IDE?
> > > > > > >
> > > > > > > Did you try other versions of surefire? Like the latest from
> 2.x
> > > > > release
> > > > > > > line?
> > > > > > >
> > > > > > > Regards
> > > > > > > Enrico
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Cheers
> > > Tibor
> > >
> >
>
