Thank you for explaining. Yes, turns out a dependency is missing in my reproducer and looks like M5 correctly detects that error; that's great news.
Interestingly, if I correct that by adding the dep, M4 correctly works with this reproducer too, so it's not a reproducer of the intended error! That is not the problem in the customer app though - both 2.22.2 and IDEA detect the NoSuchBeanDefinitionException in it. Thanks again for the help; looking forward to M5 release. On Sun, Jun 14, 2020 at 3:55 AM Tibor Digana <tibordig...@apache.org> wrote: > Now we are able to handle the errors on the class level. > Before in 2.22.x and in M4 we handled the test errors only and ignored the > class level errors. > > I would trust NoClassDefFoundError because ControllerAdvice really could > not be found in IntelliJ IDEA. > So the IDE is telling me the same thing, and this is perhaps the root cause > for NoSuchBeanDefinitionException. You know your application better than > me. > > Even if i use the IntelliJ IDEA and run the tests, i get the same error on > the console: > > Test ignored. > > *java.lang.NoClassDefFoundError: > org/springframework/web/bind/annotation/ControllerAdvice* > at > > org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTypeExcludeFilter.<clinit>(WebMvcTypeExcludeFilter.java:59) > ... > Suppressed: java.lang.NoClassDefFoundError: Could not initialize class > > org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTypeExcludeFilter > at > > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at > > java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) > ... > Caused by: java.lang.ClassNotFoundException: > org.springframework.web.bind.annotation.ControllerAdvice > at > > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:604) > at > > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > ... 61 more > > > > > > > > > On Sat, Jun 13, 2020 at 10:51 PM Jeff Jensen <jjen...@apache.org> wrote: > > > Great news that it fails! Thank you Tibor. > > > > I note it emits a different exception than with 2.22.2 or running in IDE. > > 2.22.2 emits an IllegalStateException with root > > NoSuchBeanDefinitionException but per below M5 emits > NoClassDefFoundError. > > Is this ok because you changed something locally or is this a concern? > > > > > > On Sat, Jun 13, 2020 at 12:47 PM Tibor Digana <tibordig...@apache.org> > > wrote: > > > > > We have changed the resolution of junit5 engines 2 times. > > > Now I have got the result with 3.0.0-M5. I think it is what you expect: > > > > > > [INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ > > > surefire-no-build-fail --- > > > [INFO] > > > [INFO] ------------------------------------------------------- > > > [INFO] T E S T S > > > [INFO] ------------------------------------------------------- > > > [INFO] Running com.jeffjensen.surefire.ApplicationTest > > > 19:43:18.789 [main] DEBUG > > org.springframework.test.context.BootstrapUtils - > > > Instantiating CacheAwareContextLoaderDelegate from class > > > > > > > > > [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate] > > > 19:43:18.821 [main] DEBUG > > org.springframework.test.context.BootstrapUtils - > > > Instantiating BootstrapContext using constructor [public > > > > > > > > > org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)] > > > 19:43:18.910 [main] DEBUG > > org.springframework.test.context.BootstrapUtils - > > > Instantiating TestContextBootstrapper for test class > > > [com.jeffjensen.surefire.ApplicationTest] from class > > > > > > > > > [org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper] > > > 19:43:18.944 [main] INFO > > > > > > > > > org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper > > > - Neither @ContextConfiguration nor @ContextHierarchy found for test > > class > > > [com.jeffjensen.surefire.ApplicationTest], using > SpringBootContextLoader > > > 19:43:18.951 [main] DEBUG > > > org.springframework.test.context.support.AbstractContextLoader - Did > not > > > detect default resource location for test class > > > [com.jeffjensen.surefire.ApplicationTest]: class path resource > > > [com/jeffjensen/surefire/ApplicationTest-context.xml] does not exist > > > 19:43:18.952 [main] DEBUG > > > org.springframework.test.context.support.AbstractContextLoader - Did > not > > > detect default resource location for test class > > > [com.jeffjensen.surefire.ApplicationTest]: class path resource > > > [com/jeffjensen/surefire/ApplicationTestContext.groovy] does not exist > > > 19:43:18.952 [main] INFO > > > org.springframework.test.context.support.AbstractContextLoader - Could > > not > > > detect default resource locations for test class > > > [com.jeffjensen.surefire.ApplicationTest]: no resource found for > suffixes > > > {-context.xml, Context.groovy}. > > > 19:43:18.953 [main] INFO > > > > > > org.springframework.test.context.support.AnnotationConfigContextLoaderUtils > > > - Could not detect default configuration classes for test class > > > [com.jeffjensen.surefire.ApplicationTest]: ApplicationTest does not > > declare > > > any static, non-private, non-final, nested classes annotated with > > > @Configuration. > > > 19:43:19.024 [main] DEBUG > > org.springframework.test.context.BootstrapUtils - > > > Instantiating CacheAwareContextLoaderDelegate from class > > > > > > > > > [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate] > > > 19:43:19.024 [main] DEBUG > > org.springframework.test.context.BootstrapUtils - > > > Instantiating BootstrapContext using constructor [public > > > > > > > > > org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)] > > > 19:43:19.025 [main] DEBUG > > org.springframework.test.context.BootstrapUtils - > > > Instantiating TestContextBootstrapper for test class > > > [com.jeffjensen.surefire.ApplicationTest] from class > > > > > > > > > [org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper] > > > 19:43:19.026 [main] INFO > > > > > > > > > org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper > > > - Neither @ContextConfiguration nor @ContextHierarchy found for test > > class > > > [com.jeffjensen.surefire.ApplicationTest], using > SpringBootContextLoader > > > 19:43:19.027 [main] DEBUG > > > org.springframework.test.context.support.AbstractContextLoader - Did > not > > > detect default resource location for test class > > > [com.jeffjensen.surefire.ApplicationTest]: class path resource > > > [com/jeffjensen/surefire/ApplicationTest-context.xml] does not exist > > > 19:43:19.028 [main] DEBUG > > > org.springframework.test.context.support.AbstractContextLoader - Did > not > > > detect default resource location for test class > > > [com.jeffjensen.surefire.ApplicationTest]: class path resource > > > [com/jeffjensen/surefire/ApplicationTestContext.groovy] does not exist > > > 19:43:19.029 [main] INFO > > > org.springframework.test.context.support.AbstractContextLoader - Could > > not > > > detect default resource locations for test class > > > [com.jeffjensen.surefire.ApplicationTest]: no resource found for > suffixes > > > {-context.xml, Context.groovy}. > > > 19:43:19.029 [main] INFO > > > > > > org.springframework.test.context.support.AnnotationConfigContextLoaderUtils > > > - Could not detect default configuration classes for test class > > > [com.jeffjensen.surefire.ApplicationTest]: ApplicationTest does not > > declare > > > any static, non-private, non-final, nested classes annotated with > > > @Configuration. > > > 19:43:19.035 [main] DEBUG > > org.springframework.test.context.BootstrapUtils - > > > Instantiating CacheAwareContextLoaderDelegate from class > > > > > > > > > [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate] > > > 19:43:19.035 [main] DEBUG > > org.springframework.test.context.BootstrapUtils - > > > Instantiating BootstrapContext using constructor [public > > > > > > > > > org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)] > > > 19:43:19.037 [main] DEBUG > > org.springframework.test.context.BootstrapUtils - > > > Instantiating TestContextBootstrapper for test class > > > [com.jeffjensen.surefire.ApplicationTest] from class > > > > > > > > > [org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper] > > > 19:43:19.038 [main] INFO > > > > > > > > > org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTestContextBootstrapper > > > - Neither @ContextConfiguration nor @ContextHierarchy found for test > > class > > > [com.jeffjensen.surefire.ApplicationTest], using > SpringBootContextLoader > > > 19:43:19.040 [main] DEBUG > > > org.springframework.test.context.support.AbstractContextLoader - Did > not > > > detect default resource location for test class > > > [com.jeffjensen.surefire.ApplicationTest]: class path resource > > > [com/jeffjensen/surefire/ApplicationTest-context.xml] does not exist > > > 19:43:19.040 [main] DEBUG > > > org.springframework.test.context.support.AbstractContextLoader - Did > not > > > detect default resource location for test class > > > [com.jeffjensen.surefire.ApplicationTest]: class path resource > > > [com/jeffjensen/surefire/ApplicationTestContext.groovy] does not exist > > > 19:43:19.040 [main] INFO > > > org.springframework.test.context.support.AbstractContextLoader - Could > > not > > > detect default resource locations for test class > > > [com.jeffjensen.surefire.ApplicationTest]: no resource found for > suffixes > > > {-context.xml, Context.groovy}. > > > 19:43:19.041 [main] INFO > > > > > > org.springframework.test.context.support.AnnotationConfigContextLoaderUtils > > > - Could not detect default configuration classes for test class > > > [com.jeffjensen.surefire.ApplicationTest]: ApplicationTest does not > > declare > > > any static, non-private, non-final, nested classes annotated with > > > @Configuration. > > > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > > > 0.417 s <<< FAILURE! - in com.jeffjensen.surefire.ApplicationTest > > > [ERROR] com.jeffjensen.surefire.ApplicationTest Time elapsed: 0.417 s > > <<< > > > ERROR! > > > java.lang.NoClassDefFoundError: > > > org/springframework/web/bind/annotation/ControllerAdvice > > > Caused by: java.lang.ClassNotFoundException: > > > org.springframework.web.bind.annotation.ControllerAdvice > > > > > > [INFO] > > > [INFO] Results: > > > [INFO] > > > [ERROR] Errors: > > > [ERROR] ApplicationTest » NoClassDefFound > > > org/springframework/web/bind/annotation/Cont... > > > [INFO] > > > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 > > > [INFO] > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] BUILD FAILURE > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Total time: 6.723 s > > > [INFO] Finished at: 2020-06-13T19:43:19+02:00 > > > [INFO] > > > > ------------------------------------------------------------------------ > > > > > > > > > > > > On Sat, Jun 13, 2020 at 6:56 PM Jeff Jensen <jjen...@apache.org> > wrote: > > > > > > > This reproduces the issue: > > > > https://github.com/jeffjensen/maven-surefire-3.0.0-m4-no-build-fail > > > > > > > > Please let me know if I can help further. > > > > > > > > > > > > On Sat, Jun 13, 2020 at 9:09 AM Tibor Digaňa > > > > <tibor.dig...@googlemail.com.invalid> wrote: > > > > > > > > > yes, the reproducible project would be great to have. > > > > > Thx > > > > > > > > > > On Sat, Jun 13, 2020 at 4:04 PM Jeff Jensen <jjen...@apache.org> > > > wrote: > > > > > > > > > > > Thank you Enrico and Tibor for verifying it is a probable new > > issue. > > > > > > > > > > > > I cannot upload this project (customer product/private repo) but > > will > > > > > make > > > > > > a project that reproduces it and reply again. > > > > > > > > > > > > > > > > > > On Sat, Jun 13, 2020 at 8:24 AM Enrico Olivelli < > > eolive...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > Il Sab 13 Giu 2020, 15:07 Jeff Jensen <jjen...@apache.org> ha > > > > scritto: > > > > > > > > > > > > > > > Hi Enrico, > > > > > > > > Thanks for the reply. > > > > > > > > > > > > > > > > Surefire 2.22.2 correctly fails the build. This is with > Java 8 > > > and > > > > > > JUnit > > > > > > > > 5. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sounds interesting. > > > > > > > > > > > > > > It looks like a big bug to me and it is worth a JIRA > > > > > > > > > > > > > > It would help a lot if you could attach a simple reproducer for > > the > > > > > > > problem. > > > > > > > If you know surefire codebase it would also help a lot to > create > > > the > > > > > > > reproducer as an integration test > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Jun 13, 2020 at 12:41 AM Enrico Olivelli < > > > > > eolive...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Jeff > > > > > > > > > > > > > > > > > > Il Sab 13 Giu 2020, 00:15 Jeff Jensen < > > > > > > > jeffjen...@upstairstechnology.com > > > > > > > > > > > > > > > > > > ha scritto: > > > > > > > > > > > > > > > > > > > I looked for this issue in JIRA but haven't found > anything > > > yet. > > > > > > > Anyone > > > > > > > > > > know if this has been reported and/or fixed? > > > > > > > > > > > > > > > > > > > > Our scenario is a failure occurs at startup of tests but > > > > Surefire > > > > > > > > doesn't > > > > > > > > > > fail the build. Specifically, Spring controller tests > > aren't > > > > > > failing > > > > > > > > > when > > > > > > > > > > there is a Spring configuration problem. > > > > > > > > > > > > > > > > > > > > Spring top-level thrown exception is: > > > > > > > > > > java.lang.IllegalStateException: Failed to load > > > > > > ApplicationContext > > > > > > > > > > > > > > > > > > > > but Surefire doesn't report fail. Surefire output > results > > > is: > > > > > > > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, > > > > > > > > > > > > > > > > > > > > Running the tests in an IDE correctly fail for the setup > > > > problem. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Which IDE? > > > > > > > > > > > > > > > > > > Did you try other versions of surefire? Like the latest > from > > > 2.x > > > > > > > release > > > > > > > > > line? > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Cheers > > > > > Tibor > > > > > > > > > > > > > > >