On Tue, 22 Mar 2022 at 14:40, Tibor Digana <tibordig...@apache.org> wrote:
> Sorry for late reply. > > I have created a demo project > https://github.com/Tibor17/maven.test.failure.ignore/ which simulates > OOM. > > According to the definition of the parameter "maven.failure.test.ignore" > the failures (also errors) should be ignored during testing. > The word "during" is important because the time when it was a failure > before JVM startup it was not a test failure - it is MOJO failure. The > author who wrote this config param knew this difference. > This perfectly fits JVM illegal arguments like -Xxxx. > But I have investigated OOM for the same reason with one JVM. My suspicion > was not confirmed, see the next... > If there are two JVMs, we are breaking the JVM during testing. It is a very > unlikely situation that the JVM argument would not fail in JVM1 but it > would fail in JVM2 later. It may happen only if the user > uses @surefire.forkCount@ interpolation but it is again very unlikely and > the users use it in current working directory configuration. > > I understand Olivier's PR but I have one objection to the code branching in > the class *SurefireHelper*. It is based on 3 levels but I have to disagree > due to there still should be 2 levels as before: > 1. ignored failures (my change is only here - yellow - only added throwing > exception) > 2. processes failures > > and so, I have created a PR > https://github.com/apache/maven-surefire/pull/496 > Pls have a look into it, thx. > This diff looks very simple: > > if ( reportParameters.isTestFailureIgnore() ) > { > String errorMessage = createErrorMessage( reportParameters, > result, firstForkException ); > > if ( firstForkException instanceof SurefireBooterForkException ) > { > throw new MojoExecutionException( errorMessage, firstForkException > ); > > } > > log.error( errorMessage ); > } > else > { > throwException( reportParameters, result, firstForkException ); > } > > > > *$ mvn test* > > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 2.445 s <<< FAILURE! - in org.apache.maven.OOMTest > [ERROR] org.apache.maven.OOMTest.test Time elapsed: 2.063 s <<< ERROR! > java.lang.OutOfMemoryError: Java heap space > at org.apache.maven.OOMTest.test(OOMTest.java:15) > [INFO] > [INFO] Results: > [INFO] > [ERROR] Errors: > [ERROR] OOMTest.test:15 » OutOfMemory Java heap space > [INFO] > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 > [INFO] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > > *$ mvn test -Dmaven.test.failure.ignore* > > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 2.472 s <<< FAILURE! - in org.apache.maven.OOMTest > [ERROR] org.apache.maven.OOMTest.test Time elapsed: 2.262 s <<< ERROR! > java.lang.OutOfMemoryError: Java heap space > at org.apache.maven.OOMTest.test(OOMTest.java:15) > [INFO] > [INFO] Results: > [INFO] > [ERROR] Errors: > [ERROR] OOMTest.test:15 » OutOfMemory Java heap space > [INFO] > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 > [INFO] > [ERROR] > > Please refer to > C:\vcs\github\maven.test.failure.ignore\target\surefire-reports for the > individual test results. > Please refer to dump files (if any exist) [date].dump, > [date]-jvmRun[N].dump and [date].dumpstream. > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > Please use a project with multiple modules, not a single one where you can see the line with OOME. Any project with a huge number of modules will hide this and nobody will ever notice it. git clone -b first-module-fail https://github.com/apache/maven maven-core cd maven-core [INFO] --- maven-assembly-plugin:3.3.0:single (create-distro-packages) @ apache-maven --- [INFO] Reading assembly descriptor: src/main/assembly/bin.xml [INFO] Building zip: /Volumes/workspace/dev/sources/maven/maven-core/target/maven-core/apache-maven/target/apache-maven-3.8.6-SNAPSHOT-bin.zip [INFO] Building tar: /Volumes/workspace/dev/sources/maven/maven-core/target/maven-core/apache-maven/target/apache-maven-3.8.6-SNAPSHOT-bin.tar.gz [INFO] [INFO] --- maven-checkstyle-plugin:3.0.0:check (checkstyle-check) @ apache-maven --- [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Apache Maven 3.8.6-SNAPSHOT: [INFO] [INFO] Apache Maven ....................................... SUCCESS [ 3.220 s] [INFO] Maven Model ........................................ SUCCESS [ 4.734 s] [INFO] Maven Artifact ..................................... SUCCESS [ 3.142 s] [INFO] Maven Plugin API ................................... SUCCESS [ 1.763 s] [INFO] Maven Builder Support .............................. SUCCESS [ 1.288 s] [INFO] Maven Model Builder ................................ SUCCESS [ 5.129 s] [INFO] Maven Settings ..................................... SUCCESS [ 0.694 s] [INFO] Maven Settings Builder ............................. SUCCESS [ 1.730 s] [INFO] Maven Repository Metadata Model .................... SUCCESS [ 1.470 s] [INFO] Maven Artifact Resolver Provider ................... SUCCESS [ 4.255 s] [INFO] Maven Core ......................................... SUCCESS [ 19.075 s] [INFO] Maven SLF4J Simple Provider ........................ SUCCESS [ 0.972 s] [INFO] Maven Embedder ..................................... SUCCESS [ 4.293 s] [INFO] Maven Compat ....................................... SUCCESS [ 10.350 s] [INFO] Apache Maven Distribution .......................... SUCCESS [ 4.898 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:07 min [INFO] Finished at: 2022-03-22T20:19:24+10:00 [INFO] ------------------------------------------------------------------------ even CI is green https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven/job/first-module-fail/ so all looks good to merge a PR... but hey in fact a module has not run any tests... Do you really think it's right? (yes this problem exists for a very very long time but we can choose to fix problems even after a long period) Does your PR fix that? > > > > On Sat, Mar 19, 2022 at 12:17 PM Slawomir Jaranowski < > s.jaranow...@gmail.com> > wrote: > > > Hi, > > > > I start to feel that we first try to define technical details but we > don't > > know for what. > > Each business requirement should be resolved as simply as possible from a > > technical perspective. > > In other ways we will start to build complicated and many features which > > probably were not needed at all. > > > > Please first define business requirements with examples, next we can find > > technical ways to cover it. > > > > Example scenario: > > > > Businesses requirement: > > We want to collect history of test result in order to detect flaky > tests, > > analize tests execution time, etc > > > > Given: > > - build is done by CI system > > - there is system which collect reports for executed tests and build > > history of it > > - there are broken tests in build - we know them and accept (eg, on some > > os) > > > > When: > > - build finished with success on CI > > > > Then: > > - all tests result are reported > > - failed tests are reported > > > > > > > > > > sob., 19 mar 2022 o 06:27 Christoph Läubrich <m...@laeubi-soft.de> > > napisał(a): > > > > > Hi Tibor, > > > > > > just to make it clear, I don't talkin about log-levels here but how to > > > intepret/handle a failure/failing/crashing test. > > > > > > > > > Am 18.03.22 um 21:54 schrieb Tibor Digaňa: > > > > Christoph, just let me briefly explain the log level hierarchy. > > > > If you select WARN log level, then ERROR can be printed too. > > > > Similar with INFO, means that WARN and ERROR would be printed as > well. > > > > > > > > The real decision making in the plugin is a bit more complicated, see > > the > > > > pull request https://github.com/apache/maven-surefire/pull/478 > > > > After I have replied to this email, I would verify the behavior of > the > > > > plugin simulating OOM. > > > > > > > > > > > > T > > > > > > > > > > > > On Fri, Mar 18, 2022 at 5:42 AM Christoph Läubrich < > > m...@laeubi-soft.de> > > > > wrote: > > > > > > > >> No I think more about > > > >> > > > >> if (severity == "WARN") { > > > >> log.warn("There is something wrong"); > > > >> } else if (severity == "ERROR") { > > > >> throw new MojoFailureException("...") { > > > >> } else { > > > >> throw new MojoExecutionException("...") { > > > >> } > > > >> > > > >> That way the plugin can handle any error/failure/... in a way that > the > > > >> user can "downgrade" a certain problem if desired. > > > >> > > > >> > > > >> Am 18.03.22 um 00:06 schrieb Tibor Digana: > > > >>> @Christoph > > > >>> FATAL , WARN , ERROR > > > >>> They are log levels? > > > >>> The plugin does not control the log level after caught exception in > > > Maven > > > >>> core. The Maven Core does! > > > >>> I think it's time to make a demo. > > > >>> > > > >>> On Thu, Mar 17, 2022 at 6:21 AM Christoph Läubrich < > > > m...@laeubi-soft.de> > > > >>> wrote: > > > >>> > > > >>>> Hi Tibor, > > > >>>> > > > >>>> it shouldn't be to hard to guess another property like > > > >>>> > > > >>>> maven.test.jvmcrash=FATAL > > > >>>> maven.test.failure=WARN > > > >>>> maven.test.error=ERROR > > > >>>> > > > >>>> :-) > > > >>>> > > > >>>> Am 16.03.22 um 12:25 schrieb Tibor Digana: > > > >>>>> Hi Christoph, > > > >>>>> > > > >>>>> Such a granularity with error/failure might be also an additional > > > >>>>> requirement but still you miss the third option to JVM error > which > > is > > > >>>>> different from test error/failure - they don;t have the same > > meaning. > > > >>>>> > > > >>>>> T > > > >>>>> > > > >>>>> > > > >>>>> On Mon, Mar 14, 2022 at 10:57 AM Christoph Läubrich < > > > >> m...@laeubi-soft.de > > > >>>>> > > > >>>>> wrote: > > > >>>>> > > > >>>>>> Just wondering but maybe it would be better to configure the > > > severity > > > >>>>>> instead of a true/false, something like: > > > >>>>>> > > > >>>>>> maven.test.failure=WARN > > > >>>>>> maven.test.error=ERROR > > > >>>>>> > > > >>>>>> would only warn about failing tests but thrw exception if > starting > > > the > > > >>>>>> test fails? > > > >>>>>> > > > >>>>>> Am 14.03.22 um 10:52 schrieb Tibor Digana: > > > >>>>>>> Romain, it is not a bug. > > > >>>>>>> Don't consider this as a bug. It was a feature request for > change > > > by > > > >>>>>>> Olivier, and not a bug. > > > >>>>>>> I closed both issues years ago but not because of ignorance but > > > >> because > > > >>>>>> the > > > >>>>>>> appearance of the exceptional behavior is a wrong compromise > and > > > >> which > > > >>>>>> does > > > >>>>>>> not help anyone and even it makes the situation worse because > > > >> typically > > > >>>>>>> other group of users would fire a new Jira tickets. You would > not > > > be > > > >>>> able > > > >>>>>>> to satisfy two contradictory parties which have completely > > > different > > > >>>>>>> opinions, and so we use to introduce new params and this way we > > > >> satisfy > > > >>>>>>> both parties, they may combine the params as they wish, and > > > everybody > > > >>>>>> would > > > >>>>>>> be happy and nobody would claim. Many technical solutions might > > > >> exist, > > > >>>>>> e.g. > > > >>>>>>> param=boolean|string or new param=boolean, whatever. > > > >>>>>>> > > > >>>>>>> The truth is that this problem with (java --add-reads ...) > > happened > > > >> in > > > >>>>>> our > > > >>>>>>> ASF environment on Java 8 which in good configuration would not > > > >> happen > > > >>>>>> and > > > >>>>>>> should not. > > > >>>>>>> It is not right way that we abuse "maven.test.failure.ignore" > > which > > > >>>> would > > > >>>>>>> tell us "Hey, you have illegal configuration in your build > system > > > and > > > >>>> it > > > >>>>>>> would not work by JDK design", it is not the goal of the plugin > > to > > > >> tell > > > >>>>>> you > > > >>>>>>> that you have configured the build wrong because it won't and > the > > > >> same > > > >>>>>>> configuration of the plugin (not the build) says "ignore the > > > error". > > > >>>>>>> Yesterday I discussed this problem with Herve and we > > independently > > > >>>>>> observed > > > >>>>>>> equal opinions and that's not everything because we understood > > that > > > >> the > > > >>>>>>> purpose of the config parameter is to not throw MOJO exception > > > which > > > >> is > > > >>>>>>> right, but the exceptional behavior should have an exact new > > param > > > >> for > > > >>>>>> the > > > >>>>>>> exact use case. > > > >>>>>>> SPI for this parameter is too much because no user would > > implement > > > it > > > >>>>>> for a > > > >>>>>>> trivial parameter;; the SPI is used to be implemented by > > frameworks > > > >> or > > > >>>>>>> systems or application servers but this is not our case. > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> On Mon, Mar 14, 2022 at 9:11 AM Romain Manni-Bucau < > > > >>>>>> rmannibu...@gmail.com> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> +1 > > > >>>>>>>> if it is to investigate a CI issue, it is generally easy to > add > > > >> debug > > > >>>>>>>> insights (by code or agent) so a SPI sounds like the sanest > for > > > the > > > >>>>>> plugin > > > >>>>>>>> to me. > > > >>>>>>>> > > > >>>>>>>> Romain Manni-Bucau > > > >>>>>>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog > > > >>>>>>>> <https://rmannibucau.metawerx.net/> | Old Blog > > > >>>>>>>> <http://rmannibucau.wordpress.com> | Github < > > > >>>>>>>> https://github.com/rmannibucau> | > > > >>>>>>>> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book > > > >>>>>>>> < > > > >>>>>>>> > > > >>>>>> > > > >>>> > > > >> > > > > > > https://www.packtpub.com/application-development/java-ee-8-high-performance > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> Le lun. 14 mars 2022 à 09:08, Guillaume Nodet < > > gno...@apache.org> > > > a > > > >>>>>> écrit > > > >>>>>>>> : > > > >>>>>>>> > > > >>>>>>>>> If that's not currently possible, maybe a SPI should be > > provided > > > so > > > >>>>>> that > > > >>>>>>>>> people can use plug in extensions to analyze the test result > > and > > > >>>>>> override > > > >>>>>>>>> it if necessary (transforming an error into a warning, > storing > > > >>>> results > > > >>>>>>>> in a > > > >>>>>>>>> way which is easier to use by other tools later...) ? > > > >>>>>>>>> > > > >>>>>>>>> Guillaume > > > >>>>>>>>> > > > >>>>>>>>> Le lun. 14 mars 2022 à 07:43, Christoph Läubrich < > > > >>>> m...@laeubi-soft.de> > > > >>>>>> a > > > >>>>>>>>> écrit : > > > >>>>>>>>> > > > >>>>>>>>>> I also agree that the test at least should run, we use this > > > >> property > > > >>>>>> to > > > >>>>>>>>>> run the test and produce result and later on have a > buildstep > > > that > > > >>>>>>>>>> analyze the results (and probably fail the build job). > > > >>>>>>>>>> > > > >>>>>>>>>> As it is not recommend, I wonder what is the recommended way > > to > > > >>>>>> archive > > > >>>>>>>>>> something similar? > > > >>>>>>>>>> > > > >>>>>>>>>> Am 14.03.22 um 06:29 schrieb Olivier Lamy: > > > >>>>>>>>>>> On Mon, 14 Mar 2022 at 11:55, Tibor Digana < > > > >> tibordig...@apache.org > > > >>>>> > > > >>>>>>>>>> wrote: > > > >>>>>>>>>>> > > > >>>>>>>>>>>> In case of the user property *maven.test.failure.ignore* > the > > > >> MOJO > > > >>>>>>>> must > > > >>>>>>>>>> not > > > >>>>>>>>>>>> throw any exception which is interpreted by the Maven Core > > as > > > >>>> BUILD > > > >>>>>>>>>>>> SUCCESS. > > > >>>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> This is a very simple reduction of the problem description. > > > >>>>>>>>>>> The documentation here > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>> > > > >>>> > > > >> > > > > > > https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#testFailureIgnore > > > >>>>>>>>>>> says > > > >>>>>>>>>>> "Set this to "true" to ignore a failure during testing. Its > > use > > > >> is > > > >>>>>>>> NOT > > > >>>>>>>>>>> RECOMMENDED, but quite convenient on occasion" > > > >>>>>>>>>>> > > > >>>>>>>>>>> Personally, I understand this to ignore failures in junit > > > results > > > >>>> BUT > > > >>>>>>>>> at > > > >>>>>>>>>>> least I want the tests to run. > > > >>>>>>>>>>> I guess this is how our users use this feature (feature we > do > > > not > > > >>>>>>>>>> recommend > > > >>>>>>>>>>> by the way...) > > > >>>>>>>>>>> And this is perfectly explained here > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>> > > > >>>> > > > >> > > > > > > https://issues.apache.org/jira/browse/SUREFIRE-1426?focusedCommentId=16188077&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16188077 > > > >>>>>>>>>>> there is a difference between ignoring some junit failures > > and > > > >>>>>>>>> ignoring a > > > >>>>>>>>>>> configuration error because some jvm args has been > > > misconfigured > > > >>>> for > > > >>>>>>>>> many > > > >>>>>>>>>>> reasons and surefire cannot start. > > > >>>>>>>>>>> > > > >>>>>>>>>>> If I follow your reasoning ("MOJO must not throw any > > exception > > > ") > > > >>>> we > > > >>>>>>>>>> should > > > >>>>>>>>>>> ignore every surefire configuration error and keep running > > the > > > >>>> build > > > >>>>>>>>>> until > > > >>>>>>>>>>> the end and says BUILD SUCCESS > > > >>>>>>>>>>> what about > > > >>>>>>>>>>> > > > >>>>>>>>>>> mvn test -Dsurefire.rerunFailingTestsCount=notanumber > > > >>>>>>>>>>> -Dmaven.test.failure.ignore=true > > > >>>>>>>>>>> > > > >>>>>>>>>>> we should not throw any exceptions as you said below and > keep > > > >>>> saying > > > >>>>>>>>>> BUILD > > > >>>>>>>>>>> SUCCESS? > > > >>>>>>>>>>> argLine is a configuration element as any others so it > should > > > >> fail > > > >>>>>>>>>> directly > > > >>>>>>>>>>> and not be ignored especially when the surefire plugin > cannot > > > >> run. > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>>> We have received an internal requirement to change the > > > behavior > > > >> of > > > >>>>>>>> the > > > >>>>>>>>>> user > > > >>>>>>>>>>>> property *maven.test.failure.ignore* so that the behavior > > will > > > >>>> have > > > >>>>>>>>> one > > > >>>>>>>>>>>> exception. > > > >>>>>>>>>>> > > > >>>>>>>>>>> Suppose that you have JDK 1.8 but you use: > > > >>>>>>>>>>>> /jre/java --add-reads ... > > > >>>>>>>>>>>> The outcome is JVM exit with an error message. > > > >>>>>>>>>>>> I agree with Herve who also says that > > > >> *maven.test.failure.ignore* > > > >>>>>>>>>> should > > > >>>>>>>>>>>> not allow the MOJO to throw the exception. It is not a > > typical > > > >> JVM > > > >>>>>>>>>>>> segmentation fault or another JVM crash where we cannot do > > > >>>> anything > > > >>>>>>>>>> about > > > >>>>>>>>>>>> it, and where the entire build would crash in the command > > line > > > >>>> which > > > >>>>>>>>>>>> of course means that the build cannot normally be > > interpreted > > > as > > > >>>>>>>> BUILD > > > >>>>>>>>>>>> SUCCESS. So we are still on the same level of failures > > related > > > >> to > > > >>>>>>>> the > > > >>>>>>>>>> test > > > >>>>>>>>>>>> purposes. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> On the other hand, Olivier has reopened the issues > > > SUREFIRE-1426 > > > >>>> and > > > >>>>>>>>>>>> SUREFIRE-1681 where the exceptional behavior of the > feature > > is > > > >>>>>>>>> expected. > > > >>>>>>>>>>>> This is exactly the reason why I closed these tickets > > several > > > >>>> years > > > >>>>>>>>> ago > > > >>>>>>>>>> and > > > >>>>>>>>>>>> my proposal was to not to have any exceptions in the > feature > > > >>>>>>>> behavior > > > >>>>>>>>>> and > > > >>>>>>>>>>>> the proposal was to introduce a new user property for > exact > > > use > > > >>>>>>>> cases. > > > >>>>>>>>>>>> The next idea, which comes from two developers, would > > provide > > > us > > > >>>>>>>> with > > > >>>>>>>>>> the > > > >>>>>>>>>>>> same non exceptional and exact behavior of the user > property > > > >>>>>>>>>>>> *maven.test.failure.ignore* but it would also provide us > > with > > > >> new > > > >>>>>>>> user > > > >>>>>>>>>>>> property in the case with fine grade control of the build > > > >> errors, > > > >>>>>>>> e.g. > > > >>>>>>>>>>>> *maven.test.jvm.error.ignore*. > > > >>>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> with a default of? > > > >>>>>>>>>>> honestly I just see this new parameter as introducing more > > > >>>> complexity > > > >>>>>>>>> in > > > >>>>>>>>>> an > > > >>>>>>>>>>> already very complex code and not worth it. > > > >>>>>>>>>>> again read both issue's comments and my comments. > > > >>>>>>>>>>> Please try to have a user POV and think about making our > > users' > > > >>>>>>>>>> experience > > > >>>>>>>>>>> more simple. > > > >>>>>>>>>>> > > > >>>>>>>>>>> This should be very simple: > > > >>>>>>>>>>> If surefire forked jvm cannot start it's build error and > > cannot > > > >> run > > > >>>>>>>> any > > > >>>>>>>>>>> tests, it's a problem users want to know immediately > because > > it > > > >> can > > > >>>>>>>> be > > > >>>>>>>>>> for > > > >>>>>>>>>>> a lot of reasons: wrong argLine, not enough memory on the > > > system > > > >>>>>>>> etc... > > > >>>>>>>>>>> > > > >>>>>>>>>>> AND AGAIN it is very different from ignoring junit result > > > >> failures. > > > >>>>>>>>>>> > > > >>>>>>>>>>> Try to look at a user point of view and think about what is > > the > > > >> use > > > >>>>>>>>> case > > > >>>>>>>>>> of > > > >>>>>>>>>>> the property maven.test.failure.ignore=true, I guess 99% of > > the > > > >>>> time, > > > >>>>>>>>>> users > > > >>>>>>>>>>> wants to run all their tests (even on a CI with different > > > matrix) > > > >>>> so > > > >>>>>>>>> they > > > >>>>>>>>>>> can collect all the tests results which has runned to see > if > > > >> there > > > >>>> is > > > >>>>>>>>> any > > > >>>>>>>>>>> issue for some combination of their matrix and eventually > > turn > > > >> the > > > >>>>>>>>> result > > > >>>>>>>>>>> as unstable (this is a very typical use case in Jenkins and > > was > > > >>>> even > > > >>>>>>>> a > > > >>>>>>>>>>> built in feature of the previous Jenkins Maven plugin) > > > >>>>>>>>>>> BUT if for any reasons one of the module do not start his > > tests > > > >>>>>>>> because > > > >>>>>>>>>> the > > > >>>>>>>>>>> jvm cannot be start the users will not see that and will be > > > >> totally > > > >>>>>>>>> blind > > > >>>>>>>>>>> until maybe someone look inside a very very large log file > > > (which > > > >>>>>>>> means > > > >>>>>>>>>>> probably never) > > > >>>>>>>>>>> > > > >>>>>>>>>>> Long story short as my experience as a user facing this > > > >>>> problem/bug: > > > >>>>>>>>>>> I had the case on a very large multi modules (~250 modules) > > > build > > > >>>> of > > > >>>>>>>> a > > > >>>>>>>>>> very > > > >>>>>>>>>>> used open source project. > > > >>>>>>>>>>> I was using this maven.test.failure.ignore property but one > > of > > > >> the > > > >>>>>>>>>> modules > > > >>>>>>>>>>> had a bad jpms configuration for a jdk17 profile on the CI. > > > >>>>>>>>>>> The build has a matrix of 2 os and 4 jdks and different > maven > > > run > > > >>>>>>>> which > > > >>>>>>>>>>> means around ~60k tests and a Jenkins log file about 40M > > > >>>>>>>>>>> So because of this property the build was not failing and > > kept > > > >>>> saying > > > >>>>>>>>>> BUILD > > > >>>>>>>>>>> SUCCESS for weeks/months and basically not testing one > module > > > >> with > > > >>>>>>>> jdk > > > >>>>>>>>>> 17... > > > >>>>>>>>>>> And frankly you do not dig into a log file of 32M after > each > > > run > > > >>>>>>>>>> especially > > > >>>>>>>>>>> when it says success... > > > >>>>>>>>>>> 3 days after the first release claiming supporting jdk 17 > we > > > >>>>>>>> received a > > > >>>>>>>>>> bug > > > >>>>>>>>>>> report about a something not working with jdk17.... > > > >>>>>>>>>>> and guess what? Where was this feature supposed to be > tested? > > > >>>>>>>>>>> > > > >>>>>>>>>>> so I frankly believe we do not need a complex new property, > > in > > > >> this > > > >>>>>>>>> case > > > >>>>>>>>>>> this should fail directly because this will improve user > > > >>>> experience. > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> I would like to open the discussion on this topic. You're > > > >> welcome! > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Cheers > > > >>>>>>>>>>>> Tibor > > > >>>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>> > > --------------------------------------------------------------------- > > > >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > >>>>>>>>>> For additional commands, e-mail: dev-h...@maven.apache.org > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> -- > > > >>>>>>>>> ------------------------ > > > >>>>>>>>> Guillaume Nodet > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>>>> > > > --------------------------------------------------------------------- > > > >>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > >>>>>> For additional commands, e-mail: dev-h...@maven.apache.org > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>>> > > > >>>> > > --------------------------------------------------------------------- > > > >>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > >>>> For additional commands, e-mail: dev-h...@maven.apache.org > > > >>>> > > > >>>> > > > >>> > > > >> > > > >> > --------------------------------------------------------------------- > > > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > >> For additional commands, e-mail: dev-h...@maven.apache.org > > > >> > > > >> > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > For additional commands, e-mail: dev-h...@maven.apache.org > > > > > > > > > > -- > > Sławomir Jaranowski > > >
