Re: Inclusion of Failsafe plugin by default
Hi Karl, On Sun, Dec 10, 2023 at 1:37 PM Karl Heinz Marbaise wrote: > > First your assumption is correct it's not part of the default life > cycle[1] which means it is not bound by default. If you like to use it > you have to make the binding yourself. > > Maven Failsafe Plugin is intended for running integration tests based on > the naming schema "*IT.java" (etc.) > > So why isn't it added by default? > > The answer is more or less simple. Because it's not always needed nor used. > Maybe a silly question, but is there a cost (as in runtime or maintenance) associated to having Failsafe being added? I can give you some statistical information based on the downloads via > central. > > The maven-surefire-plugin (over all versions) is downloaded ca. 30 > million times a month. The maven-failsafe-plugin (over all versions) is > downloaded ca. 5 million times a month[2]. > > In other words the majority of the cases people running unit tests but > not integration tests... (ca. 1/6).. > Thanks a lot for the link to those statistics, that's very insightful. I agree that maven-surefire-plugin is significantly more used than maven-failsafe-plugin, but in absolute numbers, it's still quite popular, isn't it? My initial understanding was that the fact that the Failsafe plugin was mentioned as "core plugin" on https://maven.apache.org/plugins/index.html meant that it would be automatically added to the default bindings. But now, having seen the numbers, I suppose that the (relative) popularity of the plugin decides whether it is included or not? > Another question would arise if you like to make it part of the default > life cycle: > > Should both goals "integration-test" bound to the lifecycle as well as > "verify" or just one of them? > I'm probably biased as Spring Boot user, but this framework has chosen to bind to both "integration-test" and "verify": https://github.com/spring-projects/spring-boot/blob/main/spring-boot-project/spring-boot-starters/spring-boot-starter-parent/build.gradle#L90 . > Another thing is very often people create a profile to activate the > integration tests whenever they like based on the time consumption of > integration tests. That's simply because integration tests are running a > way longer than unit test. Something like "mvn verify -Prun-its" (which > by the way the maven teams does as well for all the plugins). > I'm indeed familiar with the use of the profiles for such cases, but I'm wondering, what about simply calling either "mvn test" or "mvn verify" whether one wants to run unit tests only, or unit tests *plus* integration tests? > Another question arises: Is this enough because running integration test > could require other things? For example starting a thing like > tomcat/jetty etc. or docker containers and even many other things? Or > would it be better to control that via the integration test itself (via > testcontainers or alike) etc. > Indeed, my personal experience has been to use Testcontainers lately. I should probably have been more explicit in my initial message about the kind of behaviour that a beginner Maven user may find unpredictable. Out of the box, if I'm not mistaken: 1. calling "mvn test" with a class called "AnythingTest" triggers Surefire. 2. calling "mvn integration-test" with a class called "AnythingIT" does not trigger Failsafe (thus ignoring the integration test silently). I'm wondering if that could explain why people (mis)use Surefire to run integration tests. As I said above, as my project uses the Spring Boot parent POM, I only need to declare "org.apache.maven.plugins:maven-failsafe-plugin" and I'm good to go. So that's clearly not a huge issue for me. I was just wondering why the current situation was like this. Regards, -- Damiano Albani
Re: Inclusion of Failsafe plugin by default
On 09.12.23 23:28, Damiano Albani wrote: Hello, I'm a long-time Maven user but I've just recently discovered that, unless I'm mistaken, the Failsafe plugin isn't part of the default set of plugins which are configured "out of the box". Is this correct? If so, I find this quite strange given that Failsafe is mentioned on https://maven.apache.org/plugins/index.html as a "core plugin", just like its sibling Surefire, the latter being indeed included. In order to change that, would it simply be a matter of adding Failsafe to https://github.com/apache/maven/blob/maven-3.9.6/maven-core/src/main/resources/META-INF/plexus/default-bindings.xml ? Or are there locations in the Maven codebase that would need to be changed? First your assumption is correct it's not part of the default life cycle[1] which means it is not bound by default. If you like to use it you have to make the binding yourself. Maven Failsafe Plugin is intended for running integration tests based on the naming schema "*IT.java" (etc.) So why isn't it added by default? The answer is more or less simple. Because it's not always needed nor used. I can give you some statistical information based on the downloads via central. The maven-surefire-plugin (over all versions) is downloaded ca. 30 million times a month. The maven-failsafe-plugin (over all versions) is downloaded ca. 5 million times a month[2]. In other words the majority of the cases people running unit tests but not integration tests... (ca. 1/6).. Another question would arise if you like to make it part of the default life cycle: Should both goals "integration-test" bound to the lifecycle as well as "verify" or just one of them? Another thing is very often people create a profile to activate the integration tests whenever they like based on the time consumption of integration tests. That's simply because integration tests are running a way longer than unit test. Something like "mvn verify -Prun-its" (which by the way the maven teams does as well for all the plugins). Another question arises: Is this enough because running integration test could require other things? For example starting a thing like tomcat/jetty etc. or docker containers and even many other things? Or would it be better to control that via the integration test itself (via testcontainers or alike) etc. Kind regards Karl Heinz Marbaise Apache Maven PMC [1]: https://maven.apache.org/ref/3.9.6/maven-core/default-bindings.html [2]: https://blog.soebes.io/posts/2023/08/2023-08-06-download-statistics-for-maven/ - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
RE: Inclusion of Failsafe plugin by default
It would need to be added to the super pom, I suppose... https://maven.apache.org/ref/3.9.6/maven-model-builder/super-pom.html Note that based on the comment on line 50, I suspect they are not likely to want to do this -Original Message- From: Damiano Albani Sent: Saturday, December 9, 2023 5:28 PM To: users@maven.apache.org Subject: Inclusion of Failsafe plugin by default CAUTION: This email originated from outside our organisation - damiano.alb...@gmail.com Do not click on links, open attachments, or respond unless you recognize the sender and can validate the content is safe. Hello, I'm a long-time Maven user but I've just recently discovered that, unless I'm mistaken, the Failsafe plugin isn't part of the default set of plugins which are configured "out of the box". Is this correct? If so, I find this quite strange given that Failsafe is mentioned on https://clicktime.symantec.com/15t5ejWR7ZXQJ7SejrESU?h=Qd-1cRIUr7dtc-zYyImHm56sVUcbukg2-o2L3nNyhvA=&u=https://maven.apache.org/plugins/index.html as a "core plugin", just like its sibling Surefire, the latter being indeed included. In order to change that, would it simply be a matter of adding Failsafe to https://clicktime.symantec.com/15t5ZuK8ewqotAcjCHqHr?h=-VNU9pFgfbgyo-3dqUgSPtAvthtlLSzDQLuUFwCOv04=&u=https://github.com/apache/maven/blob/maven-3.9.6/maven-core/src/main/resources/META-INF/plexus/default-bindings.xml ? Or are there locations in the Maven codebase that would need to be changed? Regards, -- Damiano Albani This message is for information purposes only. It is not a recommendation, advice, offer or solicitation to buy or sell a product or service, nor an official confirmation of any transaction. It is directed at persons who are professionals and is intended for the recipient(s) only. It is not directed at retail customers. This message is subject to the terms at: https://www.cib.barclays/disclosures/web-and-email-disclaimer.html. For important disclosures, please see: https://www.cib.barclays/disclosures/sales-and-trading-disclaimer.html regarding marketing commentary from Barclays Sales and/or Trading desks, who are active market participants; https://www.cib.barclays/disclosures/barclays-global-markets-disclosures.html regarding our standard terms for Barclays Corporate and Investment Bank where we trade with you in principal-to-principal wholesale markets transactions; and in respect to Barclays Research, including disclosures relating to specific issuers, see: http://publicresearch.barclays.com. __ If you are incorporated or operating in Australia, read these important disclosures: https://www.cib.barclays/disclosures/important-disclosures-asia-pacific.html. __ For more details about how we use personal information, see our privacy notice: https://www.cib.barclays/disclosures/personal-information-use.html. __
Inclusion of Failsafe plugin by default
Hello, I'm a long-time Maven user but I've just recently discovered that, unless I'm mistaken, the Failsafe plugin isn't part of the default set of plugins which are configured "out of the box". Is this correct? If so, I find this quite strange given that Failsafe is mentioned on https://maven.apache.org/plugins/index.html as a "core plugin", just like its sibling Surefire, the latter being indeed included. In order to change that, would it simply be a matter of adding Failsafe to https://github.com/apache/maven/blob/maven-3.9.6/maven-core/src/main/resources/META-INF/plexus/default-bindings.xml ? Or are there locations in the Maven codebase that would need to be changed? Regards, -- Damiano Albani
