I just tried with master and the problem is the same. The reason is that in my use case, the forked goals are executed before the standard executions, and not even in the same project.
Le mar. 18 mai 2021 à 20:41, Martin Kanters <martinkant...@apache.org> a écrit : > Not sure how I've missed this post. Have you tried this build with the > master build of Maven? > MNG-6566 [1] should prevent any unnecessary double executions, thus > optimizing the buildplan. > I'm interested if this will solve the problem at hand. > > Martin > > [1] https://issues.apache.org/jira/browse/MNG-6566 > > Op di 18 mei 2021 om 14:57 schreef Guillaume Nodet <gno...@apache.org>: > > > I'm looking a bit at aggregator goals. > > At first glance, it seems most of the problem comes from the fact that > the > > build ordering is done mostly per-project. This causes obvious problems > > for aggregators when a mojo needs a given phase to be executed for all > > children before it can run (for example the compilation phase before > > creating the javadoc). Another problem comes from the clean task which > has > > obvious side effects. Calling clean with an aggregator goal involved in > > the build seems like a recipe for problems... > > > > Would you see bad side effects to split the build in smaller chunks > > (basically down to a mojo execution) so that ordering can be changed in a > > more meaningful way ? For example when running an aggregate javadoc, the > > top level project would be built first until before the javadoc aggregate > > goal. The build would then go to all other projects (like the forked > > lifecycle) until all required goals have been run, then resume from the > > javadoc goal. > > Note that with the current behavior, when you run "clean verify", the > > execution is the following > > > > [INFO] Scanning for projects... > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] Reactor Build Order: > > [INFO] > > [INFO] forked > > [pom] > > [INFO] forked-mod1 > > [jar] > > [INFO] forked-mod2 > > [jar] > > [INFO] > > [INFO] ---------------< org.mvndaemon.mvnd.test.forked:forked > > >---------------- > > [INFO] Building forked 0.0.1-SNAPSHOT > > [1/3] > > [INFO] --------------------------------[ pom > > ]--------------------------------- > > [INFO] > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked --- > > [INFO] >>> maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) > > > compile @ forked >>> > > [INFO] > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > [INFO] Forking forked-mod1 0.0.1-SNAPSHOT > > [INFO] > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > forked-mod1 --- > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > forked-mod1 --- > > [INFO] > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > [INFO] Forking forked-mod2 0.0.1-SNAPSHOT > > [INFO] > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > forked-mod2 --- > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > forked-mod2 --- > > [INFO] <<< maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) < > > compile @ forked <<< > > [INFO] --- maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) @ > > forked --- > > [INFO] > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod1 > > >------------- > > [INFO] Building forked-mod1 0.0.1-SNAPSHOT > > [2/3] > > [INFO] --------------------------------[ jar > > ]--------------------------------- > > [INFO] > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod1 --- > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > forked-mod1 --- > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > forked-mod1 --- > > [INFO] --- maven-resources-plugin:2.6:testResources > (default-testResources) > > @ forked-mod1 --- > > [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) > @ > > forked-mod1 --- > > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ forked-mod1 > > --- > > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod1 --- > > [INFO] > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod2 > > >------------- > > [INFO] Building forked-mod2 0.0.1-SNAPSHOT > > [3/3] > > [INFO] --------------------------------[ jar > > ]--------------------------------- > > [INFO] > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod2 --- > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > forked-mod2 --- > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > forked-mod2 --- > > [INFO] --- maven-resources-plugin:2.6:testResources > (default-testResources) > > @ forked-mod2 --- > > [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) > @ > > forked-mod2 --- > > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ forked-mod2 > > --- > > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod2 --- > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] Reactor Summary for forked 0.0.1-SNAPSHOT: > > [INFO] > > [INFO] forked ............................................. SUCCESS [ > > 1.921 s] > > [INFO] forked-mod1 ........................................ SUCCESS [ > > 0.498 s] > > [INFO] forked-mod2 ........................................ SUCCESS [ > > 0.046 s] > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] BUILD SUCCESS > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] Total time: 2.535 s > > [INFO] Finished at: 2021-05-18T14:32:09+02:00 > > [INFO] > > ------------------------------------------------------------------------ > > > > As you can see, the resources and compile goals are run twice for each > > project. Furthermore, given the clean goal is in the loop, the compiler > > actually compiles things twice. > > > > The build plan could be rewritten as > > > > [INFO] Scanning for projects... > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] Reactor Build Order: > > [INFO] > > [INFO] forked > > [pom] > > [INFO] forked-mod1 > > [jar] > > [INFO] forked-mod2 > > [jar] > > [INFO] > > [INFO] ---------------< org.mvndaemon.mvnd.test.forked:forked > > >---------------- > > [INFO] Building forked 0.0.1-SNAPSHOT > > [1/3] > > [INFO] --------------------------------[ pom > > ]--------------------------------- > > [INFO] > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked --- > > ... normal lifecycle interrupted because the next goal is the aggregator > > goal ... > > [INFO] > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod1 > > >------------- > > [INFO] Building forked-mod1 0.0.1-SNAPSHOT > > [2/3] > > [INFO] --------------------------------[ jar > > ]--------------------------------- > > [INFO] > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod1 --- > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > forked-mod1 --- > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > forked-mod1 --- > > ... interrupting forked-mod1 build as it reached the required point > > execution ... > > [INFO] > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod2 > > >------------- > > [INFO] Building forked-mod2 0.0.1-SNAPSHOT > > [3/3] > > [INFO] --------------------------------[ jar > > ]--------------------------------- > > [INFO] > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod2 --- > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > forked-mod2 --- > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > forked-mod2 --- > > ... interrupting forked-mod2 build as it reached the required point > > execution ... > > ... resuming the top level project... > > [INFO] > > [INFO] ---------------< org.mvndaemon.mvnd.test.forked:forked > > >---------------- > > [INFO] Building forked 0.0.1-SNAPSHOT > > [1/3] > > [INFO] --------------------------------[ pom > > ]--------------------------------- > > [INFO] > > [INFO] --- maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) @ > > forked --- > > ... then resuming the forked-mod1 project... > > [INFO] > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod1 > > >------------- > > [INFO] Building forked-mod1 0.0.1-SNAPSHOT > > [2/3] > > [INFO] --------------------------------[ jar > > ]--------------------------------- > > [INFO] > > [INFO] --- maven-resources-plugin:2.6:testResources > (default-testResources) > > @ forked-mod1 --- > > [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) > @ > > forked-mod1 --- > > [INFO] --- maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) @ > > forked --- > > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ forked-mod1 > > --- > > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod1 --- > > ... and resuming the forked-mod2 project... > > [INFO] > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod2 > > >------------- > > [INFO] Building forked-mod2 0.0.1-SNAPSHOT > > [3/3] > > [INFO] --------------------------------[ jar > > ]--------------------------------- > > [INFO] > > [INFO] --- maven-resources-plugin:2.6:testResources > (default-testResources) > > @ forked-mod2 --- > > [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) > @ > > forked-mod2 --- > > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ forked-mod2 > > --- > > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod2 --- > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] Reactor Summary for forked 0.0.1-SNAPSHOT: > > [INFO] > > [INFO] forked ............................................. SUCCESS [ > > 1.921 s] > > [INFO] forked-mod1 ........................................ SUCCESS [ > > 0.498 s] > > [INFO] forked-mod2 ........................................ SUCCESS [ > > 0.046 s] > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] BUILD SUCCESS > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] Total time: 2.535 s > > [INFO] Finished at: 2021-05-18T14:32:09+02:00 > > [INFO] > > ------------------------------------------------------------------------ > > > > This would also solve the parallel build problem because each project > would > > not be able to be both "cleaned" and "compiled" concurrently, which is > > currently possible in the current state. > > > > > > Le lun. 17 mai 2021 à 14:06, Guillaume Nodet <gno...@apache.org> a > écrit : > > > > > I've raised https://issues.apache.org/jira/browse/MNG-7156 > > > > > > Le mer. 12 mai 2021 à 17:57, Falko Modler <f.mod...@gmx.net> a écrit : > > > > > >> Hi Guillaume, > > >> > > >> aggregation goals and parallel builds in combination are a bit of a > > >> mess, e.g.: > > >> > > >> - https://issues.apache.org/jira/browse/MNG-6843 > > >> - https://github.com/apache/maven/pull/413 > > >> - https://www.mail-archive.com/dev@maven.apache.org/msg123439.html > > >> > > >> Cheers, > > >> > > >> Falko > > >> > > >> Am 12.05.2021 um 17:25 schrieb Guillaume Nodet: > > >> > Hi > > >> > > > >> > I've analyzed a bug reported on mvnd this afternoon ( > > >> > https://github.com/mvndaemon/mvnd/issues/408). It appears that the > > >> parent > > >> > pom executes the javadoc aggregate goal, which forks the lifecycle > of > > >> the > > >> > children modules in order to compile the sources. In a traditional > > >> build, > > >> > this does not cause any real problem, but in a parallel build, a > > clean > > >> > verify can definitely cause issues if the forked lifecycle and the > > >> normal > > >> > project build (and especially the clean) are run concurrently. > > >> > This definitely looks like an issue to me. Any idea where I should > > >> look at > > >> > how to solve the problem ? I wonder if the MojoExecutor should > > somehow > > >> > delegate to the Builder which is responsible for synchronizing the > > >> > executions in the case of a multithreaded build... > > >> > Thoughts ? > > >> > > > >> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > >> For additional commands, e-mail: dev-h...@maven.apache.org > > >> > > >> > > > > > > -- > > > ------------------------ > > > Guillaume Nodet > > > > > > > > > > -- > > ------------------------ > > Guillaume Nodet > > > -- ------------------------ Guillaume Nodet
