Ah, too bad. I thought it was relevant. Perhaps I can take a look at this with Maarten Mulders, but that will not be in the coming week at least.
Op di 18 mei 2021 om 21:34 schreef Guillaume Nodet <gno...@apache.org>: > I just tried with master and the problem is the same. > The reason is that in my use case, the forked goals are executed before the > standard executions, and not even in the same project. > > Le mar. 18 mai 2021 à 20:41, Martin Kanters <martinkant...@apache.org> a > écrit : > > > Not sure how I've missed this post. Have you tried this build with the > > master build of Maven? > > MNG-6566 [1] should prevent any unnecessary double executions, thus > > optimizing the buildplan. > > I'm interested if this will solve the problem at hand. > > > > Martin > > > > [1] https://issues.apache.org/jira/browse/MNG-6566 > > > > Op di 18 mei 2021 om 14:57 schreef Guillaume Nodet <gno...@apache.org>: > > > > > I'm looking a bit at aggregator goals. > > > At first glance, it seems most of the problem comes from the fact that > > the > > > build ordering is done mostly per-project. This causes obvious > problems > > > for aggregators when a mojo needs a given phase to be executed for all > > > children before it can run (for example the compilation phase before > > > creating the javadoc). Another problem comes from the clean task which > > has > > > obvious side effects. Calling clean with an aggregator goal involved > in > > > the build seems like a recipe for problems... > > > > > > Would you see bad side effects to split the build in smaller chunks > > > (basically down to a mojo execution) so that ordering can be changed > in a > > > more meaningful way ? For example when running an aggregate javadoc, > the > > > top level project would be built first until before the javadoc > aggregate > > > goal. The build would then go to all other projects (like the forked > > > lifecycle) until all required goals have been run, then resume from the > > > javadoc goal. > > > Note that with the current behavior, when you run "clean verify", the > > > execution is the following > > > > > > [INFO] Scanning for projects... > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Reactor Build Order: > > > [INFO] > > > [INFO] forked > > > [pom] > > > [INFO] forked-mod1 > > > [jar] > > > [INFO] forked-mod2 > > > [jar] > > > [INFO] > > > [INFO] ---------------< org.mvndaemon.mvnd.test.forked:forked > > > >---------------- > > > [INFO] Building forked 0.0.1-SNAPSHOT > > > [1/3] > > > [INFO] --------------------------------[ pom > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked --- > > > [INFO] >>> maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) > > > > compile @ forked >>> > > > [INFO] > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > [INFO] Forking forked-mod1 0.0.1-SNAPSHOT > > > [INFO] > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > > forked-mod1 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > > forked-mod1 --- > > > [INFO] > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > [INFO] Forking forked-mod2 0.0.1-SNAPSHOT > > > [INFO] > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > > forked-mod2 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > > forked-mod2 --- > > > [INFO] <<< maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) < > > > compile @ forked <<< > > > [INFO] --- maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) @ > > > forked --- > > > [INFO] > > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod1 > > > >------------- > > > [INFO] Building forked-mod1 0.0.1-SNAPSHOT > > > [2/3] > > > [INFO] --------------------------------[ jar > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod1 > --- > > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > > forked-mod1 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > > forked-mod1 --- > > > [INFO] --- maven-resources-plugin:2.6:testResources > > (default-testResources) > > > @ forked-mod1 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:testCompile > (default-testCompile) > > @ > > > forked-mod1 --- > > > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ > forked-mod1 > > > --- > > > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod1 --- > > > [INFO] > > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod2 > > > >------------- > > > [INFO] Building forked-mod2 0.0.1-SNAPSHOT > > > [3/3] > > > [INFO] --------------------------------[ jar > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod2 > --- > > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > > forked-mod2 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > > forked-mod2 --- > > > [INFO] --- maven-resources-plugin:2.6:testResources > > (default-testResources) > > > @ forked-mod2 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:testCompile > (default-testCompile) > > @ > > > forked-mod2 --- > > > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ > forked-mod2 > > > --- > > > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod2 --- > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Reactor Summary for forked 0.0.1-SNAPSHOT: > > > [INFO] > > > [INFO] forked ............................................. SUCCESS [ > > > 1.921 s] > > > [INFO] forked-mod1 ........................................ SUCCESS [ > > > 0.498 s] > > > [INFO] forked-mod2 ........................................ SUCCESS [ > > > 0.046 s] > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] BUILD SUCCESS > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Total time: 2.535 s > > > [INFO] Finished at: 2021-05-18T14:32:09+02:00 > > > [INFO] > > > > ------------------------------------------------------------------------ > > > > > > As you can see, the resources and compile goals are run twice for > each > > > project. Furthermore, given the clean goal is in the loop, the > compiler > > > actually compiles things twice. > > > > > > The build plan could be rewritten as > > > > > > [INFO] Scanning for projects... > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Reactor Build Order: > > > [INFO] > > > [INFO] forked > > > [pom] > > > [INFO] forked-mod1 > > > [jar] > > > [INFO] forked-mod2 > > > [jar] > > > [INFO] > > > [INFO] ---------------< org.mvndaemon.mvnd.test.forked:forked > > > >---------------- > > > [INFO] Building forked 0.0.1-SNAPSHOT > > > [1/3] > > > [INFO] --------------------------------[ pom > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked --- > > > ... normal lifecycle interrupted because the next goal is the > aggregator > > > goal ... > > > [INFO] > > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod1 > > > >------------- > > > [INFO] Building forked-mod1 0.0.1-SNAPSHOT > > > [2/3] > > > [INFO] --------------------------------[ jar > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod1 > --- > > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > > forked-mod1 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > > forked-mod1 --- > > > ... interrupting forked-mod1 build as it reached the required point > > > execution ... > > > [INFO] > > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod2 > > > >------------- > > > [INFO] Building forked-mod2 0.0.1-SNAPSHOT > > > [3/3] > > > [INFO] --------------------------------[ jar > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ forked-mod2 > --- > > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > > forked-mod2 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ > > > forked-mod2 --- > > > ... interrupting forked-mod2 build as it reached the required point > > > execution ... > > > ... resuming the top level project... > > > [INFO] > > > [INFO] ---------------< org.mvndaemon.mvnd.test.forked:forked > > > >---------------- > > > [INFO] Building forked 0.0.1-SNAPSHOT > > > [1/3] > > > [INFO] --------------------------------[ pom > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) @ > > > forked --- > > > ... then resuming the forked-mod1 project... > > > [INFO] > > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod1 > > > >------------- > > > [INFO] Building forked-mod1 0.0.1-SNAPSHOT > > > [2/3] > > > [INFO] --------------------------------[ jar > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-resources-plugin:2.6:testResources > > (default-testResources) > > > @ forked-mod1 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:testCompile > (default-testCompile) > > @ > > > forked-mod1 --- > > > [INFO] --- maven-javadoc-plugin:3.2.0:aggregate-jar (aggregate-jar) @ > > > forked --- > > > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ > forked-mod1 > > > --- > > > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod1 --- > > > ... and resuming the forked-mod2 project... > > > [INFO] > > > [INFO] -------------< org.mvndaemon.mvnd.test.forked:forked-mod2 > > > >------------- > > > [INFO] Building forked-mod2 0.0.1-SNAPSHOT > > > [3/3] > > > [INFO] --------------------------------[ jar > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-resources-plugin:2.6:testResources > > (default-testResources) > > > @ forked-mod2 --- > > > [INFO] --- maven-compiler-plugin:3.8.0:testCompile > (default-testCompile) > > @ > > > forked-mod2 --- > > > [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ > forked-mod2 > > > --- > > > [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ forked-mod2 --- > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Reactor Summary for forked 0.0.1-SNAPSHOT: > > > [INFO] > > > [INFO] forked ............................................. SUCCESS [ > > > 1.921 s] > > > [INFO] forked-mod1 ........................................ SUCCESS [ > > > 0.498 s] > > > [INFO] forked-mod2 ........................................ SUCCESS [ > > > 0.046 s] > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] BUILD SUCCESS > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Total time: 2.535 s > > > [INFO] Finished at: 2021-05-18T14:32:09+02:00 > > > [INFO] > > > > ------------------------------------------------------------------------ > > > > > > This would also solve the parallel build problem because each project > > would > > > not be able to be both "cleaned" and "compiled" concurrently, which is > > > currently possible in the current state. > > > > > > > > > Le lun. 17 mai 2021 à 14:06, Guillaume Nodet <gno...@apache.org> a > > écrit : > > > > > > > I've raised https://issues.apache.org/jira/browse/MNG-7156 > > > > > > > > Le mer. 12 mai 2021 à 17:57, Falko Modler <f.mod...@gmx.net> a > écrit : > > > > > > > >> Hi Guillaume, > > > >> > > > >> aggregation goals and parallel builds in combination are a bit of a > > > >> mess, e.g.: > > > >> > > > >> - https://issues.apache.org/jira/browse/MNG-6843 > > > >> - https://github.com/apache/maven/pull/413 > > > >> - https://www.mail-archive.com/dev@maven.apache.org/msg123439.html > > > >> > > > >> Cheers, > > > >> > > > >> Falko > > > >> > > > >> Am 12.05.2021 um 17:25 schrieb Guillaume Nodet: > > > >> > Hi > > > >> > > > > >> > I've analyzed a bug reported on mvnd this afternoon ( > > > >> > https://github.com/mvndaemon/mvnd/issues/408). It appears that > the > > > >> parent > > > >> > pom executes the javadoc aggregate goal, which forks the lifecycle > > of > > > >> the > > > >> > children modules in order to compile the sources. In a > traditional > > > >> build, > > > >> > this does not cause any real problem, but in a parallel build, a > > > clean > > > >> > verify can definitely cause issues if the forked lifecycle and the > > > >> normal > > > >> > project build (and especially the clean) are run concurrently. > > > >> > This definitely looks like an issue to me. Any idea where I > should > > > >> look at > > > >> > how to solve the problem ? I wonder if the MojoExecutor should > > > somehow > > > >> > delegate to the Builder which is responsible for synchronizing the > > > >> > executions in the case of a multithreaded build... > > > >> > Thoughts ? > > > >> > > > > >> > > > >> > > > >> > --------------------------------------------------------------------- > > > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > >> For additional commands, e-mail: dev-h...@maven.apache.org > > > >> > > > >> > > > > > > > > -- > > > > ------------------------ > > > > Guillaume Nodet > > > > > > > > > > > > > > -- > > > ------------------------ > > > Guillaume Nodet > > > > > > > > -- > ------------------------ > Guillaume Nodet >
