[jira] [Commented] (MNG-6265) Controlling reactor order of modules
[ https://issues.apache.org/jira/browse/MNG-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109351#comment-16109351 ] August Shi commented on MNG-6265: - Thank you for pointing out the --builder option. We implemented a custom Builder and it seems to work for our use case; we can run modules for testing in an order different from the reactor order. The printing of modules is still in the reactor order, but it is something we can work on later. > Controlling reactor order of modules > > > Key: MNG-6265 > URL: https://issues.apache.org/jira/browse/MNG-6265 > Project: Maven > Issue Type: New Feature > Components: core >Affects Versions: 3.5.0 >Reporter: August Shi > Attachments: p.patch > > > We have a use case for controlling the reactor order in Maven, specifically > for running tests on Travis. The common pattern is to run "mvn install > -DskipTests; mvn test". After modules are installed (in the default order), > the modules can be run in any order during testing, e.g., running the failed > modules first, as in -Dsurefire.runOrder=failedfirst for Surefire. > Would this feature be useful for others? I am attaching a *draft* patch that > implements this feature by modifying DefaultMaven.java. I see no clean way of > implementing this feature through an extension, unless some new callback, > e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to > improve this draft? > One can test the feature by applying the patch, building Maven, and running > tests with the modules in a random order like this: > $ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean > install -DskipTests > $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d > maven* | shuf) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MNG-6265) Controlling reactor order of modules
[ https://issues.apache.org/jira/browse/MNG-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106596#comment-16106596 ] August Shi commented on MNG-6265: - We understand your concern about "feature creep", especially that you mentioned user-facing options. However, for our purposes, we may not need a new command-line option to the core as in the proposed draft patch; adding just one API callback, say "afterProjectsOrdered", similar to "afterSessionStarted" or "afterProjectsRead", would suffice. > If C breaks the build and you only change C In our use case A and/or B also changes. For example, an integration test in C may have failed, requiring to change not only C but also A and/or B. We then need to run multiple modules anyhow, and we just want a different order for "mvn test" -- compilation (with "mvn install -DskipTests") still obeys the reactor order. > we could make Maven respect that order [of multiple projects in -pl], as long > as the graph-order is respected! That would be already a step forward, although that would change an existing option, and some users may have relied on the order they get now. Adding a new option, "-opl", might be better, even if it respects both the given order and the graph. However, our use case really needs a way that does not respect the graph, only for testing purposes (after everything is compiled). > Controlling reactor order of modules > > > Key: MNG-6265 > URL: https://issues.apache.org/jira/browse/MNG-6265 > Project: Maven > Issue Type: New Feature > Components: core >Affects Versions: 3.5.0 >Reporter: August Shi > Attachments: p.patch > > > We have a use case for controlling the reactor order in Maven, specifically > for running tests on Travis. The common pattern is to run "mvn install > -DskipTests; mvn test". After modules are installed (in the default order), > the modules can be run in any order during testing, e.g., running the failed > modules first, as in -Dsurefire.runOrder=failedfirst for Surefire. > Would this feature be useful for others? I am attaching a *draft* patch that > implements this feature by modifying DefaultMaven.java. I see no clean way of > implementing this feature through an extension, unless some new callback, > e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to > improve this draft? > One can test the feature by applying the patch, building Maven, and running > tests with the modules in a random order like this: > $ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean > install -DskipTests > $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d > maven* | shuf) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MNG-6265) Controlling reactor order of modules
[
https://issues.apache.org/jira/browse/MNG-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106260#comment-16106260
]
August Shi commented on MNG-6265:
-
We are familiar with most of the options you listed, but none of them does what
we want: we want to run *all* the modules (not only some) but run them in a
prioritized order (that may differ from the reactor order). Note that
supporting our feature may not require big changes in the core. If just one
callback is added, we can implement the feature as an extension and not affect
the core further.
In our example, suppose that we run "mvn install -DskipTests" first (as
commonly done on Travis), and then we want to run "mvn test " to
run tests from all modules A, B, and C but in the order [C, A, B]. For example,
we may have changed code in all three modules, but we expect that C is the most
likely to fail.
This is what the default "mvn test" *conceptually* (not exactly) does:
mvn -pl A test
mvn -pl B test
mvn -pl C test
This is what our patch "mvn test -Dmaven.project.order.file=<(echo C; echo A;
echo B)" would allow:
mvn -pl C test # this is the key difference in order
mvn -pl A test
mvn -pl B test
If we run "mvn test -rf C":
mvn -pl C test # only runs C, does not run A and B
If we run "mvn test -pl C -am" (assuming C depends on both A and B):
mvn -pl A test
mvn -pl B test
mvn -pl C test # still runs C last
The option -fae does not change the order (and could not run C first) but
continues even after failure. Also, a combination of -rf and -am, i.e., "mvn
test -rf C -am", ends up the same as "mvn test -rf C", and the dependencies of
C are not built.
> Karl Heinz has been working on an extension which detects scm changes
That is indeed a useful feature. There is actually another Maven plugin we are
aware of that is similar:
https://github.com/vackosar/gitflow-incremental-builder
However, that plugin only *selects* to run *some* modules, and it could end up
selecting to run all modules in our example. It (1) does not prioritize in
what order to run them and (2) does not run them all (although we may not even
need to run all if the selection is safe). In the end, though, we still want
to prioritize the order of modules.
> Takari has a
> [takari-smart-builder|http://takari.io/book/30-team-maven.html#takari-smart-builder]
That is indeed another useful extension for parallel builds. Some researchers
have implemented similar parallelization for testing in Maven:
http://jonbell.net/preprint/releng15poster.pdf
http://jonbell.net/preprint/releng15abstract.pdf
Unfortunately, these features still do not prioritize the order of modules to
allow for [C, A, B].
> "Finding a failing test as soon as possible" is a valid requirement, but IMHO
> should work out-of-the-box
I don't see how that would work without manipulating the order of modules, but
I may be missing something.
> In the end I would really prefer that {{mvn test}} ... does exactly that.
We propose a really small change mvn test
-Dmaven.project.order.file=${file_with_order_of_modules}
> Controlling reactor order of modules
>
>
> Key: MNG-6265
> URL: https://issues.apache.org/jira/browse/MNG-6265
> Project: Maven
> Issue Type: New Feature
> Components: core
>Affects Versions: 3.5.0
>Reporter: August Shi
> Attachments: p.patch
>
>
> We have a use case for controlling the reactor order in Maven, specifically
> for running tests on Travis. The common pattern is to run "mvn install
> -DskipTests; mvn test". After modules are installed (in the default order),
> the modules can be run in any order during testing, e.g., running the failed
> modules first, as in -Dsurefire.runOrder=failedfirst for Surefire.
> Would this feature be useful for others? I am attaching a *draft* patch that
> implements this feature by modifying DefaultMaven.java. I see no clean way of
> implementing this feature through an extension, unless some new callback,
> e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to
> improve this draft?
> One can test the feature by applying the patch, building Maven, and running
> tests with the modules in a random order like this:
> $ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean
> install -DskipTests
> $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d
> maven* | shuf)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MNG-6265) Controlling reactor order of modules
[ https://issues.apache.org/jira/browse/MNG-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105811#comment-16105811 ] August Shi commented on MNG-6265: - Thanks for replying so quickly. Our overall goal is to find a failing test as soon as possible (if any test fails). We need to control the order in which modules are tested from the reactor base (and we further control the order in which classes are tested inside each module). Our use case (on Travis) always has two steps: first, "mvn install -DskipTests" compiles and installs the modules without running tests (and this is done in the reactor order, following the dependencies), and then "mvn test" runs the tests after install is successful (maybe a more correct way to express our use case is "mvn install -DskipTests && mvn test"). Since everything is installed in the first step, the modules can be run in any order in the second step, so we want to control that order for the second step. As an example, assume a project has modules A, B, and C. Based on the dependencies, the reactor order is [A, B, C]. In one run, "mvn install -DskipTests" builds modules in that order [A, B, C] and "mvn test" runs tests in that order [A, B, C]. However, if tests fail in module C, then for our next run, we still run "mvn install -DskipTests" in the order [A, B, C], but we want to control the order to run module C before others in the second, testing step, e.g., run "mvn test" in the order [C, A, B]. With this new order, if tests fail again in C, the build terminates faster, skipping testing in the remaining modules. This feature cannot be achieved with -Dsurefire.runOrder, because Surefire only controls the order of tests *within* one module, while we would like to control the order *across* several modules. > Controlling reactor order of modules > > > Key: MNG-6265 > URL: https://issues.apache.org/jira/browse/MNG-6265 > Project: Maven > Issue Type: New Feature > Components: core >Affects Versions: 3.5.0 >Reporter: August Shi > Attachments: p.patch > > > We have a use case for controlling the reactor order in Maven, specifically > for running tests on Travis. The common pattern is to run "mvn install > -DskipTests; mvn test". After modules are installed (in the default order), > the modules can be run in any order during testing, e.g., running the failed > modules first, as in -Dsurefire.runOrder=failedfirst for Surefire. > Would this feature be useful for others? I am attaching a *draft* patch that > implements this feature by modifying DefaultMaven.java. I see no clean way of > implementing this feature through an extension, unless some new callback, > e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to > improve this draft? > One can test the feature by applying the patch, building Maven, and running > tests with the modules in a random order like this: > $ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean > install -DskipTests > $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d > maven* | shuf) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (MNG-6265) Controlling reactor order of modules
[ https://issues.apache.org/jira/browse/MNG-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] August Shi updated MNG-6265: Description: We have a use case for controlling the reactor order in Maven, specifically for running tests on Travis. The common pattern is to run "mvn install -DskipTests; mvn test". After modules are installed (in the default order), the modules can be run in any order during testing, e.g., running the failed modules first, as in -Dsurefire.runOrder=failedfirst for Surefire. Would this feature be useful for others? I am attaching a *draft* patch that implements this feature by modifying DefaultMaven.java. I see no clean way of implementing this feature through an extension, unless some new callback, e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to improve this draft? One can test the feature by applying the patch, building Maven, and running tests with the modules in a random order like this: $ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean install -DskipTests $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d maven* | shuf) was: We have a use case for controlling the reactor order in Maven, specifically for running tests on Travis. The common pattern is to run "mvn install -DskipTests; mvn test". After modules are installed (in the default order), the modules can be run in any order during testing, e.g., running the failed modules first, as in -Dsurefire.runOrder=failedfirst for Surefire. Would this feature be useful for others? I am attaching a *draft* patch that implements this feature by modifying DefaultMaven.java. I see no clean way of implementing this feature through an extension, unless some new callback, e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to improve this draft? One can test the feature by applying the patch, building Maven, and running tests with the modules in a random order like this: $ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean install -DskipTests $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d maven-* | shuf) > Controlling reactor order of modules > > > Key: MNG-6265 > URL: https://issues.apache.org/jira/browse/MNG-6265 > Project: Maven > Issue Type: New Feature > Components: core >Affects Versions: 3.5.0 >Reporter: August Shi > Attachments: p.patch > > > We have a use case for controlling the reactor order in Maven, specifically > for running tests on Travis. The common pattern is to run "mvn install > -DskipTests; mvn test". After modules are installed (in the default order), > the modules can be run in any order during testing, e.g., running the failed > modules first, as in -Dsurefire.runOrder=failedfirst for Surefire. > Would this feature be useful for others? I am attaching a *draft* patch that > implements this feature by modifying DefaultMaven.java. I see no clean way of > implementing this feature through an extension, unless some new callback, > e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to > improve this draft? > One can test the feature by applying the patch, building Maven, and running > tests with the modules in a random order like this: > $ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean > install -DskipTests > $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d > maven* | shuf) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (MNG-6265) Controlling reactor order of modules
[
https://issues.apache.org/jira/browse/MNG-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
August Shi updated MNG-6265:
Description:
We have a use case for controlling the reactor order in Maven, specifically for
running tests on Travis. The common pattern is to run "mvn install -DskipTests;
mvn test". After modules are installed (in the default order), the modules can
be run in any order during testing, e.g., running the failed modules first, as
in -Dsurefire.runOrder=failedfirst for Surefire.
Would this feature be useful for others? I am attaching a *draft* patch that
implements this feature by modifying DefaultMaven.java. I see no clean way of
implementing this feature through an extension, unless some new callback, e.g.,
afterProjectsOrdered, is implemented first. Would it be worthwhile to improve
this draft?
One can test the feature by applying the patch, building Maven, and running
tests with the modules in a random order like this:
$ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean install
-DskipTests
$ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d
maven-* | shuf)
was:
We have a use case for controlling the reactor order in Maven, specifically for
running tests on Travis. The common pattern is to run "mvn install -DskipTests;
mvn test". After modules are installed (in the default order), the modules can
be run in any order during testing, e.g., running the failed modules first, as
in -Dsurefire.runOrder=failedfirst for Surefire.
Would this feature be useful for others? I am attaching a *draft* patch that
implements this feature by modifying DefaultMaven.java. I see no clean way of
implementing this feature through an extension, unless some new callback, e.g.,
afterProjectsOrdered, is implemented first. Would it be worthwhile to improve
this draft?
One can test the feature by applying the patch, building Maven, and running
tests with the modules in a random order like this:
{{$ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean
install -DskipTests
$ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d
maven-* | shuf)}}
> Controlling reactor order of modules
>
>
> Key: MNG-6265
> URL: https://issues.apache.org/jira/browse/MNG-6265
> Project: Maven
> Issue Type: New Feature
> Components: core
>Affects Versions: 3.5.0
>Reporter: August Shi
> Attachments: p.patch
>
>
> We have a use case for controlling the reactor order in Maven, specifically
> for running tests on Travis. The common pattern is to run "mvn install
> -DskipTests; mvn test". After modules are installed (in the default order),
> the modules can be run in any order during testing, e.g., running the failed
> modules first, as in -Dsurefire.runOrder=failedfirst for Surefire.
> Would this feature be useful for others? I am attaching a *draft* patch that
> implements this feature by modifying DefaultMaven.java. I see no clean way of
> implementing this feature through an extension, unless some new callback,
> e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to
> improve this draft?
> One can test the feature by applying the patch, building Maven, and running
> tests with the modules in a random order like this:
> $ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean
> install -DskipTests
> $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d
> maven-* | shuf)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (MNG-6265) Controlling reactor order of modules
[
https://issues.apache.org/jira/browse/MNG-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
August Shi updated MNG-6265:
Description:
We have a use case for controlling the reactor order in Maven, specifically for
running tests on Travis. The common pattern is to run "mvn install -DskipTests;
mvn test". After modules are installed (in the default order), the modules can
be run in any order during testing, e.g., running the failed modules first, as
in -Dsurefire.runOrder=failedfirst for Surefire.
Would this feature be useful for others? I am attaching a *draft* patch that
implements this feature by modifying DefaultMaven.java. I see no clean way of
implementing this feature through an extension, unless some new callback, e.g.,
afterProjectsOrdered, is implemented first. Would it be worthwhile to improve
this draft?
One can test the feature by applying the patch, building Maven, and running
tests with the modules in a random order like this:
{{$ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean
install -DskipTests
$ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d
maven-* | shuf)}}
was:
We have a use case for controlling the reactor order in Maven, specifically for
running tests on Travis. The common pattern is to run "mvn install -DskipTests;
mvn test". After modules are installed (in the default order), the modules can
be run in any order during testing, e.g., running the failed modules first, as
in -Dsurefire.runOrder=failedfirst for Surefire.
Would this feature be useful for others? I am attaching a *draft* patch that
implements this feature by modifying DefaultMaven.java. I see no clean way of
implementing this feature through an extension, unless some new callback, e.g.,
afterProjectsOrdered, is implemented first. Would it be worthwhile to improve
this draft?
One can test the feature by applying the patch, building Maven, and running
tests with the modules in a random order like this:
$ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean install
-DskipTests
$ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d
maven-* | shuf)
> Controlling reactor order of modules
>
>
> Key: MNG-6265
> URL: https://issues.apache.org/jira/browse/MNG-6265
> Project: Maven
> Issue Type: New Feature
> Components: core
>Affects Versions: 3.5.0
>Reporter: August Shi
> Attachments: p.patch
>
>
> We have a use case for controlling the reactor order in Maven, specifically
> for running tests on Travis. The common pattern is to run "mvn install
> -DskipTests; mvn test". After modules are installed (in the default order),
> the modules can be run in any order during testing, e.g., running the failed
> modules first, as in -Dsurefire.runOrder=failedfirst for Surefire.
> Would this feature be useful for others? I am attaching a *draft* patch that
> implements this feature by modifying DefaultMaven.java. I see no clean way of
> implementing this feature through an extension, unless some new callback,
> e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to
> improve this draft?
> One can test the feature by applying the patch, building Maven, and running
> tests with the modules in a random order like this:
> {{$ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean
> install -DskipTests
> $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d
> maven-* | shuf)}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (MNG-6265) Controlling reactor order of modules
August Shi created MNG-6265: --- Summary: Controlling reactor order of modules Key: MNG-6265 URL: https://issues.apache.org/jira/browse/MNG-6265 Project: Maven Issue Type: New Feature Components: core Affects Versions: 3.5.0 Reporter: August Shi Attachments: p.patch We have a use case for controlling the reactor order in Maven, specifically for running tests on Travis. The common pattern is to run "mvn install -DskipTests; mvn test". After modules are installed (in the default order), the modules can be run in any order during testing, e.g., running the failed modules first, as in -Dsurefire.runOrder=failedfirst for Surefire. Would this feature be useful for others? I am attaching a *draft* patch that implements this feature by modifying DefaultMaven.java. I see no clean way of implementing this feature through an extension, unless some new callback, e.g., afterProjectsOrdered, is implemented first. Would it be worthwhile to improve this draft? One can test the feature by applying the patch, building Maven, and running tests with the modules in a random order like this: $ mvn -DdistributionTargetDir=$(pwd)/apache-maven-3.5.x-SNAPSHOT clean install -DskipTests $ apache-maven-3.5.x-SNAPSHOT/bin/mvn test -Dmaven.project.order.file=<(ls -d maven-* | shuf) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
