[jira] [Commented] (MNG-7615) Multithreaded model builder
[ https://issues.apache.org/jira/browse/MNG-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783528#comment-17783528 ] ASF GitHub Bot commented on MNG-7615: - gnodet merged PR #893: URL: https://github.com/apache/maven/pull/893 > Multithreaded model builder > --- > > Key: MNG-7615 > URL: https://issues.apache.org/jira/browse/MNG-7615 > Project: Maven > Issue Type: Task >Reporter: Guillaume Nodet >Assignee: Guillaume Nodet >Priority: Major > > Parsing all reactor models can be very lengthy, so use a map/reduce algorithm > to make the computation in parallel. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7615) Multithreaded model builder
[ https://issues.apache.org/jira/browse/MNG-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776392#comment-17776392 ] ASF GitHub Bot commented on MNG-7615: - cstamas commented on PR #893: URL: https://github.com/apache/maven/pull/893#issuecomment-1767179898 Ran ITs of this PR + resolver 2.0.0 (both combined) and ITs are OK > Multithreaded model builder > --- > > Key: MNG-7615 > URL: https://issues.apache.org/jira/browse/MNG-7615 > Project: Maven > Issue Type: Task >Reporter: Guillaume Nodet >Assignee: Guillaume Nodet >Priority: Major > > Parsing all reactor models can be very lengthy, so use a map/reduce algorithm > to make the computation in parallel. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7615) Multithreaded model builder
[ https://issues.apache.org/jira/browse/MNG-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645502#comment-17645502 ] ASF GitHub Bot commented on MNG-7615: - gnodet commented on PR #893: URL: https://github.com/apache/maven/pull/893#issuecomment-1344854328 So the PR works with 1 one thread here, but may deadlock irrespective on the number of threads. The deadlock happen in the following scenario : 3 projects P1, P2, P3, P3 depends on P2, P2 depends on P1. The three projects are submitted to the FJ pool, which is able to steal work, so that jobs may be executed without any particular order. So P2 is being executed, which requires P1. P1 is being joined by the FJ, but it decides to execute P3. P3 is being executed in the same thread and then waits for P2 being finished which can't happen. > Multithreaded model builder > --- > > Key: MNG-7615 > URL: https://issues.apache.org/jira/browse/MNG-7615 > Project: Maven > Issue Type: Task >Reporter: Guillaume Nodet >Assignee: Guillaume Nodet >Priority: Major > > Parsing all reactor models can be very lengthy, so use a map/reduce algorithm > to make the computation in parallel. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7615) Multithreaded model builder
[ https://issues.apache.org/jira/browse/MNG-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642871#comment-17642871 ] ASF GitHub Bot commented on MNG-7615: - cstamas commented on PR #893: URL: https://github.com/apache/maven/pull/893#issuecomment-1336197409 Those two runs above were Java 11. Switched to Java 17, first run was OK, but then it deadlocked again: https://gist.github.com/cstamas/c44475bc3231363bc159d1a1d52f27c0 > Multithreaded model builder > --- > > Key: MNG-7615 > URL: https://issues.apache.org/jira/browse/MNG-7615 > Project: Maven > Issue Type: Task >Reporter: Guillaume Nodet >Assignee: Guillaume Nodet >Priority: Major > > Parsing all reactor models can be very lengthy, so use a map/reduce algorithm > to make the computation in parallel. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7615) Multithreaded model builder
[ https://issues.apache.org/jira/browse/MNG-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642869#comment-17642869 ] ASF GitHub Bot commented on MNG-7615: - cstamas commented on PR #893: URL: https://github.com/apache/maven/pull/893#issuecomment-1336196855 Deadlock seems consistent, Ctrl-C after 6 minutes previous build, re-run and again: https://gist.github.com/cstamas/62818af78abb293088cbb9383a278285 > Multithreaded model builder > --- > > Key: MNG-7615 > URL: https://issues.apache.org/jira/browse/MNG-7615 > Project: Maven > Issue Type: Task >Reporter: Guillaume Nodet >Assignee: Guillaume Nodet >Priority: Major > > Parsing all reactor models can be very lengthy, so use a map/reduce algorithm > to make the computation in parallel. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7615) Multithreaded model builder
[
https://issues.apache.org/jira/browse/MNG-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642868#comment-17642868
]
ASF GitHub Bot commented on MNG-7615:
-
gnodet commented on code in PR #893:
URL: https://github.com/apache/maven/pull/893#discussion_r1038806942
##
maven-model-builder/src/main/java/org/apache/maven/model/superpom/DefaultSuperPomProvider.java:
##
@@ -52,8 +53,8 @@ public DefaultSuperPomProvider(ModelProcessor modelProcessor)
{
@Override
public Model getSuperModel(String version) {
-if (superModel == null) {
-String resource = "/org/apache/maven/model/pom-" + version +
".xml";
+return SUPER_MODELS.computeIfAbsent(version, v -> {
Review Comment:
It could, though the only place it's called is with `4.0.0` afaik, and
that's since ages. I found the code does not reflect the javadoc and that the
`version` parameter is not used, so I fixed it.
If that's not intended, then we should remove the parameter completely.
The version was originally hardcoded, see
https://github.com/apache/maven/commit/dd635ca490bae271442d7b69eafdf595a627cafd
The commit added a version parameter, but it's only used the first time and
subsequent calls return the cached version.
As for having multiple superpoms, I think the problem is that the version
has never been updated when the pom changed. So not sure what the future will
be for the supermom...
> Multithreaded model builder
> ---
>
> Key: MNG-7615
> URL: https://issues.apache.org/jira/browse/MNG-7615
> Project: Maven
> Issue Type: Task
>Reporter: Guillaume Nodet
>Assignee: Guillaume Nodet
>Priority: Major
>
> Parsing all reactor models can be very lengthy, so use a map/reduce algorithm
> to make the computation in parallel.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MNG-7615) Multithreaded model builder
[ https://issues.apache.org/jira/browse/MNG-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642867#comment-17642867 ] ASF GitHub Bot commented on MNG-7615: - cstamas commented on PR #893: URL: https://github.com/apache/maven/pull/893#issuecomment-1336195768 Tried to test drive it, seems it deadlocked (console stopped at line 12): https://gist.github.com/cstamas/d54715ffc24d7513e735353d17d8f78b > Multithreaded model builder > --- > > Key: MNG-7615 > URL: https://issues.apache.org/jira/browse/MNG-7615 > Project: Maven > Issue Type: Task >Reporter: Guillaume Nodet >Assignee: Guillaume Nodet >Priority: Major > > Parsing all reactor models can be very lengthy, so use a map/reduce algorithm > to make the computation in parallel. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MNG-7615) Multithreaded model builder
[
https://issues.apache.org/jira/browse/MNG-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642854#comment-17642854
]
ASF GitHub Bot commented on MNG-7615:
-
cstamas commented on code in PR #893:
URL: https://github.com/apache/maven/pull/893#discussion_r1038801109
##
maven-model-builder/src/main/java/org/apache/maven/model/superpom/DefaultSuperPomProvider.java:
##
@@ -52,8 +53,8 @@ public DefaultSuperPomProvider(ModelProcessor modelProcessor)
{
@Override
public Model getSuperModel(String version) {
-if (superModel == null) {
-String resource = "/org/apache/maven/model/pom-" + version +
".xml";
+return SUPER_MODELS.computeIfAbsent(version, v -> {
Review Comment:
Does this mean that for example maven 4.0.1 could return super POM of Maven
3.8.6 (ie. newer could have all super POMs of older Mavens?)
> Multithreaded model builder
> ---
>
> Key: MNG-7615
> URL: https://issues.apache.org/jira/browse/MNG-7615
> Project: Maven
> Issue Type: Task
>Reporter: Guillaume Nodet
>Assignee: Guillaume Nodet
>Priority: Major
>
> Parsing all reactor models can be very lengthy, so use a map/reduce algorithm
> to make the computation in parallel.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MNG-7615) Multithreaded model builder
[ https://issues.apache.org/jira/browse/MNG-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642372#comment-17642372 ] ASF GitHub Bot commented on MNG-7615: - gnodet opened a new pull request, #893: URL: https://github.com/apache/maven/pull/893 Following this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/browse/MNG) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[MNG-XXX] SUMMARY`, where you replace `MNG-XXX` and `SUMMARY` with the appropriate JIRA issue. Best practice is to use the JIRA issue title in the pull request title and in the first line of the commit message. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] You have run the [Core IT][core-its] successfully. If your pull request is about ~20 lines of code you don't need to sign an [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf) if you are unsure please ask on the developers list. To make clear that you license your contribution under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) you have to acknowledge this by using the following check-box. - [ ] I hereby declare this contribution to be licenced under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) - [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). [core-its]: https://maven.apache.org/core-its/core-it-suite/ > Multithreaded model builder > --- > > Key: MNG-7615 > URL: https://issues.apache.org/jira/browse/MNG-7615 > Project: Maven > Issue Type: Task >Reporter: Guillaume Nodet >Assignee: Guillaume Nodet >Priority: Major > > Parsing all reactor models can be very lengthy, so use a map/reduce algorithm > to make the computation in parallel. -- This message was sent by Atlassian Jira (v8.20.10#820010)
