[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[ https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776396#comment-17776396 ] Tamas Cservenak commented on MRESOLVER-412: --- Also, tested resolver + this PR https://github.com/apache/maven/pull/893 "Multi-threaded Model Builder" and all ITs pass OK, so I'd vote for this applied to maven core. This is not a resolver nor BF collector issue definitely. > BF collector causes same model to be loaded up multiple times > - > > Key: MRESOLVER-412 > URL: https://issues.apache.org/jira/browse/MRESOLVER-412 > Project: Maven Resolver > Issue Type: Bug > Components: Resolver >Reporter: Tamas Cservenak >Priority: Major > Fix For: 2.0.0 > > > This bug probably affects all released versions of new BF collector. > Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both > test methods. The IT verifies that POM XMLs are _read only once_ with and > without "consumer POM" feature enabled. Both IT methods fail (so failure is > irrelevant of consumer POM feature being enabled). Error consistently happens > on each subsequent IT invokcation, whether empty or prepopulated local repo. > Most probably cause: old DF was single threaded, while BF uses internally > thread pool, that seemingly causes that same POM XML is loaded up multiple > times as ModelBuilder is being asked multi threaded for different models by > BF collector. > Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable > DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default > is BF). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[
https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776304#comment-17776304
]
Tamas Cservenak commented on MRESOLVER-412:
---
It seems my suspicion was right: this is ModelBuilder issue, BF collector is to
"blame" only for invoking ModelBuilder on several threads :)
Locally was able to "fix" this issue by synchronizing the two method:
* org.apache.maven.model.building.DefaultModelBuilder#readRawModel
* org.apache.maven.model.building.DefaultModelBuilder#readFileModel
My guess is that what happens is that two competing threads both get {{null}}
from cache, then they both go to load/parse same (parent) POM, and finally both
put it into cache...
> BF collector causes same model to be loaded up multiple times
> -
>
> Key: MRESOLVER-412
> URL: https://issues.apache.org/jira/browse/MRESOLVER-412
> Project: Maven Resolver
> Issue Type: Bug
> Components: Resolver
>Reporter: Tamas Cservenak
>Priority: Major
> Fix For: 2.0.0
>
>
> This bug probably affects all released versions of new BF collector.
> Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both
> test methods. The IT verifies that POM XMLs are _read only once_ with and
> without "consumer POM" feature enabled. Both IT methods fail (so failure is
> irrelevant of consumer POM feature being enabled). Error consistently happens
> on each subsequent IT invokcation, whether empty or prepopulated local repo.
> Most probably cause: old DF was single threaded, while BF uses internally
> thread pool, that seemingly causes that same POM XML is loaded up multiple
> times as ModelBuilder is being asked multi threaded for different models by
> BF collector.
> Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable
> DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default
> is BF).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[ https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776276#comment-17776276 ] Tamas Cservenak commented on MRESOLVER-412: --- ModelBuilder is invoked implicitly, whenever this line is hit: https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.16/maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/collect/bf/BfDependencyCollector.java#L464 And if you imagine POMs that share parents (like Maven project or apache commons project, etc), this causes that _same parent_ is parsed on different threads but again, this is just my guess currently, and I must miss something, as there should be model cache in play as well, but something seems off (in Maven or ModelBuilder maybe) > BF collector causes same model to be loaded up multiple times > - > > Key: MRESOLVER-412 > URL: https://issues.apache.org/jira/browse/MRESOLVER-412 > Project: Maven Resolver > Issue Type: Bug > Components: Resolver >Reporter: Tamas Cservenak >Priority: Major > Fix For: 2.0.0 > > > This bug probably affects all released versions of new BF collector. > Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both > test methods. The IT verifies that POM XMLs are _read only once_ with and > without "consumer POM" feature enabled. Both IT methods fail (so failure is > irrelevant of consumer POM feature being enabled). Error consistently happens > on each subsequent IT invokcation, whether empty or prepopulated local repo. > Most probably cause: old DF was single threaded, while BF uses internally > thread pool, that seemingly causes that same POM XML is loaded up multiple > times as ModelBuilder is being asked multi threaded for different models by > BF collector. > Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable > DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default > is BF). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[ https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776271#comment-17776271 ] Tamas Cservenak commented on MRESOLVER-412: --- [~wecai] the IT is introduced for maven4, yes. But the point is that this IT never run with BF before (as it did not alter default collector). Now that resolver 2.0.0 makes BF default collector (MRESOLVER-324), the IT fails. Also, my guess is that the IT would fail with all released versions of new BF collector. It MAY be that the IT is wrong, am still looking. > BF collector causes same model to be loaded up multiple times > - > > Key: MRESOLVER-412 > URL: https://issues.apache.org/jira/browse/MRESOLVER-412 > Project: Maven Resolver > Issue Type: Bug > Components: Resolver >Reporter: Tamas Cservenak >Priority: Major > Fix For: 2.0.0 > > > This bug probably affects all released versions of new BF collector. > Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both > test methods. The IT verifies that POM XMLs are _read only once_ with and > without "consumer POM" feature enabled. Both IT methods fail (so failure is > irrelevant of consumer POM feature being enabled). Error consistently happens > on each subsequent IT invokcation, whether empty or prepopulated local repo. > Most probably cause: old DF was single threaded, while BF uses internally > thread pool, that seemingly causes that same POM XML is loaded up multiple > times as ModelBuilder is being asked multi threaded for different models by > BF collector. > Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable > DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default > is BF). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[ https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775754#comment-17775754 ] wei cai commented on MRESOLVER-412: --- [~cstamas] [~gnodet] MNG-5669 fix is firstly introduced in maven 4.x, right? I did not see such issue when running ITs in maven 3.9.x. Where do you see ModelBuilder invoked on multiple threads? > BF collector causes same model to be loaded up multiple times > - > > Key: MRESOLVER-412 > URL: https://issues.apache.org/jira/browse/MRESOLVER-412 > Project: Maven Resolver > Issue Type: Bug > Components: Resolver >Reporter: Tamas Cservenak >Priority: Major > Fix For: 2.0.0 > > > This bug probably affects all released versions of new BF collector. > Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both > test methods. The IT verifies that POM XMLs are _read only once_ with and > without "consumer POM" feature enabled. Both IT methods fail (so failure is > irrelevant of consumer POM feature being enabled). Error consistently happens > on each subsequent IT invokcation, whether empty or prepopulated local repo. > Most probably cause: old DF was single threaded, while BF uses internally > thread pool, that seemingly causes that same POM XML is loaded up multiple > times as ModelBuilder is being asked multi threaded for different models by > BF collector. > Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable > DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default > is BF). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[
https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775501#comment-17775501
]
Tamas Cservenak commented on MRESOLVER-412:
---
CI reported this: different output (due concurrency?) but still parents only:
{noformat}
Error: Failures:
Error:
MavenITmng5669ReadPomsOnce.testWithBuildConsumer:114->AbstractMavenIntegrationTestCase.assertTrue:359
Duplicate items: org.apache.maven:maven:3.2.5
org.apache.commons:commons-parent:52
org.apache.maven:maven-parent:25 ==> expected: but was:
Error:
MavenITmng5669ReadPomsOnce.testWithoutBuildConsumer:78->AbstractMavenIntegrationTestCase.assertTrue:359
Duplicate items: org.codehaus.plexus:plexus:3.3.1
org.codehaus.plexus:plexus:10
org.sonatype.spice:spice-parent:17 ==> expected: but was:
[INFO]
Error: Tests run: 899, Failures: 2, Errors: 0, Skipped: 34
{noformat}
> BF collector causes same model to be loaded up multiple times
> -
>
> Key: MRESOLVER-412
> URL: https://issues.apache.org/jira/browse/MRESOLVER-412
> Project: Maven Resolver
> Issue Type: Bug
> Components: Resolver
>Reporter: Tamas Cservenak
>Priority: Major
> Fix For: 2.0.0
>
>
> This bug probably affects all released versions of new BF collector.
> Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both
> test methods. The IT verifies that POM XMLs are _read only once_ with and
> without "consumer POM" feature enabled. Both IT methods fail (so failure is
> irrelevant of consumer POM feature being enabled). Error consistently happens
> on each subsequent IT invokcation, whether empty or prepopulated local repo.
> Most probably cause: old DF was single threaded, while BF uses internally
> thread pool, that seemingly causes that same POM XML is loaded up multiple
> times as ModelBuilder is being asked multi threaded for different models by
> BF collector.
> Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable
> DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default
> is BF).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[
https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775498#comment-17775498
]
Tamas Cservenak commented on MRESOLVER-412:
---
The both IT methods (w/ and w/o "consume POM" enabled) lists these models as
"duplicates" (twice loaded up):
{noformat}
org.apache.maven:maven:3.2.5
org.codehaus.plexus:plexus:3.3.1
org.eclipse.aether:aether:1.0.0.v20140518
org.apache.commons:commons-parent:52
org.codehaus.plexus:plexus:10
org.apache:apache:15
org.sonatype.forge:forge-parent:10
org.codehaus.plexus:plexus-components:6.6
org.apache.maven:maven-parent:25
org.codehaus.plexus:plexus-compiler:2.11.1
{noformat}
And as I see, these are all parents...
> BF collector causes same model to be loaded up multiple times
> -
>
> Key: MRESOLVER-412
> URL: https://issues.apache.org/jira/browse/MRESOLVER-412
> Project: Maven Resolver
> Issue Type: Bug
> Components: Resolver
>Reporter: Tamas Cservenak
>Priority: Major
> Fix For: 2.0.0
>
>
> This bug probably affects all released versions of new BF collector.
> Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both
> test methods. The IT verifies that POM XMLs are _read only once_ with and
> without "consumer POM" feature enabled. Both IT methods fail (so failure is
> irrelevant of consumer POM feature being enabled). Error consistently happens
> on each subsequent IT invokcation, whether empty or prepopulated local repo.
> Most probably cause: old DF was single threaded, while BF uses internally
> thread pool, that seemingly causes that same POM XML is loaded up multiple
> times as ModelBuilder is being asked multi threaded for different models by
> BF collector.
> Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable
> DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default
> is BF).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[ https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775497#comment-17775497 ] Tamas Cservenak commented on MRESOLVER-412: --- I think this is not (directly) BF, but ModelBuilder invoked on multiple threads, like: * T1 "give me asf.P1 model"... has parent asf.parent1 * T2 "give me asf.P2 model"... has parent asf.parent1 So it is not project POMs that are (re)loaded but parents as I see... > BF collector causes same model to be loaded up multiple times > - > > Key: MRESOLVER-412 > URL: https://issues.apache.org/jira/browse/MRESOLVER-412 > Project: Maven Resolver > Issue Type: Bug > Components: Resolver >Reporter: Tamas Cservenak >Priority: Major > Fix For: 2.0.0 > > > This bug probably affects all released versions of new BF collector. > Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both > test methods. The IT verifies that POM XMLs are _read only once_ with and > without "consumer POM" feature enabled. Both IT methods fail (so failure is > irrelevant of consumer POM feature being enabled). Error consistently happens > on each subsequent IT invokcation, whether empty or prepopulated local repo. > Most probably cause: old DF was single threaded, while BF uses internally > thread pool, that seemingly causes that same POM XML is loaded up multiple > times as ModelBuilder is being asked multi threaded for different models by > BF collector. > Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable > DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default > is BF). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[
https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775402#comment-17775402
]
Guillaume Nodet commented on MRESOLVER-412:
---
Shouldn't BF use a {{ForkJoinPool}} somehow and when there's a need to read a
POM, create a job for it, post it to the pool and wait for the result ? This
would avoid reading POMs multiple times in theory.
> BF collector causes same model to be loaded up multiple times
> -
>
> Key: MRESOLVER-412
> URL: https://issues.apache.org/jira/browse/MRESOLVER-412
> Project: Maven Resolver
> Issue Type: Bug
> Components: Resolver
>Reporter: Tamas Cservenak
>Priority: Major
> Fix For: 2.0.0
>
>
> This bug probably affects all released versions of new BF collector.
> Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both
> test methods. The IT verifies that POM XMLs are _read only once_ with and
> without "consumer POM" feature enabled. Both IT methods fail (so failure is
> irrelevant of consumer POM feature being enabled). Error consistently happens
> on each subsequent IT invokcation, whether empty or prepopulated local repo.
> Most probably cause: old DF was single threaded, while BF uses internally
> thread pool, that seemingly causes that same POM XML is loaded up multiple
> times as ModelBuilder is being asked multi threaded for different models by
> BF collector.
> Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable
> DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default
> is BF).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MRESOLVER-412) BF collector causes same model to be loaded up multiple times
[ https://issues.apache.org/jira/browse/MRESOLVER-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775382#comment-17775382 ] Tamas Cservenak commented on MRESOLVER-412: --- [~wecai] ping > BF collector causes same model to be loaded up multiple times > - > > Key: MRESOLVER-412 > URL: https://issues.apache.org/jira/browse/MRESOLVER-412 > Project: Maven Resolver > Issue Type: Bug > Components: Resolver >Reporter: Tamas Cservenak >Priority: Major > Fix For: 2.0.0 > > > This bug probably affects all released versions of new BF collector. > Symptom is that Maven with BF collector fails MavenITmng5669ReadPomsOnce both > test methods. The IT verifies that POM XMLs are _read only once_ with and > without "consumer POM" feature enabled. Both IT methods fail (so failure is > irrelevant of consumer POM feature being enabled). > Most probably cause: old DF was single threaded, while BF uses internally > thread pool, that seemingly causes that same POM XML is loaded up multiple > times as ModelBuilder is being asked multi threaded for different models by > BF collector. > Reproduce: either run MavenITmng5669ReadPomsOnce but modify the IT to enable > DF (if using Maven w/ resolver 1.x) or build Maven 4 w/ Resolver 2 (default > is BF). -- This message was sent by Atlassian Jira (v8.20.10#820010)
