[
https://jira.codehaus.org/browse/MNG-5315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=326367#comment-326367
]
Sandy McPherson edited comment on MNG-5315 at 6/7/13 2:21 PM:
--------------------------------------------------------------
After a few false starts I found a plausible solution.
I Patched to only allow one thread at a time to do dependency resolution. I
tried to do this with finer granularity (per artifact) but failed miserably due
to the program structure. However the performance gain we have achieved in the
builds we were targeting was one order of magnitude!
{code}
DefaultProjectDependenciesResolver
public DependencyResolutionResult resolve(
DependencyResolutionRequest request)
throws DependencyResolutionException {
long start = System.currentTimeMillis();
System.out.println("0 **** locking
"+DefaultProjectDependenciesResolver.class);
synchronized (DefaultProjectDependenciesResolver.class) {
System.out.println((System.currentTimeMillis()-start)+" **** locked
"+DefaultProjectDependenciesResolver.class);
RequestTrace trace = RequestTrace.newChild(null, request);
.
.
.
.
.
System.out.println((System.currentTimeMillis()-start)+" ****
unlocked "+DefaultProjectDependenciesResolver.class);
return result;
}
{code}
Results are now
No failures so far. Massive speed improvements on larger builds with similar
dependencies:
serial: 10-11m
-T 1C 1.5m
-T 2C 1.25m
-T 3C nearly 1m
was (Author: sandymcp):
After a few false starts I found a plausible solution.
I Patched to only allow one thread at a time to do dependency resolution. I
tried to do this with finer granularity (per artifact) but failed miserably due
to the program structure. However the performance gain we have achieved in the
builds we were targeting was one order of magnitude!
DefaultProjectDependenciesResolver
public DependencyResolutionResult resolve(
DependencyResolutionRequest request)
throws DependencyResolutionException {
long start = System.currentTimeMillis();
System.out.println("0 **** locking
"+DefaultProjectDependenciesResolver.class);
synchronized (DefaultProjectDependenciesResolver.class) {
System.out.println((System.currentTimeMillis()-start)+" **** locked
"+DefaultProjectDependenciesResolver.class);
RequestTrace trace = RequestTrace.newChild(null, request);
.
.
.
.
.
System.out.println((System.currentTimeMillis()-start)+" ****
unlocked "+DefaultProjectDependenciesResolver.class);
return result;
}
Results are now
No failures so far. Massive speed improvements on larger builds with similar
dependencies:
serial: 10-11m
-T 1C 1.5m
-T 2C 1.25m
-T 3C nearly 1m
> Artifact resolution sporadically fails in parallel builds
> ---------------------------------------------------------
>
> Key: MNG-5315
> URL: https://jira.codehaus.org/browse/MNG-5315
> Project: Maven 2 & 3
> Issue Type: Bug
> Components: Artifacts and Repositories
> Affects Versions: 3.0.4
> Reporter: Ivan S. Dubrov
> Priority: Minor
>
> Artifact resolution can fail during the parallel build if it was downloaded
> during the same session.
> Scenario:
> 1) Delete the whole Maven local repository.
> 2) Run build "mvn compile -T1.5C"
> 3) Build fails (see log extracts below)
> 4) If I run build again, it succeeds.
> It seems like the problem is due to two thread trying to resolve same
> artifact concurrently. This problem never happens once I have all
> dependencies cached in the local repository.
> Extracts from the log output:
> {noformat}Downloading:
> http://nexus/content/repositories/thirdparty/com/googlecode/guava-osgi/guava-osgi/11.0.0/guava-osgi-11.0.0.jar
> 12444/13937 KB ...
> ...
> [DEBUG] Skipped remote update check for commons-cli:commons-cli:jar:1.2,
> already updated during this session.
> [DEBUG] Skipped remote update check for
> com.googlecode.guava-osgi:guava-osgi:jar:11.0.0, already updated during this
> session.
> [DEBUG] Skipped remote update check for org.slf4j:slf4j-api:jar:1.6.4,
> already updated during this session.
> [DEBUG] Skipped remote update check for org.slf4j:slf4j-jdk14:jar:1.6.4,
> already updated during this session.
> ...
> [ERROR] Failed to execute goal on project util: Could not resolve
> dependencies for project com.guidewire.pl:util:bundle:1.0-SNAPSHOT: The
> following artifacts could not be resolved: commons-cli:commons-cli:jar:1.2,
> com.googlecode.guava-osgi:guava-osgi:jar:11.0.0,
> org.slf4j:slf4j-api:jar:1.6.4, org.slf4j:slf4j-jdk14:jar:1.6.4: Failure to
> find commons-cli:commons-cli:jar:1.2 in
> http://nexus/content/repositories/thirdparty was cached in the local
> repository, resolution will not be reattempted until the update interval of
> gw.thirdparty has elapsed or updates are forced -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project util: Could not resolve dependencies for project
> com.guidewire.pl:util:bundle:1.0-SNAPSHOT: The following artifacts could not
> be resolved: commons-cli:commons-cli:jar:1.2,
> com.googlecode.guava-osgi:guava-osgi:jar:11.0.0,
> org.slf4j:slf4j-api:jar:1.6.4, org.slf4j:slf4j-jdk14:jar:1.6.4: Failure to
> find commons-cli:commons-cli:jar:1.2 in
> http://nexus/content/repositories/thirdparty was cached in the local
> repository, resolution will not be reattempted until the update interval of
> gw.thirdparty has elapsed or updates are forced
> at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:210)
> at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:117)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:258)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:201)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at
> org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:167)
> at
> org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:163)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.maven.project.DependencyResolutionException: Could not
> resolve dependencies for project com.guidewire.pl:util:bundle:1.0-SNAPSHOT:
> The following artifacts could not be resolved:
> commons-cli:commons-cli:jar:1.2,
> com.googlecode.guava-osgi:guava-osgi:jar:11.0.0,
> org.slf4j:slf4j-api:jar:1.6.4, org.slf4j:slf4j-jdk14:jar:1.6.4: Failure to
> find commons-cli:commons-cli:jar:1.2 in
> http://nexus/content/repositories/thirdparty was cached in the local
> repository, resolution will not be reattempted until the update interval of
> gw.thirdparty has elapsed or updates are forced
> at
> org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:189)
> at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:185)
> ... 16 more
> Caused by: org.sonatype.aether.resolution.DependencyResolutionException: The
> following artifacts could not be resolved: commons-cli:commons-cli:jar:1.2,
> com.googlecode.guava-osgi:guava-osgi:jar:11.0.0,
> org.slf4j:slf4j-api:jar:1.6.4, org.slf4j:slf4j-jdk14:jar:1.6.4: Failure to
> find commons-cli:commons-cli:jar:1.2 in
> http://nexus/content/repositories/thirdparty was cached in the local
> repository, resolution will not be reattempted until the update interval of
> gw.thirdparty has elapsed or updates are forced
> at
> org.sonatype.aether.impl.internal.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:375)
> at
> org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:183)
> ... 17 more
> Caused by: org.sonatype.aether.resolution.ArtifactResolutionException: The
> following artifacts could not be resolved: commons-cli:commons-cli:jar:1.2,
> com.googlecode.guava-osgi:guava-osgi:jar:11.0.0,
> org.slf4j:slf4j-api:jar:1.6.4, org.slf4j:slf4j-jdk14:jar:1.6.4: Failure to
> find commons-cli:commons-cli:jar:1.2 in
> http://nexus/content/repositories/thirdparty was cached in the local
> repository, resolution will not be reattempted until the update interval of
> gw.thirdparty has elapsed or updates are forced
> at
> org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:538)
> at
> org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:216)
> at
> org.sonatype.aether.impl.internal.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:358)
> ... 18 more
> Caused by: org.sonatype.aether.transfer.ArtifactNotFoundException: Failure to
> find commons-cli:commons-cli:jar:1.2 in
> http://nexus/content/repositories/thirdparty was cached in the local
> repository, resolution will not be reattempted until the update interval of
> gw.thirdparty has elapsed or updates are forced
> at
> org.sonatype.aether.impl.internal.DefaultUpdateCheckManager.newException(DefaultUpdateCheckManager.java:230)
> at
> org.sonatype.aether.impl.internal.DefaultUpdateCheckManager.checkArtifact(DefaultUpdateCheckManager.java:176)
> at
> org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:427)
> ... 20 more
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
