[ https://issues.apache.org/jira/browse/MDEP-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684909#comment-16684909 ]
James B commented on MDEP-518: ------------------------------ Confirmed fixed by using Takari's builder, which provides a thread-safe implementation of the maven local repo. [http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository] Simply place takari-smart-builder-VERSION.jar in your $MAVEN_HOME/lib/ext and invoke maven with "--builder smart" option. > Dependency resolution fails in threaded builds > ---------------------------------------------- > > Key: MDEP-518 > URL: https://issues.apache.org/jira/browse/MDEP-518 > Project: Maven Dependency Plugin > Issue Type: Bug > Components: resolve, resolve-plugins > Affects Versions: 2.10 > Environment: Linux > Reporter: Troy Telford > Priority: Major > > This issue sounds similar to MDEP-442, but it may be unrelated. > I suspect the same problem may occur in both {{dependency:resolve}} and > {{dependency:resolve-plugins}}. > The same error occurs when I use {{mvn install}}; but {{dependency:resolve}} > is a more tightly-focused problem report. > I have a large multi-module project, with more than 40 modules, and a > large-ish build node with 24 cores. {{mvn -T1C dependency:resolve > dependency:resolve-plugins}} will typically (94/100 attempts) fail to > download all of the dependencies required, with an error similar to: > {noformat} > Failed to read artifact descriptor for com.foo:machine:jar:1.2.3-SNAPSHOT: > Could not transfer artifact com.foo.somegroup:dependency_4:jar:3.0.0 from/to > snapshot-repository (https://nexus.foo.com/content/repositories/releases): > /home/userid/.m2/repository/com/foo/somegroup/dependency_4/3.0.0/dependency_4-3.0.0.pom.part > (No such file or directory) -> [Help 1] > -> com.foo.somegroup:dependency_4:jar:3.0.0 > at > org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:220) > at > org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:127) > at > org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:257) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:200) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) > at > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:188) > at > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:184) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:722) > Caused by: org.apache.maven.project.DependencyResolutionException: Could not > resolve dependencies for project com.foo:machine:jar:1.2.3-SNAPSHOT: Failed > to collect dependencies at com.foo:dependency_1:jar:1.2.3-SNAPSHOT -> > com.foo:dependency_2:jar:1.2.3-SNAPSHOT -> > com.foo:dependency_3:jar:1.2.3-SNAPSHOT -> > com.foo.somegroup:dependency_4:jar:3.0.0 > at > org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:180) > at > org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:195) > ... 16 more > Caused by: org.eclipse.aether.collection.DependencyCollectionException: > Failed to collect dependencies at com.foo:dependency_1:jar:1.2.3-SNAPSHOT -> > com.foo:dependency_2:jar:1.2.3-SNAPSHOT -> > com.foo:dependency_3:jar:1.2.3-SNAPSHOT -> > com.foo.somegroup:dependency_4:jar:3.0.0 > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:291) > at > org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies(DefaultRepositorySystem.java:316) > at > org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:172) > ... 17 more > Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed > to read artifact descriptor for com.foo.somegroup:dependency_4:jar:3.0.0 > at > org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:302) > at > org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.resolveCachedArtifactDescriptor(DefaultDependencyCollector.java:525) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.getArtifactDescriptorResult(DefaultDependencyCollector.java:509) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:409) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:494) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:494) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:494) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351) > at > org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:254) > ... 19 more > Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could > not transfer artifact com.foo.somegroup:dependency_4:pom:3.0.0 from/to > release-repository (https://nexus.foo.com/content/repositories/releases): > /home/userid/.m2/repository/com/foo/somegroup/dependency_4/3.0.0/dependency_4-3.0.0.pom.part > (No such file or directory) > at > org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444) > at > org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246) > at > org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223) > at > org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:287) > ... 38 more > Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not > transfer artifact com.foo.somegroup:dependency_4:pom:3.0.0 from/to > release-repository (https://nexus.foo.com/content/repositories/releases): > /home/userid/.m2/repository/com/foo/somegroup/dependency_4/3.0.0/dependency_4-3.0.0.pom.part > (No such file or directory) > at > org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:43) > at > org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:355) > at > org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67) > at > org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:581) > at > org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:249) > at > org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:520) > at > org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421) > ... 41 more > Caused by: java.io.FileNotFoundException: > /home/userid/.m2/repository/com/foo/somegroup/dependency_4/3.0.0/dependency_4-3.0.0.pom.part > (No such file or directory) > at java.io.FileInputStream.open(Native Method) > at java.io.FileInputStream.<init>(FileInputStream.java:138) > at > org.eclipse.aether.internal.impl.DefaultFileProcessor.copy(DefaultFileProcessor.java:148) > at > org.eclipse.aether.internal.impl.DefaultFileProcessor.copy(DefaultFileProcessor.java:136) > at > org.eclipse.aether.internal.impl.DefaultFileProcessor.move(DefaultFileProcessor.java:211) > at > org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:472) > at > org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:350) > ... 46 more > {noformat} > * Note the '{{.part}}' extension to the missing pomfile. I see the same > things with jars, wars, etc. '{{.part}}' is _always_ tacked on to the end of > the artifact filename. > * The dependency that Maven can't download -- and even the module that fails > -- changes from one run to the next. > * Subsequent builds attempts eventually download all dependencies to the > local Maven cache, after which there is no problem building. > Since I have to delete a substantial part of the local Maven cache for each > build, I can't really count on the Maven cache. > Using a single-thread build works fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)