[
https://issues.apache.org/jira/browse/MNG-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14962106#comment-14962106
]
Alexander Ashitkin edited comment on MNG-5750 at 10/17/15 10:49 PM:
--------------------------------------------------------------------
Could you please share your vision on following place
(org/apache/maven/lifecycle/internal/BuildListCalculator.java:60):
{code}
for ( MavenProject project : projects )
{
BuilderCommon.attachToThread( project );
MavenSession copiedSession = session.clone(); // suspicious
place
List<MavenProject> copiedSessionProjects =
copiedSession.getProjects();
{code}
Could be here an intention for deep copy? Because debug shows althougs the
session is being cloned, all created sessions share original projects. if some
session mutates somehow project - it actually does it globally. that could
potentially result in the issue and explains why my fix worked.
I modified code like this:
{code}
for ( MavenProject project : projects )
{
BuilderCommon.attachToThread( project ); // Not totally sure if
this is needed for anything
MavenSession copiedSession = session.clone();
List<MavenProject> copiedSessionProjects =
copiedSession.getProjects();
for ( int i = 0; i < copiedSessionProjects.size(); i++ )
{
MavenProject mavenProject = copiedSessionProjects.get( i );
MavenProject original = session.getProjects().get( i );
System.err.println( "Cloned session: " +
System.identityHashCode( copiedSession )
+ ", project "
+ mavenProject.getArtifactId() + " shared: " + (
mavenProject == original ) );
}
{code}
which gives following output on maven project itself:
{code}
c:\dev\maven-project>mvn clean instal-T 8 -q
Cloned session: 234740890, project maven shared: true
Cloned session: 234740890, project maven-modeshared: true
Cloned session: 234740890, project maven-artifact shared: true
Cloned session: 234740890, project maven-plugin-apshared: true
Cloned session: 234740890, project maven-builder-support shared: true
Cloned session: 234740890, project maven-model-builder shared: true
Cloned session: 234740890, project maven-settingshared: true
Cloned session: 234740890, project maven-settings-builder shared: true
Cloned session: 234740890, project maven-repository-metadatshared: true
Cloned session: 234740890, project maven-aether-provider shared: true
Cloned session: 234740890, project maven-corshared: true
Cloned session: 234740890, project maven-compat shared: true
Cloned session: 234740890, project maven-embedder shared: true
Cloned session: 234740890, project apache-maven shared: true
Cloned session: 79290250, project maven shared: true
Cloned session: 79290250, project maven-modeshared: true
Cloned session: 79290250, project maven-artifact shared: true
Cloned session: 79290250, project maven-plugin-apshared: true
Cloned session: 79290250, project maven-builder-support shared: true
Cloned session: 79290250, project maven-model-builder shared: true
Cloned session: 79290250, project maven-settingshared: true
Cloned session: 79290250, project maven-settings-builder shared: true
Cloned session: 79290250, project maven-repository-metadatshared: true
Cloned session: 79290250, project maven-aether-provider shared: true
Cloned session: 79290250, project maven-corshared: true
Cloned session: 79290250, project maven-compat shared: true
Cloned session: 79290250, project maven-embedder shared: true
Cloned session: 79290250, project apache-maven shared: true
Cloned session: 1586151649, project maven shared: true
Cloned session: 1586151649, project maven-modeshared: true
Cloned session: 1586151649, project maven-artifact shared: true
Cloned session: 1586151649, project maven-plugin-apshared: true
Cloned session: 1586151649, project maven-builder-support shared: true
Cloned session: 1586151649, project maven-model-builder shared: true
Cloned session: 1586151649, project maven-settingshared: true
Cloned session: 1586151649, project maven-settings-builder shared: true
Cloned session: 1586151649, project maven-repository-metadatshared: true
Cloned session: 1586151649, project maven-aether-provider shared: true
Cloned session: 1586151649, project maven-corshared: true
Cloned session: 1586151649, project maven-compat shared: true
Cloned session: 1586151649, project maven-embedder shared: true
Cloned session: 1586151649, project apache-maven shared: true
Cloned session: 179294202, project maven shared: true
.....
{code}
please share your thoughts
thanks in advance, Alexander
was (Author: alexander ashitkin):
Could you please share your vision on following place
(org/apache/maven/lifecycle/internal/BuildListCalculator.java:60):
{code}
for ( MavenProject project : projects )
{
BuilderCommon.attachToThread( project );
MavenSession copiedSession = session.clone(); // suspicious
place
List<MavenProject> copiedSessionProjects =
copiedSession.getProjects();
{code}
Could be here an intention for deep copy? Because debug shows althougs the
session is being cloned, all created session share original projects. if some
session mutates somehow project - it actually does it globally. that could
potentially result in the issue and explains why my fix worked.
I modified code like this:
{code}
for ( MavenProject project : projects )
{
BuilderCommon.attachToThread( project ); // Not totally sure if
this is needed for anything
MavenSession copiedSession = session.clone();
List<MavenProject> copiedSessionProjects =
copiedSession.getProjects();
for ( int i = 0; i < copiedSessionProjects.size(); i++ )
{
MavenProject mavenProject = copiedSessionProjects.get( i );
MavenProject original = session.getProjects().get( i );
System.err.println( "Cloned session: " +
System.identityHashCode( copiedSession )
+ ", project "
+ mavenProject.getArtifactId() + " shared: " + (
mavenProject == original ) );
}
{code}
which gives following output on maven project itself:
{code}
c:\dev\maven-project>mvn clean instal-T 8 -q
Cloned session: 234740890, project maven shared: true
Cloned session: 234740890, project maven-modeshared: true
Cloned session: 234740890, project maven-artifact shared: true
Cloned session: 234740890, project maven-plugin-apshared: true
Cloned session: 234740890, project maven-builder-support shared: true
Cloned session: 234740890, project maven-model-builder shared: true
Cloned session: 234740890, project maven-settingshared: true
Cloned session: 234740890, project maven-settings-builder shared: true
Cloned session: 234740890, project maven-repository-metadatshared: true
Cloned session: 234740890, project maven-aether-provider shared: true
Cloned session: 234740890, project maven-corshared: true
Cloned session: 234740890, project maven-compat shared: true
Cloned session: 234740890, project maven-embedder shared: true
Cloned session: 234740890, project apache-maven shared: true
Cloned session: 79290250, project maven shared: true
Cloned session: 79290250, project maven-modeshared: true
Cloned session: 79290250, project maven-artifact shared: true
Cloned session: 79290250, project maven-plugin-apshared: true
Cloned session: 79290250, project maven-builder-support shared: true
Cloned session: 79290250, project maven-model-builder shared: true
Cloned session: 79290250, project maven-settingshared: true
Cloned session: 79290250, project maven-settings-builder shared: true
Cloned session: 79290250, project maven-repository-metadatshared: true
Cloned session: 79290250, project maven-aether-provider shared: true
Cloned session: 79290250, project maven-corshared: true
Cloned session: 79290250, project maven-compat shared: true
Cloned session: 79290250, project maven-embedder shared: true
Cloned session: 79290250, project apache-maven shared: true
Cloned session: 1586151649, project maven shared: true
Cloned session: 1586151649, project maven-modeshared: true
Cloned session: 1586151649, project maven-artifact shared: true
Cloned session: 1586151649, project maven-plugin-apshared: true
Cloned session: 1586151649, project maven-builder-support shared: true
Cloned session: 1586151649, project maven-model-builder shared: true
Cloned session: 1586151649, project maven-settingshared: true
Cloned session: 1586151649, project maven-settings-builder shared: true
Cloned session: 1586151649, project maven-repository-metadatshared: true
Cloned session: 1586151649, project maven-aether-provider shared: true
Cloned session: 1586151649, project maven-corshared: true
Cloned session: 1586151649, project maven-compat shared: true
Cloned session: 1586151649, project maven-embedder shared: true
Cloned session: 1586151649, project apache-maven shared: true
Cloned session: 179294202, project maven shared: true
.....
{code}
please share your thoughts
thanks in advance, Alexander
> Sporadic failures in concurrent build
> -------------------------------------
>
> Key: MNG-5750
> URL: https://issues.apache.org/jira/browse/MNG-5750
> Project: Maven
> Issue Type: Bug
> Components: General
> Affects Versions: 3.1.1, 3.2.1, 3.2.2, 3.2.5
> Environment: SLES 3.0.80-0.7-default SMP x86_64 GNU/Linux Oracle
> HotSpot JDK 7u25
> windows server 2008 x64 Oracle HotSpot JDK 7u65/8u25
> Reporter: Alexander Ashitkin
> Priority: Critical
>
> We have a large project of 300+ modules which regularly fails with different
> kind of errors in different places. The issue is reliably reproduced with
> parallel build and is not happens in single threaded. The optimal concurrency
> level for our project ~10 threads. At this level ~%20 of builds fail. To
> workaround the issue we reduced concurrency to 4 in development builds and
> reverted production build to 1 thread.
> Main point of failures:
> # Surefire ClassNotFound. Reported and investigated in SUREFIRE-1132. Points
> to a problem with MavenProject#getArtifacts - empty set unexpectedly returned.
> # Compiler - unexpected failure because of incorrect classpath (literally all
> dependencies are not on the classpath), like: {code}
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[3,30] package ...
> does not exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[4,30] package ...
> does not exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[8,25] package ...
> does not exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[9,21] package
> org.joda.time does not exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[11,37] package
> com.google.common.base does not exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[11,1] static
> import only from classes and interfaces
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[12,37] package
> com.google.common.base does not exist
> 20:20:54 [ERROR] /D:/jenkins/work/workspace/..Request.java:[12,1] static
> import only from classes and interfaces
> {code}
> # Jar - unexpected NPE. Reported with stack traces in MJAR-192. (assembly
> plusgin seems to be also affected)
> At this point the issue looks like problem with MavenProject#getArtifacts in
> concurrent builds.
> To help with the issue im happy to implement or evaluate any custom assembly
> to trace this down. Unfortunately i cannot submit my project - it is
> proprietary.
> Thanks in advance, Alexander
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
