Hi Stuart,
The synchronized let me see what was happening as far as ordering was
concerned. It appears that the contextualize() call races with calls to
buildDependencyGraph() and that that Container is not performing lifecycle
operations on the DefaultDependencyGraphBuilder object before consumers of
that object invoke methods.
I think the maven-bundle-plugin needs to have @threadSafe = false since
it's not thread safe due its dependencies not being thread safe. I don't
know enough about Maven and Plexus to force some sort of explicit ordering
in the bean's lifecycle.
However, this patch does seem to fix the underlying issue though it could
be subject to a dead lock if all of the build threads wait on the latch.
The real solution is for the container to properly follow concurrent
lifecycle rules.
===================================================================
---
src/main/java/org/apache/maven/shared/dependency/graph/internal/DefaultDependencyGraphBuilder.java
(revision 1726911)
+++
src/main/java/org/apache/maven/shared/dependency/graph/internal/DefaultDependencyGraphBuilder.java
(working copy)
@@ -1,5 +1,9 @@
package org.apache.maven.shared.dependency.graph.internal;
+import java.util.Collection;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
@@ -33,9 +37,6 @@
import org.codehaus.plexus.logging.AbstractLogEnabled;
import
org.codehaus.plexus.personality.plexus.lifecycle.phase.Contextualizable;
-import java.util.Collection;
-import java.util.Collections;
-
/**
* Default dependency graph builder that detects current Maven version to
delegate to either Maven 2 or Maven 3 specific
* code.
@@ -52,6 +53,8 @@
{
protected PlexusContainer container;
+ private final CountDownLatch l = new CountDownLatch(1);
+
/**
* Builds a dependency graph.
*
@@ -81,6 +84,8 @@
{
try
{
+ l.await(30, TimeUnit.SECONDS);
+
String hint = isMaven31() ? "maven31" : isMaven2x() ? "maven2"
: "maven3";
DependencyGraphBuilder effectiveGraphBuilder =
@@ -90,6 +95,9 @@
return effectiveGraphBuilder.buildDependencyGraph( project,
filter, reactorProjects );
}
+ catch ( InterruptedException e ) {
+ throw new DependencyGraphBuilderException( e.getMessage(), e );
+ }
catch ( ComponentLookupException e )
{
throw new DependencyGraphBuilderException( e.getMessage(), e );
@@ -136,5 +144,7 @@
throws ContextException
{
container = (PlexusContainer) context.get(
PlexusConstants.PLEXUS_KEY );
+
+ l.countDown();
}
}
Stephen
On Tue, Feb 2, 2016 at 10:14 AM, Stephen Evanchik <[email protected]>
wrote:
> Hi Stuart,
>
> I think I found the problem. In maven-dependency-tree:{2.1,2.2} the
> Contextualizable.contextualize() method is invoked concurrently with the
> two buildDependencyGraph() methods. This is why the container field is
> null. I simply synchronized the methods against each other.
>
> Stephen
>
> On Mon, Feb 1, 2016 at 10:35 AM, Stephen Evanchik <[email protected]>
> wrote:
>
>> Hi Stuart,
>>
>> I was looking at some parallel builds we ran over the weekend and notice
>> some failures. I started to track them down and noticed that there was a
>> very strange error message which seems to indicate that maven-bundle-plugin
>> is not thread safe or has a fairly serious bug:
>>
>> [ERROR] Failed to execute goal
>> org.apache.felix:maven-bundle-plugin:3.0.1:manifest (bundle-manifest) on
>> project PROJECT_A: Execution bundle-manifest of goal
>> org.apache.felix:maven-bundle-plugin:3.0.1:manifest failed: A Jar can only
>> accept a valid file or directory: /full/build/path/PROJECT_B/target/classes
>>
>> There is a compile dependency between the projects (PROJECT_A depends on
>> PROJECT_B) but it looks like PROJECT_B was not yet built.
>>
>> Stephen
>>
>>
>>
>> On Thu, Jan 28, 2016 at 8:37 PM, Stephen Evanchik <[email protected]>
>> wrote:
>>
>>> Hi Stuart,
>>>
>>> A final update for tonight: I was able to reproduce the problem with
>>> maven-bundle-plugin:3.0.0 but not with maven-bundle-plugin:2.5.4 even
>>> though it looks like the DefaultDependencyGraphBuilder is in all three
>>> versions.
>>>
>>> Stephen
>>>
>>> On Thu, Jan 28, 2016 at 6:18 PM, Stephen Evanchik <[email protected]>
>>> wrote:
>>>
>>>> Hi Stuart,
>>>>
>>>> Thanks for confirming the behavior. I too noticed the redundant
>>>> packaging and execution stanzas. I suspect the original developer that
>>>> added this project didn't understand how the bundle extension worked.
>>>>
>>>> When I noticed the redundancy I updated all of the projects to use the
>>>> <execution>/<package> jar pattern hoping that the problem would go away.
>>>> Unfortunately, I saw an improvement but still have build failures.
>>>>
>>>> I'm not sure if this is a bug in Maven, maven-dependency-tree or,
>>>> maven-bundle-plugin
>>>>
>>>> Stephen
>>>>
>>>>
>>>> On Thu, Jan 28, 2016 at 5:43 PM, Stuart McCulloch <[email protected]>
>>>> wrote:
>>>>
>>>>> Hi Stephen,
>>>>>
>>>>> I've been able to recreate this with Maven 3.3.9 (by cloning your
>>>>> example
>>>>> project several times in a multi-module project) - will investigate
>>>>> further
>>>>> over the weekend.
>>>>>
>>>>> I did notice you're using the 'bundle' packaging, but you also have an
>>>>> explicit execution of the 'bundle' goal which should already be
>>>>> covered by
>>>>> the 'bundle' packaging's lifecycle - is there a reason for this extra
>>>>> execution?
>>>>> On 28 Jan 2016 18:00, "Stephen Evanchik" <[email protected]> wrote:
>>>>>
>>>>> > Hi Stuart,
>>>>> >
>>>>> > I am using Maven 3.3.3 please disregard my earlier email. I did see
>>>>> this
>>>>> > problem in 3.3.1 so there doesn't seem to be a difference. I am
>>>>> going to
>>>>> > try with 3.3.9 but I'm not going to hold out hope that it will make a
>>>>> > difference.
>>>>> >
>>>>> > I think the problem is that DefaultDependencyGraphBuilder does not
>>>>> get
>>>>> > initialized properly. This only occurs in parallel builds and is
>>>>> strongly
>>>>> > associated with relative timing. For example, if I issue a mvn -T2.0C
>>>>> > install I will see the failure >80% of the time but if I use mvn -X
>>>>> -T2.0C
>>>>> > I cannot reproduce the failure at all.
>>>>> >
>>>>> > I'm not sure embedding a pom is the right approach but here's a
>>>>> pastbin
>>>>> > link to a sample project that fails:
>>>>> >
>>>>> > https://paste.apache.org/dpeQ
>>>>> >
>>>>> >
>>>>> > Stephen
>>>>> >
>>>>> >
>>>>> > On Wed, Jan 27, 2016 at 4:49 AM, Stuart McCulloch <[email protected]
>>>>> >
>>>>> > wrote:
>>>>> >
>>>>> > > On Wednesday, 27 January 2016 at 06:03, Stephen Evanchik wrote:
>>>>> > > > Hi everyone,
>>>>> > > >
>>>>> > > > I'm having trouble tracking down an intermittent but frequent
>>>>> build
>>>>> > > failure
>>>>> > > > using the maven-bundle-plugin to wrap non-OSGi projects. I'm
>>>>> using
>>>>> > Maven
>>>>> > > > 3.3.1 and see the following NPE:
>>>>> > > >
>>>>> > > > Caused by: java.lang.NullPointerException
>>>>> > > > at
>>>>> > > >
>>>>> > >
>>>>> >
>>>>> org.apache.maven.shared.dependency.graph.internal.DefaultDependencyGraphBuilder.buildDependencyGraph(DefaultDependencyGraphBuilder.java:60)
>>>>> > > > at
>>>>> > > >
>>>>> > >
>>>>> >
>>>>> org.apache.felix.bundleplugin.BundlePlugin.buildDependencyGraph(BundlePlugin.java:334)
>>>>> > > > at
>>>>> > > >
>>>>> >
>>>>> org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:359)
>>>>> > > > at
>>>>> > > >
>>>>> > >
>>>>> >
>>>>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>>>> > > > ... 11 more
>>>>> > > >
>>>>> > > > whenever I invoke a parallel build (-T2.0C for example).
>>>>> > > >
>>>>> > > > There are many projects that will fail with this exception. I can
>>>>> > > provide a
>>>>> > > > fairly simple one if that makes sense.
>>>>> > > >
>>>>> > > >
>>>>> > >
>>>>> > > a reproducible test project is always helpful - have you tried a
>>>>> more
>>>>> > > recent version of Maven like 3.3.3 or 3.3.9 to see if that helps?
>>>>> > > > It looks like the 3.0.1 version is using
>>>>> maven-dependency-tree-2.1
>>>>> > whose
>>>>> > > > buildDependency method looks like:
>>>>> > > >
>>>>> > > > public DependencyNode buildDependencyGraph( MavenProject project,
>>>>> > > > ArtifactFilter filter )
>>>>> > > > throws DependencyGraphBuilderException
>>>>> > > > {
>>>>> > > > try
>>>>> > > > {
>>>>> > > > String hint = isMaven31() ? "maven31" : isMaven2x() ? "maven2"
>>>>> > > > : "maven3";
>>>>> > > > getLogger().debug( "building " + hint + " dependency graph for
>>>>> > > > " + project.getId() );
>>>>> > > >
>>>>> > > > DependencyGraphBuilder effectiveGraphBuilder =
>>>>> > > > (DependencyGraphBuilder) container.lookup(
>>>>> > > > DependencyGraphBuilder.class.getCanonicalName(), hint );
>>>>> > > >
>>>>> > > >
>>>>> > > > where the NPE is on:
>>>>> > > >
>>>>> > > > DependencyGraphBuilder effectiveGraphBuilder =
>>>>> > > > (DependencyGraphBuilder) container.lookup(
>>>>> > > > DependencyGraphBuilder.class.getCanonicalName(), hint );
>>>>> > > >
>>>>> > > >
>>>>> > > > I'm not sure why this could NPE as it seems like the
>>>>> > > > Contextualizable.contextualize() is called successfully for other
>>>>> > > projects
>>>>> > > > in the build.
>>>>> > > >
>>>>> > > > Any ideas on how to track this down? I can't enable debugging
>>>>> (mvn -X)
>>>>> > > > because that affects the timing just enough to avoid the issue.
>>>>> > > >
>>>>> > > > Thanks,
>>>>> > > > Stephen
>>>>> > >
>>>>> > >
>>>>> > >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Stephen Evanchik
>>>>> > http://stephen.evanchik.com
>>>>> >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Stephen Evanchik
>>>> http://stephen.evanchik.com
>>>>
>>>
>>>
>>>
>>> --
>>> Stephen Evanchik
>>> http://stephen.evanchik.com
>>>
>>
>>
>>
>> --
>> Stephen Evanchik
>> http://stephen.evanchik.com
>>
>
>
>
> --
> Stephen Evanchik
> http://stephen.evanchik.com
>
--
Stephen Evanchik
http://stephen.evanchik.com
