PS. both Guice and the facade have multi-threaded concurrency tests, so once I
confirm this is what’s going on I’ll also look into why this wasn’t picked up
by the existing tests
On Tuesday, 2 February 2016 at 16:13, Stuart McCulloch wrote:
> Hi Stephen,
>
> Thanks for the update.
>
> The component shouldn’t be escaping to consumers before contextualize has
> finished, that sounds like a bug. Note that while it still looks like Plexus
> from an API perspective, ever since Maven 3 it’s actually Guice plus a thin
> facade to support the same API and behaviour as the old Plexus container. For
> example the contextualize step (along with other Plexus phases) is
> implemented by registering a custom ProvisionListener with Guice, which gets
> called as part of injection and is meant to finish before the object can be
> seen from the perspective of wherever it’s being injected.
>
> There have been some changes to Guice master in a related area, so I’ll try
> the latest development snapshot just in case that helps - I’ll also confirm
> what’s happening in the facade.
>
> --
> Cheers, Stuart
>
> On Tuesday, 2 February 2016 at 15:33, Stephen Evanchik wrote:
>
> > Hi Stuart,
> >
> > The synchronized let me see what was happening as far as ordering was
> > concerned. It appears that the contextualize() call races with calls to
> > buildDependencyGraph() and that that Container is not performing lifecycle
> > operations on the DefaultDependencyGraphBuilder object before consumers of
> > that object invoke methods.
> >
> > I think the maven-bundle-plugin needs to have @threadSafe = false since
> > it's not thread safe due its dependencies not being thread safe. I don't
> > know enough about Maven and Plexus to force some sort of explicit ordering
> > in the bean's lifecycle.
> >
> > However, this patch does seem to fix the underlying issue though it could
> > be subject to a dead lock if all of the build threads wait on the latch.
> > The real solution is for the container to properly follow concurrent
> > lifecycle rules.
> >
> > ===================================================================
> > ---
> > src/main/java/org/apache/maven/shared/dependency/graph/internal/DefaultDependencyGraphBuilder.java
> > (revision 1726911)
> > +++
> > src/main/java/org/apache/maven/shared/dependency/graph/internal/DefaultDependencyGraphBuilder.java
> > (working copy)
> > @@ -1,5 +1,9 @@
> > package org.apache.maven.shared.dependency.graph.internal;
> >
> > +import java.util.Collection;
> > +import java.util.concurrent.CountDownLatch;
> > +import java.util.concurrent.TimeUnit;
> > +
> > /*
> > * Licensed to the Apache Software Foundation (ASF) under one
> > * or more contributor license agreements. See the NOTICE file
> > @@ -33,9 +37,6 @@
> > import org.codehaus.plexus.logging.AbstractLogEnabled;
> > import
> > org.codehaus.plexus.personality.plexus.lifecycle.phase.Contextualizable;
> >
> > -import java.util.Collection;
> > -import java.util.Collections;
> > -
> > /**
> > * Default dependency graph builder that detects current Maven version to
> > delegate to either Maven 2 or Maven 3 specific
> > * code.
> > @@ -52,6 +53,8 @@
> > {
> > protected PlexusContainer container;
> >
> > + private final CountDownLatch l = new CountDownLatch(1);
> > +
> > /**
> > * Builds a dependency graph.
> > *
> > @@ -81,6 +84,8 @@
> > {
> > try
> > {
> > + l.await(30, TimeUnit.SECONDS);
> > +
> > String hint = isMaven31() ? "maven31" : isMaven2x() ? "maven2"
> > : "maven3";
> >
> > DependencyGraphBuilder effectiveGraphBuilder =
> > @@ -90,6 +95,9 @@
> >
> > return effectiveGraphBuilder.buildDependencyGraph( project,
> > filter, reactorProjects );
> > }
> > + catch ( InterruptedException e ) {
> > + throw new DependencyGraphBuilderException( e.getMessage(), e );
> > + }
> > catch ( ComponentLookupException e )
> > {
> > throw new DependencyGraphBuilderException( e.getMessage(), e );
> > @@ -136,5 +144,7 @@
> > throws ContextException
> > {
> > container = (PlexusContainer) context.get(
> > PlexusConstants.PLEXUS_KEY );
> > +
> > + l.countDown();
> > }
> > }
> >
> >
> >
> > Stephen
> >
> >
> > On Tue, Feb 2, 2016 at 10:14 AM, Stephen Evanchik <[email protected]
> > (mailto:[email protected])>
> > wrote:
> >
> > > Hi Stuart,
> > >
> > > I think I found the problem. In maven-dependency-tree:{2.1,2.2} the
> > > Contextualizable.contextualize() method is invoked concurrently with the
> > > two buildDependencyGraph() methods. This is why the container field is
> > > null. I simply synchronized the methods against each other.
> > >
> > > Stephen
> > >
> > > On Mon, Feb 1, 2016 at 10:35 AM, Stephen Evanchik <[email protected]
> > > (mailto:[email protected])>
> > > wrote:
> > >
> > > > Hi Stuart,
> > > >
> > > > I was looking at some parallel builds we ran over the weekend and notice
> > > > some failures. I started to track them down and noticed that there was a
> > > > very strange error message which seems to indicate that
> > > > maven-bundle-plugin
> > > > is not thread safe or has a fairly serious bug:
> > > >
> > > > [ERROR] Failed to execute goal
> > > > org.apache.felix:maven-bundle-plugin:3.0.1:manifest (bundle-manifest) on
> > > > project PROJECT_A: Execution bundle-manifest of goal
> > > > org.apache.felix:maven-bundle-plugin:3.0.1:manifest failed: A Jar can
> > > > only
> > > > accept a valid file or directory:
> > > > /full/build/path/PROJECT_B/target/classes
> > > >
> > > > There is a compile dependency between the projects (PROJECT_A depends on
> > > > PROJECT_B) but it looks like PROJECT_B was not yet built.
> > > >
> > > > Stephen
> > > >
> > > >
> > > >
> > > > On Thu, Jan 28, 2016 at 8:37 PM, Stephen Evanchik <[email protected]
> > > > (mailto:[email protected])>
> > > > wrote:
> > > >
> > > > > Hi Stuart,
> > > > >
> > > > > A final update for tonight: I was able to reproduce the problem with
> > > > > maven-bundle-plugin:3.0.0 but not with maven-bundle-plugin:2.5.4 even
> > > > > though it looks like the DefaultDependencyGraphBuilder is in all three
> > > > > versions.
> > > > >
> > > > > Stephen
> > > > >
> > > > > On Thu, Jan 28, 2016 at 6:18 PM, Stephen Evanchik <[email protected]
> > > > > (mailto:[email protected])>
> > > > > wrote:
> > > > >
> > > > > > Hi Stuart,
> > > > > >
> > > > > > Thanks for confirming the behavior. I too noticed the redundant
> > > > > > packaging and execution stanzas. I suspect the original developer
> > > > > > that
> > > > > > added this project didn't understand how the bundle extension
> > > > > > worked.
> > > > > >
> > > > > > When I noticed the redundancy I updated all of the projects to use
> > > > > > the
> > > > > > <execution>/<package> jar pattern hoping that the problem would go
> > > > > > away.
> > > > > > Unfortunately, I saw an improvement but still have build failures.
> > > > > >
> > > > > > I'm not sure if this is a bug in Maven, maven-dependency-tree or,
> > > > > > maven-bundle-plugin
> > > > > >
> > > > > > Stephen
> > > > > >
> > > > > >
> > > > > > On Thu, Jan 28, 2016 at 5:43 PM, Stuart McCulloch
> > > > > > <[email protected] (mailto:[email protected])>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Stephen,
> > > > > > >
> > > > > > > I've been able to recreate this with Maven 3.3.9 (by cloning your
> > > > > > > example
> > > > > > > project several times in a multi-module project) - will
> > > > > > > investigate
> > > > > > > further
> > > > > > > over the weekend.
> > > > > > >
> > > > > > > I did notice you're using the 'bundle' packaging, but you also
> > > > > > > have an
> > > > > > > explicit execution of the 'bundle' goal which should already be
> > > > > > > covered by
> > > > > > > the 'bundle' packaging's lifecycle - is there a reason for this
> > > > > > > extra
> > > > > > > execution?
> > > > > > > On 28 Jan 2016 18:00, "Stephen Evanchik" <[email protected]
> > > > > > > (mailto:[email protected])> wrote:
> > > > > > >
> > > > > > > > Hi Stuart,
> > > > > > > >
> > > > > > > > I am using Maven 3.3.3 please disregard my earlier email. I did
> > > > > > > > see
> > > > > > > this
> > > > > > > > problem in 3.3.1 so there doesn't seem to be a difference. I am
> > > > > > >
> > > > > > > going to
> > > > > > > > try with 3.3.9 but I'm not going to hold out hope that it will
> > > > > > > > make a
> > > > > > > > difference.
> > > > > > > >
> > > > > > > > I think the problem is that DefaultDependencyGraphBuilder does
> > > > > > > > not
> > > > > > > get
> > > > > > > > initialized properly. This only occurs in parallel builds and is
> > > > > > >
> > > > > > > strongly
> > > > > > > > associated with relative timing. For example, if I issue a mvn
> > > > > > > > -T2.0C
> > > > > > > > install I will see the failure >80% of the time but if I use
> > > > > > > > mvn -X
> > > > > > > >
> > > > > > >
> > > > > > > -T2.0C
> > > > > > > > I cannot reproduce the failure at all.
> > > > > > > >
> > > > > > > > I'm not sure embedding a pom is the right approach but here's a
> > > > > > > pastbin
> > > > > > > > link to a sample project that fails:
> > > > > > > >
> > > > > > > > https://paste.apache.org/dpeQ
> > > > > > > >
> > > > > > > >
> > > > > > > > Stephen
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Jan 27, 2016 at 4:49 AM, Stuart McCulloch
> > > > > > > > <[email protected] (mailto:[email protected])
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > On Wednesday, 27 January 2016 at 06:03, Stephen Evanchik
> > > > > > > > > wrote:
> > > > > > > > > > Hi everyone,
> > > > > > > > > >
> > > > > > > > > > I'm having trouble tracking down an intermittent but
> > > > > > > > > > frequent
> > > > > > > build
> > > > > > > > > failure
> > > > > > > > > > using the maven-bundle-plugin to wrap non-OSGi projects. I'm
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > using
> > > > > > > > Maven
> > > > > > > > > > 3.3.1 and see the following NPE:
> > > > > > > > > >
> > > > > > > > > > Caused by: java.lang.NullPointerException
> > > > > > > > > > at
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > org.apache.maven.shared.dependency.graph.internal.DefaultDependencyGraphBuilder.buildDependencyGraph(DefaultDependencyGraphBuilder.java:60)
> > > > > > > > > > at
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > org.apache.felix.bundleplugin.BundlePlugin.buildDependencyGraph(BundlePlugin.java:334)
> > > > > > > > > > at
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:359)
> > > > > > > > > > at
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
> > > > > > > > > > ... 11 more
> > > > > > > > > >
> > > > > > > > > > whenever I invoke a parallel build (-T2.0C for example).
> > > > > > > > > >
> > > > > > > > > > There are many projects that will fail with this exception.
> > > > > > > > > > I can
> > > > > > > > > provide a
> > > > > > > > > > fairly simple one if that makes sense.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > a reproducible test project is always helpful - have you
> > > > > > > > > tried a
> > > > > > > more
> > > > > > > > > recent version of Maven like 3.3.3 or 3.3.9 to see if that
> > > > > > > > > helps?
> > > > > > > > > > It looks like the 3.0.1 version is using
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > maven-dependency-tree-2.1
> > > > > > > > whose
> > > > > > > > > > buildDependency method looks like:
> > > > > > > > > >
> > > > > > > > > > public DependencyNode buildDependencyGraph( MavenProject
> > > > > > > > > > project,
> > > > > > > > > > ArtifactFilter filter )
> > > > > > > > > > throws DependencyGraphBuilderException
> > > > > > > > > > {
> > > > > > > > > > try
> > > > > > > > > > {
> > > > > > > > > > String hint = isMaven31() ? "maven31" : isMaven2x() ?
> > > > > > > > > > "maven2"
> > > > > > > > > > : "maven3";
> > > > > > > > > > getLogger().debug( "building " + hint + " dependency graph
> > > > > > > > > > for
> > > > > > > > > > " + project.getId() );
> > > > > > > > > >
> > > > > > > > > > DependencyGraphBuilder effectiveGraphBuilder =
> > > > > > > > > > (DependencyGraphBuilder) container.lookup(
> > > > > > > > > > DependencyGraphBuilder.class.getCanonicalName(), hint );
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > where the NPE is on:
> > > > > > > > > >
> > > > > > > > > > DependencyGraphBuilder effectiveGraphBuilder =
> > > > > > > > > > (DependencyGraphBuilder) container.lookup(
> > > > > > > > > > DependencyGraphBuilder.class.getCanonicalName(), hint );
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I'm not sure why this could NPE as it seems like the
> > > > > > > > > > Contextualizable.contextualize() is called successfully for
> > > > > > > > > > other
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > projects
> > > > > > > > > > in the build.
> > > > > > > > > >
> > > > > > > > > > Any ideas on how to track this down? I can't enable
> > > > > > > > > > debugging
> > > > > > > (mvn -X)
> > > > > > > > > > because that affects the timing just enough to avoid the
> > > > > > > > > > issue.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Stephen
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Stephen Evanchik
> > > > > > > > http://stephen.evanchik.com
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Stephen Evanchik
> > > > > > http://stephen.evanchik.com
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Stephen Evanchik
> > > > > http://stephen.evanchik.com
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Stephen Evanchik
> > > > http://stephen.evanchik.com
> > > >
> > >
> > >
> > >
> > >
> > > --
> > > Stephen Evanchik
> > > http://stephen.evanchik.com
> > >
> >
> >
> >
> >
> > --
> > Stephen Evanchik
> > http://stephen.evanchik.com
> >
> >
> >
>
>
