Build behavior differences between 3.2.5 and 3.3.9 with dependency shading
Hi, The Apache Flink project is using Maven for dependency management. We shade Google's Guava away (to org.apache.flink.shaded.com.google.commons) to avoid conflicts with user guava versions. Building Flink with Maven 3.2.5 will create a valid fat-jar without guava. However, Maven 3.3.9 (and other 3.3.x versions) are including guava in the com/google/commons namespace. Interestingly, doing only a "clean install" in the "flink-dist" package after a build of the parent pom results in a correct "flink-dist" fat jar. I'm wondering which behavior of Maven is correct 3.2 or 3.3 ? I have the feeling that 3.3 is behaving incorrectly because the dependency:tree of "flink-dist" does not contain Guava. Maybe a "clean install" on the parent pom with Maven 3.3 is not respecting the dependency-reduced poms created by the other modules? Regards, Robert
Re: Build behavior differences between 3.2.5 and 3.3.9 with dependency shading
Dependency reduced poms require mutation of the model after the build started. JvZ is investigating a different packaging type to resolve this... Workaround for now is to mark all the dependencies that are removed as true so that they are no longer transitive and that way the effective reactor Pom is the same from a transitive dependency PoV as the dependency reduced one that gets published On Thursday 10 December 2015, Robert Metzger wrote: > Hi, > > The Apache Flink project is using Maven for dependency management. We shade > Google's Guava away (to org.apache.flink.shaded.com.google.commons) to > avoid conflicts with user guava versions. > > Building Flink with Maven 3.2.5 will create a valid fat-jar without guava. > However, Maven 3.3.9 (and other 3.3.x versions) are including guava in the > com/google/commons namespace. > Interestingly, doing only a "clean install" in the "flink-dist" package > after a build of the parent pom results in a correct "flink-dist" fat jar. > > I'm wondering which behavior of Maven is correct 3.2 or 3.3 ? > I have the feeling that 3.3 is behaving incorrectly because the > dependency:tree of "flink-dist" does not contain Guava. > Maybe a "clean install" on the parent pom with Maven 3.3 is not respecting > the dependency-reduced poms created by the other modules? > > Regards, > Robert > -- Sent from my phone
Re: Build behavior differences between 3.2.5 and 3.3.9 with dependency shading
Okay, the true suggestion sounds interesting. I'll try that out. However, I'm still wondering why the behavior between 3.2 and 3.3 is different? Our CI system runs Maven 3.2, so the shading is done correctly there. If a project committer now adds a guava dependency and forgets the , we end up with guava in our final fat jar for users building with 3.3. Putting guava with optional=true into our dependency management is not an option because that would overwrite Hadoop's guava version (we shade hadoop and its guava in our project as well) On Thu, Dec 10, 2015 at 2:08 PM, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > Dependency reduced poms require mutation of the model after the build > started. JvZ is investigating a different packaging type to resolve this... > Workaround for now is to mark all the dependencies that are removed as > true so that they are no longer transitive and that > way the effective reactor Pom is the same from a transitive dependency PoV > as the dependency reduced one that gets published > > On Thursday 10 December 2015, Robert Metzger wrote: > > > Hi, > > > > The Apache Flink project is using Maven for dependency management. We > shade > > Google's Guava away (to org.apache.flink.shaded.com.google.commons) to > > avoid conflicts with user guava versions. > > > > Building Flink with Maven 3.2.5 will create a valid fat-jar without > guava. > > However, Maven 3.3.9 (and other 3.3.x versions) are including guava in > the > > com/google/commons namespace. > > Interestingly, doing only a "clean install" in the "flink-dist" package > > after a build of the parent pom results in a correct "flink-dist" fat > jar. > > > > I'm wondering which behavior of Maven is correct 3.2 or 3.3 ? > > I have the feeling that 3.3 is behaving incorrectly because the > > dependency:tree of "flink-dist" does not contain Guava. > > Maybe a "clean install" on the parent pom with Maven 3.3 is not > respecting > > the dependency-reduced poms created by the other modules? > > > > Regards, > > Robert > > > > > -- > Sent from my phone >
Re: Build behavior differences between 3.2.5 and 3.3.9 with dependency shading
You need to do this in any module that is producing a dependency reduced pom (and only in those modules) You can leave the version being inherited from dependencyManagement. I advise using the optional technique until Jason gets a new packaging for dependency reduced artifacts (or decides to agree with my suggestion of limited ability to modify the model after the reactor has been constructed) The 3.2 behaviour is currently viewed as a bug that has been abused by the shade plugin rather than a feature. 3.3 enforces the immutability of the model and I don't see that being rolled back On Thursday 10 December 2015, Robert Metzger wrote: > Okay, the true suggestion sounds interesting. I'll try that > out. > > However, I'm still wondering why the behavior between 3.2 and 3.3 is > different? > Our CI system runs Maven 3.2, so the shading is done correctly there. If a > project committer now adds a guava dependency and forgets the , > we end up with guava in our final fat jar for users building with 3.3. > Putting guava with optional=true into our dependency management is not an > option because that would overwrite Hadoop's guava version (we shade hadoop > and its guava in our project as well) > > > > On Thu, Dec 10, 2015 at 2:08 PM, Stephen Connolly < > stephen.alan.conno...@gmail.com > wrote: > > > Dependency reduced poms require mutation of the model after the build > > started. JvZ is investigating a different packaging type to resolve > this... > > Workaround for now is to mark all the dependencies that are removed as > > true so that they are no longer transitive and that > > way the effective reactor Pom is the same from a transitive dependency > PoV > > as the dependency reduced one that gets published > > > > On Thursday 10 December 2015, Robert Metzger > wrote: > > > > > Hi, > > > > > > The Apache Flink project is using Maven for dependency management. We > > shade > > > Google's Guava away (to org.apache.flink.shaded.com.google.commons) to > > > avoid conflicts with user guava versions. > > > > > > Building Flink with Maven 3.2.5 will create a valid fat-jar without > > guava. > > > However, Maven 3.3.9 (and other 3.3.x versions) are including guava in > > the > > > com/google/commons namespace. > > > Interestingly, doing only a "clean install" in the "flink-dist" package > > > after a build of the parent pom results in a correct "flink-dist" fat > > jar. > > > > > > I'm wondering which behavior of Maven is correct 3.2 or 3.3 ? > > > I have the feeling that 3.3 is behaving incorrectly because the > > > dependency:tree of "flink-dist" does not contain Guava. > > > Maybe a "clean install" on the parent pom with Maven 3.3 is not > > respecting > > > the dependency-reduced poms created by the other modules? > > > > > > Regards, > > > Robert > > > > > > > > > -- > > Sent from my phone > > > -- Sent from my phone
[Jigsaw] Get involved and stay informed about Maven and JDK9 Jigsaw
Hi, the Apache Maven team has decided to introduce a new subject prefix for the maven-dev mailinglist[1]. Messages prefixed with [Jigsaw] will be used to share information about Mavens progress on adopting JDK9 Jigsaw. Current status: I've been able to package the current codebase of Maven (core), package all our maven-plugins and run all their integration tests successfully with the following changes: - Use at least jdk9-b83 due to a classloading issue reported by us and solved together with Stuart McCulloch. - When using org.codehaus.plexus:plexus-component-metadata, use 1.7-SNAPSHOT This looks very promising, though there's still a small chance we've missed a specific jigsaw change which wasn't covered by an integration-test. Next step is to be able to compile with module-info. There's a jigsaw-ea branch of the plexus-compiler[4] which you can fork and test. The specs of -modulepath doesn't allow a direct reference to jars yet (we've reported JDK-8144665[5]) so the jars are still copied to a separate folder. I've started adding module-info files to a local fork of the Maven project, but I'm blocked by JDK-8144879[6] We have good contact with Oracle. Buildtools like Maven are very important to adopt these changes. The concepts are there, and there's still room for discussion on the details (like [5]). With the jigsaw-ea releases and the mentioned adjustments on your project configuration you can start testing and share your feedback. thanks, Robert ps. Also check the maven-jdeps-plugin[6] [1] http://maven.apache.org/mail-lists.html [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-September/004590.html [3] http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/948a1770620e [4] https://github.com/codehaus-plexus/plexus-compiler/tree/jigsaw-ea [5] https://bugs.openjdk.java.net/browse/JDK-8144665 [6] https://bugs.openjdk.java.net/browse/JDK-8144879 - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Some Maven Eclipse Questions
1. Noticed that even when I use the EMBEDDED maven inside m2e, it is creating a new JVM - different from the JVM running Eclipse - for maven install ( even for cleam ). Is this always the case ? ( Im trying to see if this happened because of some issues with my setup ) 2. I have a multip project. ( A maven pom.xml that has a bunch of s which in turn has a bunch of s and so on ). I Imported only the top level pom.xml into eclipse( on which I do the install.). Is there any way to automatically include all src/main/java folders from all nested modules into Eclipse project's Source Folder list ? 3. Is there any way to automatically include all dependencies ( jar files ) to the project 's Library ? ( So that I can use source completion/syntax checking etc. ) 4. I already had a maven project outside Eclipse, which had many levels of nested in its pom.xml files. I Imported only the top level pom into Eclipse as 'Existing Maven Projects'. Is this the right way to do this ? Im able to maven install from Eclipse. Thank you sony - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Some Maven Eclipse Questions
These questions are better suited for the m2e mailing list. https://dev.eclipse.org/mailman/listinfo/m2e-users /Anders (mobile) On Dec 11, 2015 06:16, "Sony Antony" wrote: > 1. Noticed that even when I use the EMBEDDED maven inside m2e, it is > creating a new JVM - different from the JVM running Eclipse - for > maven install ( even for cleam ). Is this always the case ? > ( Im trying to see if this happened because of some issues with my setup ) > > 2. I have a multip project. ( A maven pom.xml that has a bunch of > s which in turn has a bunch of s and so on ). I > Imported only the top level pom.xml into eclipse( on which I do the > install.). Is there any way to automatically include all src/main/java > folders from all nested modules into Eclipse project's Source Folder > list ? > > 3. Is there any way to automatically include all dependencies ( jar > files ) to the project 's Library ? ( So that I can use source > completion/syntax checking etc. ) > > 4. I already had a maven project outside Eclipse, which had many > levels of nested in its pom.xml files. I Imported only the > top level pom into Eclipse as 'Existing Maven Projects'. Is this the > right way to do this ? Im able to maven install from Eclipse. > > Thank you > sony > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > >
