[GitHub] [maven] michael-o commented on pull request #641: [MNG-7374] Mutating RelocatedArtifact does not retain type
michael-o commented on pull request #641: URL: https://github.com/apache/maven/pull/641#issuecomment-1002970252 > no, I'm not trying to automagically retain the original type: I'm just trying to use the (hand-written) factory to remove code duplication Agree, suboptimal. > digging a little bit more, it requires a maven-resolver release, which makes the update harder from a release perspective I guess we can make that for 2.0.0 happen. Maybe 1.8.0, unsure. @cstamas -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (MNG-7380) Don't log non-threadsafe warning if only building a single module
[
https://issues.apache.org/jira/browse/MNG-7380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466787#comment-17466787
]
Michael Osipov commented on MNG-7380:
-
I wonder whether this applies to forked lifecycles also...
> Don't log non-threadsafe warning if only building a single module
> -
>
> Key: MNG-7380
> URL: https://issues.apache.org/jira/browse/MNG-7380
> Project: Maven
> Issue Type: Improvement
> Components: Plugins and Lifecycle
>Affects Versions: 3.8.4
>Reporter: Falko Modler
>Priority: Minor
>
> When building a _single_ module (via {{-f}}, {{-pl}} or simply cd'ing into
> it) with e.g. {{-T1C}} in {{.mvn/maven.config}} logs a warning if a
> non-threadsafe mojo is executed, e.g.:
> {noformat}
> [WARNING] *
> [WARNING] * Your build is requesting parallel execution, but project *
> [WARNING] * contains the following plugin(s) that have goals not marked *
> [WARNING] * as @threadSafe to support parallel building. *
> [WARNING] * While this /may/ work fine, please look for plugin updates*
> [WARNING] * and/or request plugins be made thread-safe. *
> [WARNING] * If reporting an issue, report it against the plugin in*
> [WARNING] * question, not against maven-core *
> [WARNING] *
> [WARNING] The following plugins are not marked @threadSafe in
> code-with-quarkus:
> [WARNING] io.quarkus.platform:quarkus-maven-plugin:2.6.1.Final
> [WARNING] Enable debug to see more precisely which goals are not marked
> @threadSafe.
> [WARNING] *
> {noformat}
> This is unnecessary noise because with only a single module there can never
> be a concurrency issue, no matter how many threads are enabled.
> Maven should check if it's building only one module before logging this
> warning.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [maven] mthmulders commented on a change in pull request #643: [MNG-5561] Plugin relocation loses configuration
mthmulders commented on a change in pull request #643:
URL: https://github.com/apache/maven/pull/643#discussion_r776665661
##
File path:
maven-resolver-provider/src/main/java/org/apache/maven/repository/internal/RelocatedArtifact.java
##
@@ -86,6 +86,39 @@ public String getVersion()
}
}
+@Override
+public Artifact setVersion( String version )
+{
+ String current = getVersion();
+ if ( current.equals( version ) || ( version == null &&
current.length() <= 0 ) )
Review comment:
How could the length of `current` be `< 0`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [maven] hboutemy commented on pull request #641: [MNG-7374] Mutating RelocatedArtifact does not retain type
hboutemy commented on pull request #641: URL: https://github.com/apache/maven/pull/641#issuecomment-1002967160 no, I'm not trying to automagically retain the original type: I'm just trying to use the (hand-written) factory to remove code duplication digging a little bit more, it requires a maven-resolver release, which makes the update harder from a release perspective -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (MRESOLVER-228) Improve the maven dependency resolution speed by a skip & reconcile approach
[
https://issues.apache.org/jira/browse/MRESOLVER-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466786#comment-17466786
]
Michael Osipov edited comment on MRESOLVER-228 at 12/30/21, 10:30 AM:
--
[~wecai], thanks! As far as I understand your explanation several approaches do
not play nice together. We must decide which combination is best in terms of
stability/reliability and *then* performance.
was (Author: michael-o):
[~wecai], thanks! As far as I understand your explanation several approaches do
not play nice together. We must decide which combination is best in terms of
stability/reliability and *then* performace.
> Improve the maven dependency resolution speed by a skip & reconcile approach
>
>
> Key: MRESOLVER-228
> URL: https://issues.apache.org/jira/browse/MRESOLVER-228
> Project: Maven Resolver
> Issue Type: Improvement
> Components: Resolver
>Affects Versions: 1.7.2
>Reporter: wei cai
>Priority: Major
> Fix For: 1.8.0
>
> Attachments: Screen Shot 2021-11-27 at 12.58.26 PM.png, Screen Shot
> 2021-11-27 at 12.58.59 PM.png, Screen Shot 2021-11-27 at 12.59.32 PM.png
>
>
> When comes to resolve the huge amount of dependencies of an enterprise level
> project, the maven resolver is very slow to resolve the dependency
> graph/tree. Take one of our app as example, it could take *10minutes+ and 16G
> memory* to print out the result of {*}mvn dependency:tree{*}.
> This is because there are many dependencies declared in the project, and some
> of the dependencies would introduce *600+* transitive dependencies, and
> exclusions are widely used to solve dependency conflicts.
> By checking the
> [code|https://github.com/apache/maven-resolver/blob/master/maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/collect/DefaultDependencyCollector.java#L500],
> we know the exclusion is also part of the cache key. This means when the
> exclusions up the tree differs, the cached resolution result for the same GAV
> won't be picked up and need s to be recalculated.
> !Screen Shot 2021-11-27 at 12.58.26 PM.png!
> From above figure, we know:
> * In 1st case, D will be resolved only once as there are no exclusions/same
> exclusions up the tree.
> * In 2nd case, the B and C have different exclusions and D needs to be
> recalculated, if D is a heavy dependency which introduce many transitive
> dependencies, all D and its children needs to be recalculated. Recalculating
> all of these nodes introduces 2 issues:
> ** Slow in resolving dependencies.
> ** Lots of DependencyNodes cached (all calculated/recalculated nodes would
> be cached) and will consume huge memory.
> To improve the speed of maven resolver's dependency resolution, I
> implemented a skip & reconcile approach. Here is the *skip* part.
> !Screen Shot 2021-11-27 at 12.58.59 PM.png!
> From above figure, the 1st R is resolved at depth 3, and the 2nd R is
> resolved again because the depth is at 2 which is lower, the 3rd R at depth 3
> and the 4th R at depth 4 are simply skipped as R is already resolved at depth
> 2. This is because the same node with deeper depth is most likely won't be
> picked up by maven as maven employs a "{*}nearest{*} transitive dependency in
> the tree depth and the *first* in resolution" strategy.
> The 3rd R and 4th R will have children set as zero and marked as skipped by
> the R at depth 2 in 2nd tree path.
>
> Here is the *reconcile* part:
> !Screen Shot 2021-11-27 at 12.59.32 PM.png!
> When there are dependency conflicts, some of the skipped nodes need to be
> reconciled.
> In above figure, there are 4 tree paths.
> * The D1 (D with version 1) in the 1st tree path is get resolved, children
> of E and R at depth 3 are resolved and cached.
> * In the 2nd tree path, when resolving E & R of H, we simply skip these 2
> nodes as they are in deeper depth (depth: 4) than the E & R in 1st tree path.
> * In the 3rd tree path, a R node with lower path is resolved, and a E node
> at depth 5 is skipped.
> * In the 4th path, a D2 (D with version 2) node is resolved, as the depth is
> lower than D1, so maven will pick D2, this means the E & R's children cached
> in tree depth 1 should be {*}discarded{*}.
> Thus we might need to reconcile the E & R nodes in 2nd, 3rd and 4th tree
> paths. Here only E in 2nd tree path needs to be reconciled. This is because:
> * R in 3rd tree path won't be picked up as there is already a R in 2nd tree
> path with a lower depth.
> * E in 3rd tree path won't be picked up as it is enough to reconcile the E
> in 2nd tree path as the E in 2nd tree path is deeper than E in 3rd tree path.
> Here is what we've updated in the maven-resolver logic:
> # Resolve dependencies
[jira] [Commented] (MRESOLVER-228) Improve the maven dependency resolution speed by a skip & reconcile approach
[
https://issues.apache.org/jira/browse/MRESOLVER-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466786#comment-17466786
]
Michael Osipov commented on MRESOLVER-228:
--
[~wecai], thanks! As far as I understand your explanation several approaches do
not play nice together. We must decide which combination is best in terms of
stability/reliability and *then* performace.
> Improve the maven dependency resolution speed by a skip & reconcile approach
>
>
> Key: MRESOLVER-228
> URL: https://issues.apache.org/jira/browse/MRESOLVER-228
> Project: Maven Resolver
> Issue Type: Improvement
> Components: Resolver
>Affects Versions: 1.7.2
>Reporter: wei cai
>Priority: Major
> Fix For: 1.8.0
>
> Attachments: Screen Shot 2021-11-27 at 12.58.26 PM.png, Screen Shot
> 2021-11-27 at 12.58.59 PM.png, Screen Shot 2021-11-27 at 12.59.32 PM.png
>
>
> When comes to resolve the huge amount of dependencies of an enterprise level
> project, the maven resolver is very slow to resolve the dependency
> graph/tree. Take one of our app as example, it could take *10minutes+ and 16G
> memory* to print out the result of {*}mvn dependency:tree{*}.
> This is because there are many dependencies declared in the project, and some
> of the dependencies would introduce *600+* transitive dependencies, and
> exclusions are widely used to solve dependency conflicts.
> By checking the
> [code|https://github.com/apache/maven-resolver/blob/master/maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/collect/DefaultDependencyCollector.java#L500],
> we know the exclusion is also part of the cache key. This means when the
> exclusions up the tree differs, the cached resolution result for the same GAV
> won't be picked up and need s to be recalculated.
> !Screen Shot 2021-11-27 at 12.58.26 PM.png!
> From above figure, we know:
> * In 1st case, D will be resolved only once as there are no exclusions/same
> exclusions up the tree.
> * In 2nd case, the B and C have different exclusions and D needs to be
> recalculated, if D is a heavy dependency which introduce many transitive
> dependencies, all D and its children needs to be recalculated. Recalculating
> all of these nodes introduces 2 issues:
> ** Slow in resolving dependencies.
> ** Lots of DependencyNodes cached (all calculated/recalculated nodes would
> be cached) and will consume huge memory.
> To improve the speed of maven resolver's dependency resolution, I
> implemented a skip & reconcile approach. Here is the *skip* part.
> !Screen Shot 2021-11-27 at 12.58.59 PM.png!
> From above figure, the 1st R is resolved at depth 3, and the 2nd R is
> resolved again because the depth is at 2 which is lower, the 3rd R at depth 3
> and the 4th R at depth 4 are simply skipped as R is already resolved at depth
> 2. This is because the same node with deeper depth is most likely won't be
> picked up by maven as maven employs a "{*}nearest{*} transitive dependency in
> the tree depth and the *first* in resolution" strategy.
> The 3rd R and 4th R will have children set as zero and marked as skipped by
> the R at depth 2 in 2nd tree path.
>
> Here is the *reconcile* part:
> !Screen Shot 2021-11-27 at 12.59.32 PM.png!
> When there are dependency conflicts, some of the skipped nodes need to be
> reconciled.
> In above figure, there are 4 tree paths.
> * The D1 (D with version 1) in the 1st tree path is get resolved, children
> of E and R at depth 3 are resolved and cached.
> * In the 2nd tree path, when resolving E & R of H, we simply skip these 2
> nodes as they are in deeper depth (depth: 4) than the E & R in 1st tree path.
> * In the 3rd tree path, a R node with lower path is resolved, and a E node
> at depth 5 is skipped.
> * In the 4th path, a D2 (D with version 2) node is resolved, as the depth is
> lower than D1, so maven will pick D2, this means the E & R's children cached
> in tree depth 1 should be {*}discarded{*}.
> Thus we might need to reconcile the E & R nodes in 2nd, 3rd and 4th tree
> paths. Here only E in 2nd tree path needs to be reconciled. This is because:
> * R in 3rd tree path won't be picked up as there is already a R in 2nd tree
> path with a lower depth.
> * E in 3rd tree path won't be picked up as it is enough to reconcile the E
> in 2nd tree path as the E in 2nd tree path is deeper than E in 3rd tree path.
> Here is what we've updated in the maven-resolver logic:
> # Resolve dependencies by leveraging a skip approach. The node in deeper
> depth will be skipped if a node with same GAV has been resolved with a lower
> depth.
> # Cloned the nodes in step 1.
> Use maven's ConflictResolver (Transformer) to transform the cloned nodes and
> find out the conflict winners
[GitHub] [maven] michael-o commented on pull request #641: [MNG-7374] Mutating RelocatedArtifact does not retain type
michael-o commented on pull request #641: URL: https://github.com/apache/maven/pull/641#issuecomment-1002965979 > @michael-o instead of copy/pasting the 3 methods, I think we could change "private Artifact newInstance" (from AbstractArtifact) to protected, then simply override this method While you are absolutely right, this is not the target of this ticket OR I need to broaden the ticket and solve this for all derved types. I absolutely do not like the fact that `AbstractArtifact` lacks abstraction and uses `DefaultArtifact` internally. Maybe: Mutating derived Artifact does not retain type -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [maven] hboutemy commented on pull request #641: [MNG-7374] Mutating RelocatedArtifact does not retain type
hboutemy commented on pull request #641: URL: https://github.com/apache/maven/pull/641#issuecomment-1002963399 @michael-o instead of copy/pasting the 3 methods, I think we could change "private Artifact newInstance" (from AbstractArtifact) to protected, then simply override this method -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [maven-integration-testing] mthmulders commented on a change in pull request #132: [MNG-5561] Plugin relocation loses configuration
mthmulders commented on a change in pull request #132:
URL:
https://github.com/apache/maven-integration-testing/pull/132#discussion_r776661962
##
File path:
core-it-suite/src/test/java/org/apache/maven/it/MavenITmng5561PluginRelocationLosesConfigurationTest.java
##
@@ -0,0 +1,63 @@
+package org.apache.maven.it;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.File;
+
+import org.apache.maven.it.util.ResourceExtractor;
+
+public class MavenITmng5561PluginRelocationLosesConfigurationTest
+extends AbstractMavenIntegrationTestCase
+{
+
+public MavenITmng5561PluginRelocationLosesConfigurationTest()
+{
+super( "[3.8.5,)" );
+}
+
+public void testit()
+throws Exception
+{
+File testDir =
+ResourceExtractor.simpleExtractResources( getClass(),
+
"/mng-5561-plugin-relocation-loses-configuration" );
+File oldPluginWithRelocationDir = new File( testDir,
"old-plugin-with-relocation" );
+File newPluginDir = new File( testDir, "new-plugin" );
+File projectDir = new File( testDir, "project" );
+
+Verifier verifier;
+
+verifier = newVerifier( oldPluginWithRelocationDir.getAbsolutePath() );
+verifier.executeGoal( "install" );
+verifier.resetStreams();
+verifier.verifyErrorFreeLog();
+
+verifier = newVerifier( newPluginDir.getAbsolutePath() );
+verifier.executeGoal( "install" );
+verifier.resetStreams();
+verifier.verifyErrorFreeLog();
+
+verifier = newVerifier( projectDir.getAbsolutePath() );
+verifier.executeGoal( "verify" );
+verifier.resetStreams();
+verifier.verifyErrorFreeLog();
+verifier.verifyTextInLog( "[WARNING] Hello from Maven!" );
Review comment:
Ah, of course.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [maven-integration-testing] michael-o commented on a change in pull request #132: [MNG-5561] Plugin relocation loses configuration
michael-o commented on a change in pull request #132:
URL:
https://github.com/apache/maven-integration-testing/pull/132#discussion_r776661361
##
File path:
core-it-suite/src/test/java/org/apache/maven/it/MavenITmng5561PluginRelocationLosesConfigurationTest.java
##
@@ -0,0 +1,63 @@
+package org.apache.maven.it;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.File;
+
+import org.apache.maven.it.util.ResourceExtractor;
+
+public class MavenITmng5561PluginRelocationLosesConfigurationTest
+extends AbstractMavenIntegrationTestCase
+{
+
+public MavenITmng5561PluginRelocationLosesConfigurationTest()
+{
+super( "[3.8.5,)" );
+}
+
+public void testit()
+throws Exception
+{
+File testDir =
+ResourceExtractor.simpleExtractResources( getClass(),
+
"/mng-5561-plugin-relocation-loses-configuration" );
+File oldPluginWithRelocationDir = new File( testDir,
"old-plugin-with-relocation" );
+File newPluginDir = new File( testDir, "new-plugin" );
+File projectDir = new File( testDir, "project" );
+
+Verifier verifier;
+
+verifier = newVerifier( oldPluginWithRelocationDir.getAbsolutePath() );
+verifier.executeGoal( "install" );
+verifier.resetStreams();
+verifier.verifyErrorFreeLog();
+
+verifier = newVerifier( newPluginDir.getAbsolutePath() );
+verifier.executeGoal( "install" );
+verifier.resetStreams();
+verifier.verifyErrorFreeLog();
+
+verifier = newVerifier( projectDir.getAbsolutePath() );
+verifier.executeGoal( "verify" );
+verifier.resetStreams();
+verifier.verifyErrorFreeLog();
+verifier.verifyTextInLog( "[WARNING] Hello from Maven!" );
Review comment:
Yes, that is correct, but IF the configuration is applied. Run the IT
without the patch applied against Maven, it will give you: "Hello World"
because the default value will kick in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (MSHARED-1009) Allow to provide Maven executable from workspace
[
https://issues.apache.org/jira/browse/MSHARED-1009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Slawomir Jaranowski reassigned MSHARED-1009:
Assignee: Slawomir Jaranowski
> Allow to provide Maven executable from workspace
>
>
> Key: MSHARED-1009
> URL: https://issues.apache.org/jira/browse/MSHARED-1009
> Project: Maven Shared Components
> Issue Type: New Feature
> Components: maven-invoker
>Reporter: Slawomir Jaranowski
>Assignee: Slawomir Jaranowski
>Priority: Major
> Fix For: maven-invoker-3.1.1
>
>
> Currently provided Maven executable is searched in Maven home location and
> absolute path is used from Maven distribution.
> Maven project can use {{wrapper}} and Maven executable, like {{mvnw}}, exists
> in project workspace.
> Firs project workspace should be checked, if provide executable exists in
> project should be used as relative path.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (MSHARED-1009) Allow to provide Maven executable from workspace
Slawomir Jaranowski created MSHARED-1009:
Summary: Allow to provide Maven executable from workspace
Key: MSHARED-1009
URL: https://issues.apache.org/jira/browse/MSHARED-1009
Project: Maven Shared Components
Issue Type: New Feature
Components: maven-invoker
Reporter: Slawomir Jaranowski
Fix For: maven-invoker-3.1.1
Currently provided Maven executable is searched in Maven home location and
absolute path is used from Maven distribution.
Maven project can use {{wrapper}} and Maven executable, like {{mvnw}}, exists
in project workspace.
Firs project workspace should be checked, if provide executable exists in
project should be used as relative path.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (MSHARED-577) Remove usage of M2_HOME environment variable
[ https://issues.apache.org/jira/browse/MSHARED-577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slawomir Jaranowski closed MSHARED-577. --- Resolution: Fixed > Remove usage of M2_HOME environment variable > > > Key: MSHARED-577 > URL: https://issues.apache.org/jira/browse/MSHARED-577 > Project: Maven Shared Components > Issue Type: Improvement > Components: maven-invoker >Reporter: Michael Simacek >Assignee: Slawomir Jaranowski >Priority: Major > Fix For: maven-invoker-3.1.1 > > > Maven 3.5.0 drops usage of M2_HOME variable in the launcher mvn script > MNG-5607. > Invoker should drop the usage as well to be consistent. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [maven-invoker] slawekjaranowski merged pull request #35: [MSHARED-577] Remove usage of M2_HOME environment variable
slawekjaranowski merged pull request #35: URL: https://github.com/apache/maven-invoker/pull/35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (MRESOLVER-228) Improve the maven dependency resolution speed by a skip & reconcile approach
[
https://issues.apache.org/jira/browse/MRESOLVER-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466764#comment-17466764
]
wei cai commented on MRESOLVER-228:
---
[~ibabiankou]
Thanks for your comments! Kudos for the excellent work on MRESOLVER-133 and
MRESOLVER-7.
I agree combining MRESOLVER-133, MRESOLVER-7 and MRESOLVER-228, there would be
huge performance improvements, however I don't think BFS + Skip can simply work
as it may introduce scope incompatibilities.
Before I submitted the PR of skip & reconcile approach in this ticket, I
actually noticed the MRESOLVER-133 and tried breadth-first approach but no
vain, here is what I've tried:
* DFS + Skip
1% of 500+ apps failed because some low level transitive dependencies are no
longer available in the dependency tree. This is because for a skipped node, I
need set children as empty and reuse the result of the node with same GAV but a
lower depth.
When there are version conflicts in the parent paths, a skipped node might
should not be skipped and need reconcile somehow.
* BFS + Skip
This time, I'm not seeing any failures because of missing some low level
transitive dependencies. Instead I witnessed the scope of certain dependency
nodes in the dependency tree were incorrect, especially for "provided" or
"compile" scopes. Check [this
code|https://github.com/apache/maven-resolver/blob/master/maven-resolver-util/src/main/java/org/eclipse/aether/util/graph/transformer/JavaScopeSelector.java#L87]
, when there are conflict items with different scopes, maven will pick
"compile" scope as long as there is a conflict item using "compile" scope.
* DFS + Skip + Reconcile
Then I come up with this solution. Tested with 2000+ apps in our company, all
dependency:tree and dependency:list result remains the same which proves it is
enough compatible. If go with BFS + Skip approach, as the scopes of
dependencies are no longer the same, we should do some reconcile, however it is
hard to implement such reconcile logic to make scope correct with BFS solution.
> Improve the maven dependency resolution speed by a skip & reconcile approach
>
>
> Key: MRESOLVER-228
> URL: https://issues.apache.org/jira/browse/MRESOLVER-228
> Project: Maven Resolver
> Issue Type: Improvement
> Components: Resolver
>Affects Versions: 1.7.2
>Reporter: wei cai
>Priority: Major
> Fix For: 1.8.0
>
> Attachments: Screen Shot 2021-11-27 at 12.58.26 PM.png, Screen Shot
> 2021-11-27 at 12.58.59 PM.png, Screen Shot 2021-11-27 at 12.59.32 PM.png
>
>
> When comes to resolve the huge amount of dependencies of an enterprise level
> project, the maven resolver is very slow to resolve the dependency
> graph/tree. Take one of our app as example, it could take *10minutes+ and 16G
> memory* to print out the result of {*}mvn dependency:tree{*}.
> This is because there are many dependencies declared in the project, and some
> of the dependencies would introduce *600+* transitive dependencies, and
> exclusions are widely used to solve dependency conflicts.
> By checking the
> [code|https://github.com/apache/maven-resolver/blob/master/maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/collect/DefaultDependencyCollector.java#L500],
> we know the exclusion is also part of the cache key. This means when the
> exclusions up the tree differs, the cached resolution result for the same GAV
> won't be picked up and need s to be recalculated.
> !Screen Shot 2021-11-27 at 12.58.26 PM.png!
> From above figure, we know:
> * In 1st case, D will be resolved only once as there are no exclusions/same
> exclusions up the tree.
> * In 2nd case, the B and C have different exclusions and D needs to be
> recalculated, if D is a heavy dependency which introduce many transitive
> dependencies, all D and its children needs to be recalculated. Recalculating
> all of these nodes introduces 2 issues:
> ** Slow in resolving dependencies.
> ** Lots of DependencyNodes cached (all calculated/recalculated nodes would
> be cached) and will consume huge memory.
> To improve the speed of maven resolver's dependency resolution, I
> implemented a skip & reconcile approach. Here is the *skip* part.
> !Screen Shot 2021-11-27 at 12.58.59 PM.png!
> From above figure, the 1st R is resolved at depth 3, and the 2nd R is
> resolved again because the depth is at 2 which is lower, the 3rd R at depth 3
> and the 4th R at depth 4 are simply skipped as R is already resolved at depth
> 2. This is because the same node with deeper depth is most likely won't be
> picked up by maven as maven employs a "{*}nearest{*} transitive dependency in
> the tree depth and the *first* in resolution" strategy.
> The 3rd R and 4th R will have children set as
[GitHub] [maven-mvnd] yxxcrtd edited a comment on issue #554: macOS Monterey doesn
yxxcrtd edited a comment on issue #554: URL: https://github.com/apache/maven-mvnd/issues/554#issuecomment-1002938817 I config env in .zshrc export PATH=$PATH:/Users/yxx/mvnd-0.7.1/bin then execute mvnd show: zsh: permission denied: mvnd -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [maven-mvnd] Fzoss commented on issue #549: How do i set other maven repository？
Fzoss commented on issue #549: URL: https://github.com/apache/maven-mvnd/issues/549#issuecomment-1002938999 look https://github.com/apache/maven-mvnd/issues/553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [maven-mvnd] yxxcrtd commented on issue #554: macOS Monterey doesn
yxxcrtd commented on issue #554: URL: https://github.com/apache/maven-mvnd/issues/554#issuecomment-1002938817 I config env in .zshrc export PATH=$PATH:/Users/yxx/mvnd-0.7.1/bin -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [maven-mvnd] yxxcrtd commented on issue #552: How to use mvnd in Jenkins?
yxxcrtd commented on issue #552: URL: https://github.com/apache/maven-mvnd/issues/552#issuecomment-1002932385 The same question ! I config Global Tool Configuration in jenkins, and set Maven's Name is **mvnd**, and MAVEN_HOME is **/home/mvnd-0.7.1/mvn**, then build without mvnd, hurry, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [maven-integration-testing] mthmulders commented on a change in pull request #132: [MNG-5561] Plugin relocation loses configuration
mthmulders commented on a change in pull request #132:
URL:
https://github.com/apache/maven-integration-testing/pull/132#discussion_r776487433
##
File path:
core-it-suite/src/test/java/org/apache/maven/it/MavenITmng5561PluginRelocationLosesConfigurationTest.java
##
@@ -0,0 +1,63 @@
+package org.apache.maven.it;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.File;
+
+import org.apache.maven.it.util.ResourceExtractor;
+
+public class MavenITmng5561PluginRelocationLosesConfigurationTest
+extends AbstractMavenIntegrationTestCase
+{
+
+public MavenITmng5561PluginRelocationLosesConfigurationTest()
+{
+super( "[3.8.5,)" );
+}
+
+public void testit()
+throws Exception
+{
+File testDir =
+ResourceExtractor.simpleExtractResources( getClass(),
+
"/mng-5561-plugin-relocation-loses-configuration" );
+File oldPluginWithRelocationDir = new File( testDir,
"old-plugin-with-relocation" );
+File newPluginDir = new File( testDir, "new-plugin" );
+File projectDir = new File( testDir, "project" );
+
+Verifier verifier;
+
+verifier = newVerifier( oldPluginWithRelocationDir.getAbsolutePath() );
+verifier.executeGoal( "install" );
+verifier.resetStreams();
+verifier.verifyErrorFreeLog();
+
+verifier = newVerifier( newPluginDir.getAbsolutePath() );
+verifier.executeGoal( "install" );
+verifier.resetStreams();
+verifier.verifyErrorFreeLog();
+
+verifier = newVerifier( projectDir.getAbsolutePath() );
+verifier.executeGoal( "verify" );
+verifier.resetStreams();
+verifier.verifyErrorFreeLog();
+verifier.verifyTextInLog( "[WARNING] Hello from Maven!" );
Review comment:
Call me blind, but wouldn't both the "old" and the "new" plugin give the
same output?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (MDEPLOY-250) add documentation on managing network issues
[
https://issues.apache.org/jira/browse/MDEPLOY-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466719#comment-17466719
]
Herve Boutemy commented on MDEPLOY-250:
---
documentation created in
https://github.com/apache/maven-deploy-plugin/commit/c34d0b5ede097734f9433f29a94466b3b8f7cd1c
published to
https://maven.apache.org/plugins-archives/maven-deploy-plugin-LATEST/examples/deploy-network-issues.html
> add documentation on managing network issues
>
>
> Key: MDEPLOY-250
> URL: https://issues.apache.org/jira/browse/MDEPLOY-250
> Project: Maven Deploy Plugin
> Issue Type: Task
> Components: deploy:deploy
>Affects Versions: 3.0.0-M1
>Reporter: Herve Boutemy
>Assignee: Herve Boutemy
>Priority: Major
> Fix For: 3.0.0-M2
>
>
> network issues are happening sometimes, then there are strategies that have
> been found to workaround, from simple retries, to 2-steps deploy through
> local staging, like done by plc4x:
> see in [https://github.com/apache/incubator-plc4x/blob/develop/Jenkinsfile]:
> {noformat}
> // We'll deploy to a relative directory so we can save
> // that and deploy in a later step on a different node
> sh 'mvn -P${JENKINS_PROFILE},development ${MVN_TEST_FAIL_IGNORE}
> ${JQASSISTANT_NEO4J_VERSION}
> -DaltDeploymentRepository=snapshot-repo::default::file:./local-snapshots-dir
> clean deploy'
> {noformat}
> followed by:
> {noformat}
> // Deploy the artifacts using the wagon-maven-plugin.
> sh 'mvn -f jenkins.pom -X -P deploy-snapshots wagon:upload'
> {noformat}
> sharing such strategy can be useful to many
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
