[
https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-24895:
---------------------------------
Description:
Spark 2.4.0 has maven build errors because artifacts uploaded to apache maven
repo has mismatched filenames:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce
(enforce-banned-dependencies) on project spark_2.4: Execution
enforce-banned-dependencies of goal
org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce failed:
org.apache.maven.shared.dependency.graph.DependencyGraphBuilderException: Could
not resolve following dependencies:
[org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-SNAPSHOT (compile),
org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT (compile),
org.apache.spark:spark-sketch_2.11:jar:2.4.0-SNAPSHOT (compile)]: Could not
resolve dependencies for project com.databricks:spark_2.4:pom:1: The following
artifacts could not be resolved:
org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-SNAPSHOT,
org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT,
org.apache.spark:spark-sketch_2.11:jar:2.4.0-SNAPSHOT: Could not find artifact
org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-20180723.232411-177 in
apache-snapshots ([https://repository.apache.org/snapshots/]) -> [Help 1]
{code}
If you check the artifact metadata you will see the pom and jar files are
2.4.0-20180723.232411-177 instead of 2.4.0-20180723.232410-177:
{code:xml}
<metadata modelVersion="1.1.0">
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib-local_2.11</artifactId>
<version>2.4.0-SNAPSHOT</version>
<versioning>
<snapshot>
<timestamp>20180723.232411</timestamp>
<buildNumber>177</buildNumber>
</snapshot>
<lastUpdated>20180723232411</lastUpdated>
<snapshotVersions>
<snapshotVersion>
<extension>jar</extension>
<value>2.4.0-20180723.232411-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<extension>pom</extension>
<value>2.4.0-20180723.232411-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<classifier>tests</classifier>
<extension>jar</extension>
<value>2.4.0-20180723.232410-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<classifier>sources</classifier>
<extension>jar</extension>
<value>2.4.0-20180723.232410-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<classifier>test-sources</classifier>
<extension>jar</extension>
<value>2.4.0-20180723.232410-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
</snapshotVersions>
</versioning>
</metadata>
{code}
This behavior is very similar to this issue:
https://issues.apache.org/jira/browse/MDEPLOY-221
Since 2.3.0 snapshots work with the same maven 3.3.9 version and maven deploy
2.8.2 plugin, it is highly possible that we introduced a new plugin that causes
this.
The most recent addition is the spot-bugs plugin, which is known to have
incompatibilities with other plugins:
[https://github.com/spotbugs/spotbugs-maven-plugin/issues/21]
We may want to try building without it to sanity check.
was:
Spark 2.4.0 has maven build errors because artifacts uploaded to apache maven
repo has mismatched filenames:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce
(enforce-banned-dependencies) on project spark_2.4: Execution
enforce-banned-dependencies of goal
org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce failed:
org.apache.maven.shared.dependency.graph.DependencyGraphBuilderException: Could
not resolve following dependencies:
[org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-SNAPSHOT (compile),
org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT (compile),
org.apache.spark:spark-sketch_2.11:jar:2.4.0-SNAPSHOT (compile)]: Could not
resolve dependencies for project com.databricks:spark_2.4:pom:1: The following
artifacts could not be resolved:
org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-SNAPSHOT,
org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT,
org.apache.spark:spark-sketch_2.11:jar:2.4.0-SNAPSHOT: Could not find artifact
org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-20180723.232411-177 in
apache-snapshots ([https://repository.apache.org/snapshots/]) -> [Help 1]
If you check the artifact metadata you will see the pom and jar files are
2.4.0-20180723.232411-177 instead of 2.4.0-20180723.232410-177:
{code:xml}
<metadata modelVersion="1.1.0">
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib-local_2.11</artifactId>
<version>2.4.0-SNAPSHOT</version>
<versioning>
<snapshot>
<timestamp>20180723.232411</timestamp>
<buildNumber>177</buildNumber>
</snapshot>
<lastUpdated>20180723232411</lastUpdated>
<snapshotVersions>
<snapshotVersion>
<extension>jar</extension>
<value>2.4.0-20180723.232411-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<extension>pom</extension>
<value>2.4.0-20180723.232411-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<classifier>tests</classifier>
<extension>jar</extension>
<value>2.4.0-20180723.232410-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<classifier>sources</classifier>
<extension>jar</extension>
<value>2.4.0-20180723.232410-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
<snapshotVersion>
<classifier>test-sources</classifier>
<extension>jar</extension>
<value>2.4.0-20180723.232410-177</value>
<updated>20180723232411</updated>
</snapshotVersion>
</snapshotVersions>
</versioning>
</metadata>
{code}
This behavior is very similar to this issue:
https://issues.apache.org/jira/browse/MDEPLOY-221
Since 2.3.0 snapshots work with the same maven 3.3.9 version and maven deploy
2.8.2 plugin, it is highly possible that we introduced a new plugin that causes
this.
The most recent addition is the spot-bugs plugin, which is known to have
incompatibilities with other plugins:
[https://github.com/spotbugs/spotbugs-maven-plugin/issues/21]
We may want to try building without it to sanity check.
> Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames
> ------------------------------------------------------------------------------
>
> Key: SPARK-24895
> URL: https://issues.apache.org/jira/browse/SPARK-24895
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.4.0
> Reporter: Eric Chang
> Priority: Major
>
> Spark 2.4.0 has maven build errors because artifacts uploaded to apache maven
> repo has mismatched filenames:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce
> (enforce-banned-dependencies) on project spark_2.4: Execution
> enforce-banned-dependencies of goal
> org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce failed:
> org.apache.maven.shared.dependency.graph.DependencyGraphBuilderException:
> Could not resolve following dependencies:
> [org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-SNAPSHOT (compile),
> org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT (compile),
> org.apache.spark:spark-sketch_2.11:jar:2.4.0-SNAPSHOT (compile)]: Could not
> resolve dependencies for project com.databricks:spark_2.4:pom:1: The
> following artifacts could not be resolved:
> org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-SNAPSHOT,
> org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT,
> org.apache.spark:spark-sketch_2.11:jar:2.4.0-SNAPSHOT: Could not find
> artifact
> org.apache.spark:spark-mllib-local_2.11:jar:2.4.0-20180723.232411-177 in
> apache-snapshots ([https://repository.apache.org/snapshots/]) -> [Help 1]
> {code}
>
> If you check the artifact metadata you will see the pom and jar files are
> 2.4.0-20180723.232411-177 instead of 2.4.0-20180723.232410-177:
> {code:xml}
> <metadata modelVersion="1.1.0">
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-mllib-local_2.11</artifactId>
> <version>2.4.0-SNAPSHOT</version>
> <versioning>
> <snapshot>
> <timestamp>20180723.232411</timestamp>
> <buildNumber>177</buildNumber>
> </snapshot>
> <lastUpdated>20180723232411</lastUpdated>
> <snapshotVersions>
> <snapshotVersion>
> <extension>jar</extension>
> <value>2.4.0-20180723.232411-177</value>
> <updated>20180723232411</updated>
> </snapshotVersion>
> <snapshotVersion>
> <extension>pom</extension>
> <value>2.4.0-20180723.232411-177</value>
> <updated>20180723232411</updated>
> </snapshotVersion>
> <snapshotVersion>
> <classifier>tests</classifier>
> <extension>jar</extension>
> <value>2.4.0-20180723.232410-177</value>
> <updated>20180723232411</updated>
> </snapshotVersion>
> <snapshotVersion>
> <classifier>sources</classifier>
> <extension>jar</extension>
> <value>2.4.0-20180723.232410-177</value>
> <updated>20180723232411</updated>
> </snapshotVersion>
> <snapshotVersion>
> <classifier>test-sources</classifier>
> <extension>jar</extension>
> <value>2.4.0-20180723.232410-177</value>
> <updated>20180723232411</updated>
> </snapshotVersion>
> </snapshotVersions>
> </versioning>
> </metadata>
> {code}
>
> This behavior is very similar to this issue:
> https://issues.apache.org/jira/browse/MDEPLOY-221
> Since 2.3.0 snapshots work with the same maven 3.3.9 version and maven deploy
> 2.8.2 plugin, it is highly possible that we introduced a new plugin that
> causes this.
> The most recent addition is the spot-bugs plugin, which is known to have
> incompatibilities with other plugins:
> [https://github.com/spotbugs/spotbugs-maven-plugin/issues/21]
> We may want to try building without it to sanity check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
