Repository: spark
Updated Branches:
refs/heads/branch-1.3 502e1fd68 -> 5f1d1c0b8
[SPARK-8819] Fix build for maven 3.3.x
This is a workaround for MSHADE-148, which leads to an infinite loop when
building Spark with maven 3.3.x. This was originally caused by #6441, which
added a bunch of test dependencies on the spark-core test module. Recently, it
was revealed by #7193.
This patch adds a `-Prelease` profile. If present, it will set
`createDependencyReducedPom` to true. The consequences are:
- If you are releasing Spark with this profile, you are fine as long as you use
maven 3.2.x or before.
- If you are releasing Spark without this profile, you will run into SPARK-8781.
- If you are not releasing Spark but you are using this profile, you may run
into SPARK-8819.
- If you are not releasing Spark and you did not include this profile, you are
fine.
This is all documented in `pom.xml` and tested locally with both versions of
maven.
Author: Andrew Or <and...@databricks.com>
Closes #7219 from andrewor14/fix-maven-build and squashes the following commits:
1d37e87 [Andrew Or] Merge branch 'master' of github.com:apache/spark into
fix-maven-build
3574ae4 [Andrew Or] Review comments
f39199c [Andrew Or] Create a -Prelease profile that flags
`createDependencyReducedPom`
Conflicts:
dev/create-release/create-release.sh
pom.xml
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5f1d1c0b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/5f1d1c0b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/5f1d1c0b
Branch: refs/heads/branch-1.3
Commit: 5f1d1c0b826347e5518bdb079192f62f5cd4767b
Parents: 502e1fd
Author: Andrew Or <and...@databricks.com>
Authored: Mon Jul 6 19:22:30 2015 -0700
Committer: Andrew Or <and...@databricks.com>
Committed: Mon Jul 6 19:25:36 2015 -0700
----------------------------------------------------------------------
dev/create-release/create-release.sh | 4 ++--
pom.xml | 23 +++++++++++++++++++++++
2 files changed, 25 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/5f1d1c0b/dev/create-release/create-release.sh
----------------------------------------------------------------------
diff --git a/dev/create-release/create-release.sh
b/dev/create-release/create-release.sh
index 0403594..0979d5e 100755
--- a/dev/create-release/create-release.sh
+++ b/dev/create-release/create-release.sh
@@ -117,13 +117,13 @@ if [[ ! "$@" =~ --skip-publish ]]; then
echo "Created Nexus staging repository: $staged_repo_id"
build/mvn -DskipTests -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 \
- -Pyarn -Phive -Phive-thriftserver -Phadoop-2.2 -Pspark-ganglia-lgpl
-Pkinesis-asl \
+ -Pyarn -Phive -Phive-thriftserver -Phadoop-2.2 -Pspark-ganglia-lgpl
-Pkinesis-asl -Prelease-profile \
clean install
./dev/change-version-to-2.11.sh
build/mvn -DskipTests -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 \
- -Dscala-2.11 -Pyarn -Phive -Phadoop-2.2 -Pspark-ganglia-lgpl -Pkinesis-asl
\
+ -Dscala-2.11 -Pyarn -Phive -Phadoop-2.2 -Pspark-ganglia-lgpl -Pkinesis-asl
-Prelease-profile \
clean install
./dev/change-version-to-2.10.sh
http://git-wip-us.apache.org/repos/asf/spark/blob/5f1d1c0b/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index b0376ac..bcc2f51 100644
--- a/pom.xml
+++ b/pom.xml
@@ -156,6 +156,7 @@
<codehaus.jackson.version>1.8.8</codehaus.jackson.version>
<fasterxml.jackson.version>2.4.4</fasterxml.jackson.version>
<snappy.version>1.1.1.7</snappy.version>
+ <create.dependency.reduced.pom>false</create.dependency.reduced.pom>
<!--
Dependency scopes that can be overridden by enabling certain profiles.
These profiles are
@@ -1367,6 +1368,8 @@
<version>2.2</version>
<configuration>
<shadedArtifactAttached>false</shadedArtifactAttached>
+ <!-- Work around MSHADE-148. See SPARK-8819. -->
+
<createDependencyReducedPom>${create.dependency.reduced.pom}</createDependencyReducedPom>
<artifactSet>
<includes>
<!-- At a minimum we must include this to force effective pom
generation -->
@@ -1748,6 +1751,26 @@
</properties>
</profile>
+ <profile>
+ <!--
+ Use this profile only for making Spark releases. Note that due to
SPARK-8819,
+ you must use maven version 3.2.x or before to avoid running into
MSHADE-148.
+ -->
+ <id>release-profile</id>
+ <properties>
+ <!--
+ The maven shade plugin has a bug where enabling the
`createDependencyReducedPom`
+ property causes maven to go into an infinite loop (MSHADE-148).
This is only an
+ issue for the Spark build if the maven version is 3.3.x or newer
(SPARK-8819).
+
+ However, since disabling this property has the side effect of not
resolving
+ variables in the released pom files (SPARK-8781), we need to
enable this during
+ releases.
+ -->
+ <create.dependency.reduced.pom>true</create.dependency.reduced.pom>
+ </properties>
+ </profile>
+
<!--
These empty profiles are available in some sub-modules. Declare them
here so that
maven does not complain when they're provided on the command line for a
sub-module
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
