Repository: spark Updated Branches: refs/heads/master 3c24299b7 -> 37d95227a
[SPARK-17058][BUILD] Add maven snapshots-and-staging profile to build/test against staging artifacts ## What changes were proposed in this pull request? Adds a `snapshots-and-staging profile` so that RCs of projects like Hadoop and HBase can be used in developer-only build and test runs. There's a comment above the profile telling people not to use this in production. There's no attempt to do the same for SBT, as Ivy is different. ## How was this patch tested? Tested by building against the Hadoop 2.7.3 RC 1 JARs without the profile (and without any local copy of the 2.7.3 artifacts), the build failed ``` mvn install -DskipTests -Pyarn,hadoop-2.7,hive -Dhadoop.version=2.7.3 ... [INFO] ------------------------------------------------------------------------ [INFO] Building Spark Project Launcher 2.1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.pom [WARNING] The POM for org.apache.hadoop:hadoop-client:jar:2.7.3 is missing, no dependency information available Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.jar [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Spark Project Parent POM ........................... SUCCESS [ 4.482 s] [INFO] Spark Project Tags ................................. SUCCESS [ 17.402 s] [INFO] Spark Project Sketch ............................... SUCCESS [ 11.252 s] [INFO] Spark Project Networking ........................... SUCCESS [ 13.458 s] [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 9.043 s] [INFO] Spark Project Unsafe ............................... SUCCESS [ 16.027 s] [INFO] Spark Project Launcher ............................. FAILURE [ 1.653 s] [INFO] Spark Project Core ................................. SKIPPED ... ``` With the profile, the build completed ``` mvn install -DskipTests -Pyarn,hadoop-2.7,hive,snapshots-and-staging -Dhadoop.version=2.7.3 ``` Author: Steve Loughran <ste...@apache.org> Closes #14646 from steveloughran/stevel/SPARK-17058-support-asf-snapshots. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/37d95227 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/37d95227 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/37d95227 Branch: refs/heads/master Commit: 37d95227a21de602b939dae84943ba007f434513 Parents: 3c24299 Author: Steve Loughran <ste...@apache.org> Authored: Wed Nov 2 11:52:29 2016 -0700 Committer: Reynold Xin <r...@databricks.com> Committed: Wed Nov 2 11:52:29 2016 -0700 ---------------------------------------------------------------------- pom.xml | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/37d95227/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index aaf7cfa..04d2eaa 100644 --- a/pom.xml +++ b/pom.xml @@ -2694,6 +2694,54 @@ </profile> <!-- + This is a profile to enable the use of the ASF snapshot and staging repositories + during a build. It is useful when testing againt nightly or RC releases of dependencies. + It MUST NOT be used when building copies of Spark to use in production of for distribution, + --> + <profile> + <id>snapshots-and-staging</id> + <properties> + <!-- override point for ASF staging/snapshot repos --> + <asf.staging>https://repository.apache.org/content/groups/staging/</asf.staging> + <asf.snapshots>https://repository.apache.org/content/repositories/snapshots/</asf.snapshots> + </properties> + + <pluginRepositories> + <pluginRepository> + <id>ASF Staging</id> + <url>${asf.staging}</url> + </pluginRepository> + <pluginRepository> + <id>ASF Snapshots</id> + <url>${asf.snapshots}</url> + <snapshots> + <enabled>true</enabled> + </snapshots> + <releases> + <enabled>false</enabled> + </releases> + </pluginRepository> + + </pluginRepositories> + <repositories> + <repository> + <id>ASF Staging</id> + <url>${asf.staging}</url> + </repository> + <repository> + <id>ASF Snapshots</id> + <url>${asf.snapshots}</url> + <snapshots> + <enabled>true</enabled> + </snapshots> + <releases> + <enabled>false</enabled> + </releases> + </repository> + </repositories> + </profile> + + <!-- These empty profiles are available in some sub-modules. Declare them here so that maven does not complain when they're provided on the command line for a sub-module that does not have them. --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org