[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 The jetty problem has been dealt with; because of the shading declaration of jetty-util as provided (it isn't needed in spark any more), it wasn't getting into dist/jars even for those dependencies which did needed it. Fix: explicitly declaring it in the hadoop-cloud module and the hadoop cloud profile in the assembly module. The jetty-ajax dependency, which is the direct dependency of it from hadoop-azure, is also declared so that the spark build can keep in control of jetty versions everywhere. That one was getting through fine, because it wasn't being tagged as provided. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2286/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #89295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89295/testReport)** for PR 20923 at commit [`f6b9dc8`](https://github.com/apache/spark/commit/f6b9dc83d56c20d887166ddba7a7b876a57d65cb). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 I should add that the spark-shell doesn't bring up the Azure client, though it's happy with the rest, because of jetty-utils not making into dist/jars...I fear this is shading related. Here's the JAR I'm using to do diagnostics of CP setup there; it tries to bootstrap access from searching for & loading the JARs into actually reading and writing the data. Significantly more informative when things don't work. https://github.com/steveloughran/cloudstore/releases/tag/tag_2018_04_11_release --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20923 Ping @vanzin @gatorsmile , would like to hear your comments. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 Test failures are org.apache.spark.sql.sources.BucketedWriteWithoutHiveSupportSuite.; [SPARK-23894](https://issues.apache.org/jira/browse/SPARK-23894) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89058/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #89058 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89058/testReport)** for PR 20923 at commit [`52a8c28`](https://github.com/apache/spark/commit/52a8c28c564f669aa2cb2998b471f6085fb0742b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2097/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #89058 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89058/testReport)** for PR 20923 at commit [`52a8c28`](https://github.com/apache/spark/commit/52a8c28c564f669aa2cb2998b471f6085fb0742b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 bq. I think you should also update "test-dependencies.sh" to make the new deps file work. I did, but then things failed because the artifacts were only visible if you did a run with the -Psnapshots-and-staging profile, which jenkins doesn't do here -it's why test run [88953](https://github.com/apache/spark/pull/20923#issuecomment-379079140) failed. The 3.1.0 artifacts are now up on the mvnrepo, so I can turn that on. Because its out, I think it'd also be good to call the profile hadoop-3.1, to make clear that's the target release, and to avoid confusion if anyone ever creates a 3.2, 3.3, ... profile in future. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89038/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #89038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89038/testReport)** for PR 20923 at commit [`7c93d98`](https://github.com/apache/spark/commit/7c93d98aae8d74e0f0606cb03e68b0ac94bde177). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2079/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #89038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89038/testReport)** for PR 20923 at commit [`7c93d98`](https://github.com/apache/spark/commit/7c93d98aae8d74e0f0606cb03e68b0ac94bde177). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20923 I think you should also update "test-dependencies.sh" to make the new deps file work. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20923 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20923 Sorry @steveloughran for the late response. most of the deps in file is similar to "spark-deps-hadoop-2.7", so copy/rename it and run "test-dependencies.sh" will show you the diffs, based on the diffs to update the deps file will get a new hadoop3 deps file. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 Test failures are all in ` org.apache.spark.sql.sources.BucketedWriteWithoutHiveSupportSuite`. I don't see how these pom changes could have affected that --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88955/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88955 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88955/testReport)** for PR 20923 at commit [`7c93d98`](https://github.com/apache/spark/commit/7c93d98aae8d74e0f0606cb03e68b0ac94bde177). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88953/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88953 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88953/testReport)** for PR 20923 at commit [`43e3f31`](https://github.com/apache/spark/commit/43e3f31ba6d84f6ce62f7755eac201feaab35f45). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2014/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88955/testReport)** for PR 20923 at commit [`7c93d98`](https://github.com/apache/spark/commit/7c93d98aae8d74e0f0606cb03e68b0ac94bde177). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2013/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88953/testReport)** for PR 20923 at commit [`43e3f31`](https://github.com/apache/spark/commit/43e3f31ba6d84f6ce62f7755eac201feaab35f45). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 Failure is the test dependencies failing as the checker is trying to pull in hadoop-3.1.0 & its still in ASF staging ``` Performing Maven install for hadoop-3 Using `mvn` from path: /home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.3.9/bin/mvn [ERROR] Failed to execute goal on project spark-launcher_2.11: Could not resolve dependencies for project org.apache.spark:spark-launcher_2.11:jar:spark-440623: Could not find artifact org.apache.hadoop:hadoop-client:jar:3.1.0 in central (https://repo.maven.apache.org/maven2) -> ``` I'll turn off that check now, leaving the dependency list as is --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88941/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88941 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88941/testReport)** for PR 20923 at commit [`4184526`](https://github.com/apache/spark/commit/41845269f950a57968c473f90233d30b77a905dc). * This patch **fails build dependency tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2005/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88941/testReport)** for PR 20923 at commit [`4184526`](https://github.com/apache/spark/commit/41845269f950a57968c473f90233d30b77a905dc). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 I saw that, but given there isn't much in the way of a 2.8 profile though it was more of a wish list than a requirement. How do I go about creating it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20923 Hi @steveloughran , I think you missed this comment. You need to create a deps file under dev/deps and change the related script. > Also I think we need to create a related spark-deps-hadoop-3.x under dev/deps and make dependency check work for Hadoop 3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 sbt isn't going to test this profile, obviously. Ran both the mvn and sbt package targets qith profiles hadoop-3,hadoop-cloud,yarn,Psnapshots-and-staging --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88845/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88845 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88845/testReport)** for PR 20923 at commit [`58c04e9`](https://github.com/apache/spark/commit/58c04e92da4f394b4983e48981f32040e92600e0). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1918/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88845/testReport)** for PR 20923 at commit [`58c04e9`](https://github.com/apache/spark/commit/58c04e92da4f394b4983e48981f32040e92600e0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 @jerryshao the latest revision only has the POM changes, and that also excludes the build profile option to compile the hadoop-3 source trees It also switches the hadoop 3.1 version to being 3.1.0, which matches that of the current RC, and will be downloaded from the ASF staging repo if the `snapshots-and-staging` profile is enabled For example mvn install -Phadoop-3,hadoop-cloud,yarn,snapshots-and-staging Remember that if you do download the staged artifacts, then if new RCs are cut then you will need to purge the RCs from `~/.m2/repository` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20923 bq. I think we could separate cloud related stuffs to another PR, and fix only build related stuff in this PR OK --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20923 Also I think we need to create a related spark-deps-hadoop-3.x under dev/deps and make dependency check work for Hadoop 3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20923 I think we could separate cloud related stuffs to another PR, and fix only build related stuff in this PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1838/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88713/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88713/testReport)** for PR 20923 at commit [`016d690`](https://github.com/apache/spark/commit/016d69090691631343d37f9704d0f37a84ddf297). * This patch **fails RAT tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88713/testReport)** for PR 20923 at commit [`016d690`](https://github.com/apache/spark/commit/016d69090691631343d37f9704d0f37a84ddf297). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88670 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88670/testReport)** for PR 20923 at commit [`29e7324`](https://github.com/apache/spark/commit/29e73242cba9797ed24127b24bb0380c69a608d3). * This patch **fails RAT tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class BindingParquetOutputCommitter(` * `class PathOutputCommitProtocol(` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88670/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1808/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20923: [SPARK-23807][BUILD][WIP] Add Hadoop 3 profile with rele...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #88670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88670/testReport)** for PR 20923 at commit [`29e7324`](https://github.com/apache/spark/commit/29e73242cba9797ed24127b24bb0380c69a608d3). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
